* -mm -> 2.6.13 merge status @ 2005-06-21 6:54 Andrew Morton 2005-06-21 10:22 ` -mm -> 2.6.13 merge status (fuse) Miklos Szeredi ` (18 more replies) 0 siblings, 19 replies; 620+ messages in thread From: Andrew Morton @ 2005-06-21 6:54 UTC (permalink / raw) To: linux-kernel This summarises my current thinking on various patches which are presently in -mm. I cover large things and small-but-controversial things. Anything which isn't covered here (and that's a lot of material) is probably a "will merge", unless it obviously isn't. (If you reply to this email it would be a good idea to alter the Subject: to reflect which feature you are discussing) git-ocfs The OCFS2 filesystem. OK by me, although I'm not sure it's had enough review. sparsemem OK by me for a merge. Need to poke arch maintainers first, check that they've looked at it sufficiently closely. vm-early-zone-reclaim Needs some convincing benchmark numbers to back it up. Otherwise OK. avoiding-mmap-fragmentation Tricky. Addresses vm area fragmentation issues due to recent optimisations to the free-area lookup code. Will merge. periodically-drain-non-local-pagesets Will merge pcibus_to_node and users Will merge CONFIG_HZ for x86 and ia64: changes default HZ to 250, make HZ Kconfigurable. Will merge (will switch default to 1000 Hz later if that seems necessary) dmi-*.patch Will merge. I have a comment "The below break x440". Maybe it got fixed. We'll doubtless hear if not. xen-*.patch These are little cleanups and abstractions which make a Xen merge easier. May as well merge them. CPU hotplug for x86 and x86_64 Not really useful on current hardware, but these provide infrastructure which some power management patches need, and it seems sensible to make the reference architecture support hotplug. Will merge. swsusp-on-SMP Will merge. cfq version 3 Not sure. Jens seems to be setting up a few git trees. On hold. RCUification of the key management code Don't know - dhowells seemed diffident last time we discussed this. timers-fixes-improvements.patch SMP speedups for the core timer code. It was bumpy, but this seems stable now. Will merge. kprobes-* Will merge rapidio-* Will merge. namespace*.patch Awaiting viro ack. xtensa architecture Is xtensa now, or will it be in the future a sufficiently popular architecture to justify the cost of having this code in the tree? Heaven knows. Will merge. dlm-*.patch: Red Hat distributed lock manager Hard. Right now it seems that no in-kernel projects will use this and only one out-of-kernel project will use it. Shelve the problem until after Kernel Summit, where some light may be shed. Opinions are sought... connector.patch Nice idea IMO, but there are still questions around the implementation. More dialogue needed ;) connector-add-a-fork-connector.patch OK, but needs connector. inotify There are still concerns about the userspace API and internal implementation details. More slogging needed. pcmcia-*.patch Makes the pcmcia layer generate hotplug events and deprecates cardmgr. Will merge. NUMA-aware slab allocator Seems stable now, but it needs some ifdef reduction work before merging, please. CPU scheduler Will merge some of these patches. We're still discussing which ones. perfctr Not yet, but getting closer. The PPC64 guys still need to sort out a few interface issues with Mikael. We might be able to fit this into 2.6.13 if people get a move on. cachefs This is a ton of code which knows rather a lot about pagecache internals. It allows the AFS client to cache file contents on a local blockdev. I don't think it's a justified addition for only AFS and I'd prefer to see it proven for NFS as well. Issues around add-page-becoming-writable-notification.patch need to be resolved. cachefs-for-nfs A recent addition. Needs review from NFS developers and considerably more testing. These things aren't looking likely for 2.6.13. kexec and kdump I guess we should merge these. I'm still concerned that the various device shutdown problems will mean that the success rate for crashing kernels is not high enough for kdump to be considered a success. In which case in six months time we'll hear rumours about vendors shipping wholly different crashdump implementations, which would be quite bad. But I think this has gone as far as it can go in -mm, so it's a bit of a punt. reiser4 Merge it, I guess. The patches still contain all the reiser4-specific namespace enhancements, only it is disabled, so it is effectively dead code. Maybe we should ask that it actually be removed? v9fs I'm not sure that this has a sufficiently high usefulness-to-maintenance-cost ratio. fuse This is useful, but there are, AFAIK, two issues: - We're still deadlocked over some permission-checking hacks in there - It has an NFS server implementation which only works if the to-be-served file happens to be in dcache. It has been said that a userspace NFS server can be used to get full NFS server functionality with FUSE. I think the half-assed kernel implementation should be done away with. execute-in-place Will merge. Have the embedded guys commented on the usefulness of this for execute-out-of-ROM? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton @ 2005-06-21 10:22 ` Miklos Szeredi 2005-06-21 12:39 ` Alan Cox ` (2 more replies) 2005-06-21 12:01 ` -mm -> 2.6.13 merge status Andrey Panin ` (17 subsequent siblings) 18 siblings, 3 replies; 620+ messages in thread From: Miklos Szeredi @ 2005-06-21 10:22 UTC (permalink / raw) To: akpm; +Cc: linux-kernel > fuse > > This is useful, but there are, AFAIK, two issues: > > - We're still deadlocked over some permission-checking hacks in there Oh, god. Let me try to explain this again: - This is a security issue with unprivileged mounts - Since no other filesystem currently offers secure unpivileged mounts in Linux, this is something "new" - Since it's something new, there's a big resistance to acceptance. I understand this, I only ask people, to please read Documentation/filesystems/fuse.txt, before arguing against it - IMO it's not a hack, and it's not something that can be solved otherwise (no, private namespaces are NOT a solution, they are mosty orthogonal to this). So I welcome constructive discussion. However bear in mind, that I definitely don't want to disable unprivileged mounts. For me that is _the_ most important feature of FUSE. > - It has an NFS server implementation which only works if the > to-be-served file happens to be in dcache. More preciesly it relies on icache. > It has been said that a userspace NFS server can be used to > get full NFS server functionality with FUSE. I think the > half-assed kernel implementation should be done away with. I won't shed many tears if you drop fuse-nfs-export.patch. It would at least give the userspace solution some boost. However the patch is pretty small, and despite it's flaws, I know it's used by a number of people. Thanks, Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 10:22 ` -mm -> 2.6.13 merge status (fuse) Miklos Szeredi @ 2005-06-21 12:39 ` Alan Cox 2005-06-21 13:13 ` Miklos Szeredi 2005-06-21 14:28 ` Pavel Machek 2005-06-21 15:26 ` Avuton Olrich 2 siblings, 1 reply; 620+ messages in thread From: Alan Cox @ 2005-06-21 12:39 UTC (permalink / raw) To: Miklos Szeredi; +Cc: akpm, Linux Kernel Mailing List On Maw, 2005-06-21 at 11:22, Miklos Szeredi wrote: > So I welcome constructive discussion. However bear in mind, that I > definitely don't want to disable unprivileged mounts. For me that is > _the_ most important feature of FUSE. If the choice was "merge FUSE without unpriv mounts for now" or "discard fuse completely" which is preferable. It seems to me (just IMHO) that it would be better to merge FUSE without that feature and then spend the time getting that feature right _in parallel_ with people using, breaking and reviewing FUSE a lot more. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 12:39 ` Alan Cox @ 2005-06-21 13:13 ` Miklos Szeredi 0 siblings, 0 replies; 620+ messages in thread From: Miklos Szeredi @ 2005-06-21 13:13 UTC (permalink / raw) To: alan; +Cc: akpm, linux-kernel > > So I welcome constructive discussion. However bear in mind, that I > > definitely don't want to disable unprivileged mounts. For me that is > > _the_ most important feature of FUSE. > > If the choice was "merge FUSE without unpriv mounts for now" or "discard > fuse completely" which is preferable. FUSE is doing fine outside mainline, so discard wouldn't be such a big setback. Including it without unpriv mounts would effectively fork FUSE into an out-of-tree and an in-tree version, which is sure to cause confusion. So yes, I'd prefer not merging to merging without unpriv mounts. But it's GPL, so obviously I don't have any legal control over it. > It seems to me (just IMHO) that it would be better to merge FUSE without > that feature and then spend the time getting that feature right _in > parallel_ with people using, breaking and reviewing FUSE a lot more. The security measure in question is actually very simple (10 lines or so). So it's not the implementation that people have problems with but the concept. The concept itself is hard to swallow, because it does something unexpected, but what it does is in fact very logical. That's why I ask people to read the documentation, think about it and _then_ argue. Up till now the discussion with Christoph Hellwig about this hasn't been on the level of rational arguments (and he's the only definite naysayer). Thanks, Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 10:22 ` -mm -> 2.6.13 merge status (fuse) Miklos Szeredi 2005-06-21 12:39 ` Alan Cox @ 2005-06-21 14:28 ` Pavel Machek 2005-06-21 14:51 ` John Stoffel 2005-06-21 15:13 ` Miklos Szeredi 2005-06-21 15:26 ` Avuton Olrich 2 siblings, 2 replies; 620+ messages in thread From: Pavel Machek @ 2005-06-21 14:28 UTC (permalink / raw) To: Miklos Szeredi; +Cc: akpm, linux-kernel Hi! > > This is useful, but there are, AFAIK, two issues: > > > > - We're still deadlocked over some permission-checking hacks in there > > Oh, god. Let me try to explain this again: > > - This is a security issue with unprivileged mounts Pretty please, just merge it without unpriviledged mounts. I see they are usefull, but they are too strange for now. If user tries mounting themselves, he gets -EPERM, and applies 10-liner from you. Does not look like "fork" or anything serious. Pavel -- 64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 14:28 ` Pavel Machek @ 2005-06-21 14:51 ` John Stoffel 2005-06-21 15:17 ` Miklos Szeredi 2005-06-21 15:13 ` Miklos Szeredi 1 sibling, 1 reply; 620+ messages in thread From: John Stoffel @ 2005-06-21 14:51 UTC (permalink / raw) To: Pavel Machek; +Cc: Miklos Szeredi, akpm, linux-kernel I'd like to see FUSE merged too, even without user mounts, if only to get more motivated to actually play with this and see how it works. John ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 14:51 ` John Stoffel @ 2005-06-21 15:17 ` Miklos Szeredi 0 siblings, 0 replies; 620+ messages in thread From: Miklos Szeredi @ 2005-06-21 15:17 UTC (permalink / raw) To: john; +Cc: pavel, miklos, akpm, linux-kernel > I'd like to see FUSE merged too, even without user mounts, if only to > get more motivated to actually play with this and see how it works. You can try it out now. Download from fuse.sf.net, ./configure; make; make install. It's not as if it was harder than compiling a kernel :) Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 14:28 ` Pavel Machek 2005-06-21 14:51 ` John Stoffel @ 2005-06-21 15:13 ` Miklos Szeredi 2005-06-21 22:06 ` Pavel Machek 1 sibling, 1 reply; 620+ messages in thread From: Miklos Szeredi @ 2005-06-21 15:13 UTC (permalink / raw) To: pavel; +Cc: akpm, linux-kernel > > > This is useful, but there are, AFAIK, two issues: > > > > > > - We're still deadlocked over some permission-checking hacks in there > > > > Oh, god. Let me try to explain this again: > > > > - This is a security issue with unprivileged mounts > > Pretty please, just merge it without unpriviledged mounts. I see > they are usefull, but they are too strange for now. An emotional argument again. What's "strange" about it? You have a choice of: 1) believe me that the current solution is fine 2) get down and try to understand the damn thing, and then come up with technical arguments for/against it I know that 2) takes time and energy, and not a lot of people are interested enough to go through it, but why on earth do you think it will _ever_ be easier than now. Thanks, Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 15:13 ` Miklos Szeredi @ 2005-06-21 22:06 ` Pavel Machek 2005-06-22 6:20 ` Miklos Szeredi ` (2 more replies) 0 siblings, 3 replies; 620+ messages in thread From: Pavel Machek @ 2005-06-21 22:06 UTC (permalink / raw) To: Miklos Szeredi; +Cc: akpm, linux-kernel Hi! > > > > This is useful, but there are, AFAIK, two issues: > > > > > > > > - We're still deadlocked over some permission-checking hacks in there > > > > > > Oh, god. Let me try to explain this again: > > > > > > - This is a security issue with unprivileged mounts > > > > Pretty please, just merge it without unpriviledged mounts. I see > > they are usefull, but they are too strange for now. > > An emotional argument again. What's "strange" about it? Not so emotional argument... System where users can mount their own filesystems should not be called "Unix" any more. It introduces new mechanism, similar to ptrace. It restricts root in ways not seen before. How is updatedb/locate supposed to work on system with this? How is it going to interact with backup tools? Add this to your A): "by tricking some interpretter to think script is setuid". > You have a choice of: 1) believe me that the current solution is > fine > 2) get down and try to understand the damn thing, and then come up > with technical arguments for/against it Argument is "it is **** ugly". Your fuse.txt explains why it is not security hole. It does not explain why your interface is the best possible, and what alternative ways of "not security hole" exist. Pavel -- teflon -- maybe it is a trademark, but it should not be. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 22:06 ` Pavel Machek @ 2005-06-22 6:20 ` Miklos Szeredi 2005-06-22 6:39 ` Andrew Morton 2005-06-22 8:53 ` Pavel Machek 2005-06-22 12:57 ` Matthias Urlichs 2005-06-22 14:43 ` Eric Van Hensbergen 2 siblings, 2 replies; 620+ messages in thread From: Miklos Szeredi @ 2005-06-22 6:20 UTC (permalink / raw) To: pavel; +Cc: akpm, linux-kernel > Not so emotional argument... > > System where users can mount their own filesystems should not be > called "Unix" any more. It's not. It's "Linux". And anyway, sysadmin may set whatever owner/group/permissions on '/dev/fuse' to disallow or selectively allow users to be able to mount FUSE filesystems. > It introduces new mechanism, similar to ptrace. It restricts root in > ways not seen before. Not true. Root squash in NFS has similar effect. > How is updatedb/locate supposed to work on system with this? How is > it going to interact with backup tools? I assure you, that it will cause no problems whatever. These programs are able to gracefully handle errors. > Add this to your A): "by tricking some interpretter to think script is > setuid". How would you do that? > > You have a choice of: 1) believe me that the current solution is > > fine > > > 2) get down and try to understand the damn thing, and then come up > > with technical arguments for/against it > > Argument is "it is **** ugly". Yeah, that's your opinion. Mine is that it's f****** beautiful ;). There are plenty of ugly things in Unix/Linux that you've become so accustomed to, that they no longer seem ugly. Think about the sticky bit on directories for example. That one was breaking assumptions left and right when it got introduced, but people came to accept it, because it's useful. > Your fuse.txt explains why it is not security hole. It does not > explain why your interface is the best possible, and what alternative > ways of "not security hole" exist. That's because I don't see any alternative. The "preventing user from tracing root" and "preventing access to user's filesysem by root" must come together. There's doesn't seem to be any other way. BTW, thanks for reading through fuse.txt :) Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 6:20 ` Miklos Szeredi @ 2005-06-22 6:39 ` Andrew Morton 2005-06-22 7:16 ` Miklos Szeredi 2005-06-24 16:00 ` Gábor Lénárt 2005-06-22 8:53 ` Pavel Machek 1 sibling, 2 replies; 620+ messages in thread From: Andrew Morton @ 2005-06-22 6:39 UTC (permalink / raw) To: Miklos Szeredi; +Cc: pavel, linux-kernel Miklos Szeredi <miklos@szeredi.hu> wrote: > > > Not so emotional argument... > > > > System where users can mount their own filesystems should not be > > called "Unix" any more. > > It's not. It's "Linux". It would be helpful if we could have a brief description of the feature which you're discussing here. We discussed this a couple of months back, but I've forgotten most of it and it was off-list I think. Doing `grep uid fs/fuse/*.c' gets us to the implementation, yes? Which parts are controversial? How _should_ we implement unprivileged mounts, if not this way? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 6:39 ` Andrew Morton @ 2005-06-22 7:16 ` Miklos Szeredi 2005-06-22 7:49 ` Andrew Morton 2005-06-22 17:01 ` Theodore Ts'o 2005-06-24 16:00 ` Gábor Lénárt 1 sibling, 2 replies; 620+ messages in thread From: Miklos Szeredi @ 2005-06-22 7:16 UTC (permalink / raw) To: akpm; +Cc: miklos, pavel, linux-kernel > It would be helpful if we could have a brief description of the feature > which you're discussing here. We discussed this a couple of months back, > but I've forgotten most of it and it was off-list I think. > > Doing `grep uid fs/fuse/*.c' gets us to the implementation, yes? > > Which parts are controversial? The controversial part is fuse_allow_task() called from fuse_permission() and fuse_revalidate() (fs/fuse/dir.c). This function (as explained by the header comment) disallows access to the filesystem for any task, which the filesystem owner (the user who did the mount) is not allowed to ptrace. The rationale is that accessing the filesystem gives the filesystem implementor ptrace like capabilities (detailed in Documentation/filesystems/fuse.txt) It is controversial, because obviously root owned tasks are not ptrace-able by the user, and so these tasks will be denied access to the user mounted filesystem (-EACCESS is returned on stat() or any other file operation). However nobody raised _any_ concrete technical problem associated with this, and the 4 years of widespread use didn't turn up any either. So IMO it's "ugly" only in people's heads and not in reality. > How _should_ we implement unprivileged mounts, if not this way? A lot of the earlier discussion centered around making private namespaces usable, and so segregating user mounts. However because suid/sgid programs can still gain elevated privileges in a private namespace the above check is still necessary. So private namespaces are alone not a solution. I also suggested (and implemented a patch) which hides the user mounts, based on the above check. This would mean, that root and other users would simply not see the user's mount instead of getting an error. This was also received with very little enthusiasm. A further suggested alternative was to confine user mounts to a single directory (e.g. '/mnt/usermounts'). This would solve the problem if every daemon and suid/sgid program was tought to not enter this directory. This is quite impractical. Besides this is a very inflexible solution. I'm still waiting for a practical alternative. I don't consider practical the opinion held by quite a few people: "let's integrate FUSE now, without unpriv mounts, maybe we'll find a better solution later" I consider it highly unlikely, that any time will be better than now. Thanks, Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 7:16 ` Miklos Szeredi @ 2005-06-22 7:49 ` Andrew Morton 2005-06-22 9:07 ` Miklos Szeredi 2005-06-22 17:01 ` Theodore Ts'o 1 sibling, 1 reply; 620+ messages in thread From: Andrew Morton @ 2005-06-22 7:49 UTC (permalink / raw) To: Miklos Szeredi; +Cc: miklos, pavel, linux-kernel Miklos Szeredi <miklos@szeredi.hu> wrote: > > > It would be helpful if we could have a brief description of the feature > > which you're discussing here. We discussed this a couple of months back, > > but I've forgotten most of it and it was off-list I think. > > > > Doing `grep uid fs/fuse/*.c' gets us to the implementation, yes? > > > > Which parts are controversial? > > The controversial part is fuse_allow_task() called from > fuse_permission() and fuse_revalidate() (fs/fuse/dir.c). > > This function (as explained by the header comment) disallows access to > the filesystem for any task, which the filesystem owner (the user who > did the mount) is not allowed to ptrace. That's fairly weird. Overloading ptraceability is awkward, but also the *direction* is wrong. It's saying "if I can ptrace you, you can read my data". I'd have expected to see "if you can ptrace me, you can access my data". > The rationale is that accessing the filesystem gives the filesystem > implementor ptrace like capabilities (detailed in > Documentation/filesystems/fuse.txt) hrm. Makes some sense. > It is controversial, because obviously root owned tasks are not > ptrace-able by the user, and so these tasks will be denied access to > the user mounted filesystem (-EACCESS is returned on stat() or any > other file operation). > > However nobody raised _any_ concrete technical problem associated with > this, and the 4 years of widespread use didn't turn up any either. So > IMO it's "ugly" only in people's heads and not in reality. It's ugly ;) But the problem you're addressing here largely revolves around the fact that the filesystem implementation is a userspace process which is potentially owned by a different user. So you need to prevent the mount owner from peeking at the fs user's activity. That problem is unique to FUSE and so a solution within fuse is appropriate. This security feature doesn't sounds terribly important to me. So the fuse server can find out what files I'm looking at. But I've already deliberately given the fuse server the ability to ptrace my process? Can we enhance private namespaces so they can squash setuid/setgid? If so, is that adequate? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 7:49 ` Andrew Morton @ 2005-06-22 9:07 ` Miklos Szeredi 2005-06-22 9:12 ` Andrew Morton 2005-06-22 9:17 ` Pavel Machek 0 siblings, 2 replies; 620+ messages in thread From: Miklos Szeredi @ 2005-06-22 9:07 UTC (permalink / raw) To: akpm; +Cc: pavel, linux-kernel > But the problem you're addressing here largely revolves around the fact that > the filesystem implementation is a userspace process which is potentially > owned by a different user. So you need to prevent the mount owner from > peeking at the fs user's activity. That problem is unique to FUSE and so a > solution within fuse is appropriate. It's in fact not so unique to FUSE. It would equally well apply to v9fs or even samba, since both want to allow unprvileged mounts, and synthetic (or at least user-controlled) file serving. > This security feature doesn't sounds terribly important to me. So the fuse > server can find out what files I'm looking at. But I've already > deliberately given the fuse server the ability to ptrace my process? If it's deliberate, than OK. However with suid/sgid, this is not a deliberate action of the user under whose capabilities the process runs. Neither in the case, when it's a daemon doing some recursive directory traversal. And it's not just peeking at the filesystem access patterns. A much more dangerous aspect is controlling _when_ an operation returns (e.g. delaying it forever), and _what_ it returns (e.g. huge files/directories). Of course, this is only truly relevant for systems with untrusted users. But I do want to make FUSE work securely in those cases too. For the single user system, the sysadmin can turn this feature off, and be done with it. > Can we enhance private namespaces so they can squash setuid/setgid? If so, > is that adequate? We could. But that would again be overly restrictive. The goal is to make the use of FUSE filesystems for users as simple as possible. If the user has to manage multiple namespaces, each with it's own restrictions, it's becoming a very un-user-friendly environment. Thanks, Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 9:07 ` Miklos Szeredi @ 2005-06-22 9:12 ` Andrew Morton 2005-06-22 9:20 ` Miklos Szeredi 2005-06-22 9:17 ` Pavel Machek 1 sibling, 1 reply; 620+ messages in thread From: Andrew Morton @ 2005-06-22 9:12 UTC (permalink / raw) To: Miklos Szeredi; +Cc: pavel, linux-kernel Miklos Szeredi <miklos@szeredi.hu> wrote: > > > Can we enhance private namespaces so they can squash setuid/setgid? If so, > > is that adequate? > > We could. But that would again be overly restrictive. The goal is to > make the use of FUSE filesystems for users as simple as possible. If > the user has to manage multiple namespaces, each with it's own > restrictions, it's becoming a very un-user-friendly environment. I'd have thought that it would be possible to offer the same user interface as you currently have with private namespaces. Hide any complexity in the userspace tools? Where's the problem? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 9:12 ` Andrew Morton @ 2005-06-22 9:20 ` Miklos Szeredi 2005-06-22 9:44 ` Andrew Morton 0 siblings, 1 reply; 620+ messages in thread From: Miklos Szeredi @ 2005-06-22 9:20 UTC (permalink / raw) To: akpm; +Cc: miklos, pavel, linux-kernel > > We could. But that would again be overly restrictive. The goal is to > > make the use of FUSE filesystems for users as simple as possible. If > > the user has to manage multiple namespaces, each with it's own > > restrictions, it's becoming a very un-user-friendly environment. > > I'd have thought that it would be possible to offer the same user interface > as you currently have with private namespaces. Hide any complexity in the > userspace tools? Where's the problem? Sorry, I don't get it. You mean implement the permission checking in the userspace filesystem? That doesn't work, since it's running with the privileges of the filesystem owner, and as such obviously cannot be trusted with checking for access by other users. Thanks, Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 9:20 ` Miklos Szeredi @ 2005-06-22 9:44 ` Andrew Morton 2005-06-22 9:58 ` Miklos Szeredi 0 siblings, 1 reply; 620+ messages in thread From: Andrew Morton @ 2005-06-22 9:44 UTC (permalink / raw) To: Miklos Szeredi; +Cc: miklos, pavel, linux-kernel Miklos Szeredi <miklos@szeredi.hu> wrote: > > > > We could. But that would again be overly restrictive. The goal is to > > > make the use of FUSE filesystems for users as simple as possible. If > > > the user has to manage multiple namespaces, each with it's own > > > restrictions, it's becoming a very un-user-friendly environment. > > > > I'd have thought that it would be possible to offer the same user interface > > as you currently have with private namespaces. Hide any complexity in the > > userspace tools? Where's the problem? > > Sorry, I don't get it. I'm asking you to expand on what the problems would be if we were to enhance the namespace code as suggested. What's the "very un-user-friendly environment", and why cannot it be made more friendly with appropriate support tools? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 9:44 ` Andrew Morton @ 2005-06-22 9:58 ` Miklos Szeredi 2005-06-22 10:19 ` Andrew Morton 2005-06-22 12:23 ` Eric Van Hensbergen 0 siblings, 2 replies; 620+ messages in thread From: Miklos Szeredi @ 2005-06-22 9:58 UTC (permalink / raw) To: akpm; +Cc: pavel, linux-kernel > > > > We could. But that would again be overly restrictive. The goal is to > > > > make the use of FUSE filesystems for users as simple as possible. If > > > > the user has to manage multiple namespaces, each with it's own > > > > restrictions, it's becoming a very un-user-friendly environment. > > > > > > I'd have thought that it would be possible to offer the same user interface > > > as you currently have with private namespaces. Hide any complexity in the > > > userspace tools? Where's the problem? > > > > Sorry, I don't get it. > > I'm asking you to expand on what the problems would be if we were to > enhance the namespace code as suggested. OK, what I was thinking, is that the user could create a new namespace, that has all the filesystems remounted 'nosuid'. This wouldn't need any new kernel infrastructure, just a suid-root program (e.g. newns_nosuid), that would do a clone(CLONE_NEWNS), then recursively remount everything 'nosuid' in the new namespace. Then restore the user's privileges, and exec a shell. In this namespace the user could mount things to his heart's content. He could mount over system directories or even the root directory, without being able to do any harm. This is very nice, but a bit inpractical, since now all the other programs of the user, his desktop environment, login shells etc. Won't be able to see the userspace filesystems mounted in the private namespace. Of course he can enter the private namespace immediately after login, but then he won't be able to send mail for example, because 'sendmail' could be suid-mail or whatever. I think the following must be true, for a namespace environment to be useful: 1) All the processes should be able to access the same files (including new mounts) 2) User should be able to run suid/sgid programs If the suid/sgid programs want to access the contents of a userspace filesystem, that's tough. You can't have that, because it's insecure. But experience shows that this isn't a problem. The exception is the 'mount' program itself, and that is why I'm slowly working towards making sys_mount() and sys_umount() unprivileged. Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 9:58 ` Miklos Szeredi @ 2005-06-22 10:19 ` Andrew Morton 2005-06-22 10:31 ` Miklos Szeredi 2005-06-22 12:23 ` Eric Van Hensbergen 1 sibling, 1 reply; 620+ messages in thread From: Andrew Morton @ 2005-06-22 10:19 UTC (permalink / raw) To: Miklos Szeredi; +Cc: pavel, linux-kernel Miklos Szeredi <miklos@szeredi.hu> wrote: > > > > > > We could. But that would again be overly restrictive. The goal is to > > > > > make the use of FUSE filesystems for users as simple as possible. If > > > > > the user has to manage multiple namespaces, each with it's own > > > > > restrictions, it's becoming a very un-user-friendly environment. > > > > > > > > I'd have thought that it would be possible to offer the same user interface > > > > as you currently have with private namespaces. Hide any complexity in the > > > > userspace tools? Where's the problem? > > > > > > Sorry, I don't get it. > > > > I'm asking you to expand on what the problems would be if we were to > > enhance the namespace code as suggested. > > OK, what I was thinking, is that the user could create a new > namespace, that has all the filesystems remounted 'nosuid'. This > wouldn't need any new kernel infrastructure, just a suid-root program > (e.g. newns_nosuid), that would do a clone(CLONE_NEWNS), then > recursively remount everything 'nosuid' in the new namespace. Then > restore the user's privileges, and exec a shell. > > In this namespace the user could mount things to his heart's content. > He could mount over system directories or even the root directory, > without being able to do any harm. > > This is very nice, but a bit inpractical, since now all the other > programs of the user, his desktop environment, login shells etc. Won't > be able to see the userspace filesystems mounted in the private > namespace. Yup, that's useless. That makes the whole CLONE_NEWNS idea unworkable, yes? Have we exhausted the alternatives? (If, as you say, v9fs and samba (did you mean smbfs/cifs?) want unprivileged mounts, shouldn't the code which you have there be moved out of fs/fuse/ and into fs/?) ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 10:19 ` Andrew Morton @ 2005-06-22 10:31 ` Miklos Szeredi 0 siblings, 0 replies; 620+ messages in thread From: Miklos Szeredi @ 2005-06-22 10:31 UTC (permalink / raw) To: akpm; +Cc: pavel, linux-kernel > Yup, that's useless. That makes the whole CLONE_NEWNS idea unworkable, > yes? For solving this problem, yes. > Have we exhausted the alternatives? I believe yes. At least to date no workable alternative has been suggested. > (If, as you say, v9fs and samba (did you mean smbfs/cifs?) want > unprivileged mounts, shouldn't the code which you have there be > moved out of fs/fuse/ and into fs/?) Probably. But since the code is so small, and there's no other user yet, I didn't want to take the initiative. Thanks, Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 9:58 ` Miklos Szeredi 2005-06-22 10:19 ` Andrew Morton @ 2005-06-22 12:23 ` Eric Van Hensbergen 2005-06-22 16:28 ` Miklos Szeredi 1 sibling, 1 reply; 620+ messages in thread From: Eric Van Hensbergen @ 2005-06-22 12:23 UTC (permalink / raw) To: Miklos Szeredi; +Cc: akpm, pavel, linux-kernel > > > > I'm asking you to expand on what the problems would be if we were to > > enhance the namespace code as suggested. > > OK, what I was thinking, is that the user could create a new > namespace, that has all the filesystems remounted 'nosuid'. This > wouldn't need any new kernel infrastructure, just a suid-root program > (e.g. newns_nosuid), that would do a clone(CLONE_NEWNS), then > recursively remount everything 'nosuid' in the new namespace. Then > restore the user's privileges, and exec a shell. > I'm confused why everything has to be remounted nosuid. I understand enforcing synthetics to be mounted nosuid, but not the rest of the file systems. I thought all the problems revolving around the private namespace solution where the FUSE team's desire to have per-user namespace and/or per-session namespace versus per-process namespace. -eric ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 12:23 ` Eric Van Hensbergen @ 2005-06-22 16:28 ` Miklos Szeredi 2005-06-22 16:48 ` Eric Van Hensbergen 0 siblings, 1 reply; 620+ messages in thread From: Miklos Szeredi @ 2005-06-22 16:28 UTC (permalink / raw) To: ericvh; +Cc: akpm, pavel, linux-kernel > > > I'm asking you to expand on what the problems would be if we were to > > > enhance the namespace code as suggested. > > > > OK, what I was thinking, is that the user could create a new > > namespace, that has all the filesystems remounted 'nosuid'. This > > wouldn't need any new kernel infrastructure, just a suid-root program > > (e.g. newns_nosuid), that would do a clone(CLONE_NEWNS), then > > recursively remount everything 'nosuid' in the new namespace. Then > > restore the user's privileges, and exec a shell. > > > > I'm confused why everything has to be remounted nosuid. I understand > enforcing synthetics to be mounted nosuid, but not the rest of the > file systems. It's related to the problem of a suid program accessing synthetic filesystem, and filesystem doing something bad to suid program (make it hang, supply bogus data ...). This can be solved by "squashing" suid for the whole namespace (basically the Plan 9 solution). Unfortunately this is not really practical in Linux/Unix. > I thought all the problems revolving around the private namespace > solution where the FUSE team's desire to have per-user namespace > and/or per-session namespace versus per-process namespace. That's a different aspect of this thing. Private namespaces cannot do anything about the above problem. Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 16:28 ` Miklos Szeredi @ 2005-06-22 16:48 ` Eric Van Hensbergen 2005-06-22 17:19 ` Miklos Szeredi 0 siblings, 1 reply; 620+ messages in thread From: Eric Van Hensbergen @ 2005-06-22 16:48 UTC (permalink / raw) To: Miklos Szeredi; +Cc: akpm, pavel, linux-kernel On 6/22/05, Miklos Szeredi <miklos@szeredi.hu> wrote: > > > > I'm confused why everything has to be remounted nosuid. I understand > > enforcing synthetics to be mounted nosuid, but not the rest of the > > file systems. > > It's related to the problem of a suid program accessing synthetic > filesystem, and filesystem doing something bad to suid program (make > it hang, supply bogus data ...). This can be solved by "squashing" > suid for the whole namespace (basically the Plan 9 solution). > Unfortunately this is not really practical in Linux/Unix. > Just to make sure I understand you - if I don't squash suid for the entire name space, a user could mount a malicious synthetic (even with NOSUID) and then launch an SUID app from an inherited mount which would then traverse to the malicious synthetic. That's a nasty case I hadn't considered before -- however, what's the potential damage there? The user could hold up progress of the SUID app that they launched, but that wouldn't necessarily impede system progress since system-critical suid apps wouldn't be typically launched by a user. I suppose there is the possibility that if multiple instances of such an SUID app share a global lock you could get into trouble -- do we have any concrete example apps that would exhibit this kind of behavior? Are there other vunerabilities that I'm missing? -eric ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 16:48 ` Eric Van Hensbergen @ 2005-06-22 17:19 ` Miklos Szeredi 0 siblings, 0 replies; 620+ messages in thread From: Miklos Szeredi @ 2005-06-22 17:19 UTC (permalink / raw) To: ericvh; +Cc: akpm, pavel, linux-kernel > > It's related to the problem of a suid program accessing synthetic > > filesystem, and filesystem doing something bad to suid program (make > > it hang, supply bogus data ...). This can be solved by "squashing" > > suid for the whole namespace (basically the Plan 9 solution). > > Unfortunately this is not really practical in Linux/Unix. > > > > Just to make sure I understand you - if I don't squash suid for the > entire name space, a user could mount a malicious synthetic (even with > NOSUID) and then launch an SUID app from an inherited mount which > would then traverse to the malicious synthetic. Yes. > That's a nasty case I hadn't considered before -- however, what's the > potential damage there? The user could hold up progress of the SUID > app that they launched, but that wouldn't necessarily impede system > progress since system-critical suid apps wouldn't be typically > launched by a user. I suppose there is the possibility that if > multiple instances of such an SUID app share a global lock you could > get into trouble -- do we have any concrete example apps that would > exhibit this kind of behavior? I don't know any. But with 'sudo' the potential set of SUID apps is basically infinite, so you'd have a hard time proving that this sort of situation won't arise. > Are there other vunerabilities that I'm missing? Another theoretical possibility is that you make the SUID app consume some resource by feeding it a large-file/deep-directory/etc that quota would otherwise prevent (you can't do quota on a synthetic filesystem, without the filesystem's cooperation). Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 9:07 ` Miklos Szeredi 2005-06-22 9:12 ` Andrew Morton @ 2005-06-22 9:17 ` Pavel Machek 1 sibling, 0 replies; 620+ messages in thread From: Pavel Machek @ 2005-06-22 9:17 UTC (permalink / raw) To: Miklos Szeredi; +Cc: akpm, linux-kernel Hi! > > Can we enhance private namespaces so they can squash setuid/setgid? If so, > > is that adequate? > > We could. But that would again be overly restrictive. The goal is to > make the use of FUSE filesystems for users as simple as possible. If > the user has to manage multiple namespaces, each with it's own > restrictions, it's becoming a very un-user-friendly environment. Actually I think this solution is way less ugly. We have precent: if task is ptraced, suid bits on anything it execs are ignored. I don't think user interface issues belong to the kernel (and I do not think different namespaces are that bad for the user; various chroots and ld_preload hacks already work that way) Pavel -- teflon -- maybe it is a trademark, but it should not be. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 7:16 ` Miklos Szeredi 2005-06-22 7:49 ` Andrew Morton @ 2005-06-22 17:01 ` Theodore Ts'o 2005-06-22 17:48 ` Miklos Szeredi 1 sibling, 1 reply; 620+ messages in thread From: Theodore Ts'o @ 2005-06-22 17:01 UTC (permalink / raw) To: Miklos Szeredi; +Cc: akpm, pavel, linux-kernel On Wed, Jun 22, 2005 at 09:16:34AM +0200, Miklos Szeredi wrote: > The controversial part is fuse_allow_task() called from > fuse_permission() and fuse_revalidate() (fs/fuse/dir.c). > > This function (as explained by the header comment) disallows access to > the filesystem for any task, which the filesystem owner (the user who > did the mount) is not allowed to ptrace. > > The rationale is that accessing the filesystem gives the filesystem > implementor ptrace like capabilities (detailed in > Documentation/filesystems/fuse.txt) > > It is controversial, because obviously root owned tasks are not > ptrace-able by the user, and so these tasks will be denied access to > the user mounted filesystem (-EACCESS is returned on stat() or any > other file operation). I don't think this should be objectionable, since we already have times when root-owned tasks can get EACCESS when accessing some filesystem. This can happen with any distributed filesystem that enforces real security --- whether it be nfs-root-squash, or the Andrew Filesystem, or NFSv4. Root can get "permission denied" when some other userid with appropriate credentials would be allowed to access the file/directory. On the other hand, sometimes a root process, or some other user's process, might _want_ to give permission to allow a trusted FUSE filesystem the potential to monkey with it (return potentially untrusted information, or stop it entirely), in exchange for access to the filesystem. So it would be nice if there was some way that a process could tell the kernel that it is willing to give permission to allow certain FUSE filesystems to potentially affect it. Say, via a fnctl() call, perhaps. - Ted ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 17:01 ` Theodore Ts'o @ 2005-06-22 17:48 ` Miklos Szeredi 2005-06-23 23:34 ` Theodore Ts'o 0 siblings, 1 reply; 620+ messages in thread From: Miklos Szeredi @ 2005-06-22 17:48 UTC (permalink / raw) To: tytso; +Cc: akpm, pavel, linux-kernel > I don't think this should be objectionable, since we already have > times when root-owned tasks can get EACCESS when accessing some > filesystem. This can happen with any distributed filesystem that > enforces real security --- whether it be nfs-root-squash, or the > Andrew Filesystem, or NFSv4. Root can get "permission denied" when > some other userid with appropriate credentials would be allowed to > access the file/directory. Right. > On the other hand, sometimes a root process, or some other user's > process, might _want_ to give permission to allow a trusted FUSE > filesystem the potential to monkey with it (return potentially > untrusted information, or stop it entirely), in exchange for access to > the filesystem. So it would be nice if there was some way that a > process could tell the kernel that it is willing to give permission to > allow certain FUSE filesystems to potentially affect it. Say, via a > fnctl() call, perhaps. Hmm. 'su' works for root. How do you think fcntl() could be used? I think a task flag settable via prctl() would be more appropriate. Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 17:48 ` Miklos Szeredi @ 2005-06-23 23:34 ` Theodore Ts'o 2005-06-24 5:49 ` Miklos Szeredi 0 siblings, 1 reply; 620+ messages in thread From: Theodore Ts'o @ 2005-06-23 23:34 UTC (permalink / raw) To: Miklos Szeredi; +Cc: akpm, pavel, linux-kernel On Wed, Jun 22, 2005 at 07:48:50PM +0200, Miklos Szeredi wrote: > > On the other hand, sometimes a root process, or some other user's > > process, might _want_ to give permission to allow a trusted FUSE > > filesystem the potential to monkey with it (return potentially > > untrusted information, or stop it entirely), in exchange for access to > > the filesystem. So it would be nice if there was some way that a > > process could tell the kernel that it is willing to give permission to > > allow certain FUSE filesystems to potentially affect it. Say, via a > > fnctl() call, perhaps. > > Hmm. 'su' works for root. ?? Not sure what you're saying. > How do you think fcntl() could be used? I think a task flag settable > via prctl() would be more appropriate. The problem with a task flag is that a root process might not want to give permission for _all_ user-mountable filesystem, but only certain ones. So the idea is that the process should be able to specify specific mountpoints as being "ok" for the process to access. - Ted ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-23 23:34 ` Theodore Ts'o @ 2005-06-24 5:49 ` Miklos Szeredi 2005-06-26 3:04 ` Theodore Ts'o 0 siblings, 1 reply; 620+ messages in thread From: Miklos Szeredi @ 2005-06-24 5:49 UTC (permalink / raw) To: tytso; +Cc: miklos, akpm, pavel, linux-kernel > > > On the other hand, sometimes a root process, or some other user's > > > process, might _want_ to give permission to allow a trusted FUSE > > > filesystem the potential to monkey with it (return potentially > > > untrusted information, or stop it entirely), in exchange for access to > > > the filesystem. So it would be nice if there was some way that a > > > process could tell the kernel that it is willing to give permission to > > > allow certain FUSE filesystems to potentially affect it. Say, via a > > > fnctl() call, perhaps. > > > > Hmm. 'su' works for root. > > ?? Not sure what you're saying. If user X mounts a filesystem, and root wants to access it, it can do 'su X' and see the otherwise inaccesible files. > > How do you think fcntl() could be used? I think a task flag settable > > via prctl() would be more appropriate. > > The problem with a task flag is that a root process might not want to > give permission for _all_ user-mountable filesystem, but only certain > ones. So the idea is that the process should be able to specify > specific mountpoints as being "ok" for the process to access. OK, so you're saying, it should be a per-mountpoint flag, and not a per-process one. Then the solution is simple I think. Root can just remount the filesystem with the 'allow_other' flag (that's currently not possible, because the remount_fs operation is not yet implemented). Thanks, Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-24 5:49 ` Miklos Szeredi @ 2005-06-26 3:04 ` Theodore Ts'o 0 siblings, 0 replies; 620+ messages in thread From: Theodore Ts'o @ 2005-06-26 3:04 UTC (permalink / raw) To: Miklos Szeredi; +Cc: akpm, pavel, linux-kernel On Fri, Jun 24, 2005 at 07:49:50AM +0200, Miklos Szeredi wrote: > > ?? Not sure what you're saying. > > If user X mounts a filesystem, and root wants to access it, it can do > 'su X' and see the otherwise inaccesible files. That's rather awkward, and will be painful in certain applications that are trying to scan multiple filesystems, possibly run by different users X, Y, and Z. > > > How do you think fcntl() could be used? I think a task flag settable > > > via prctl() would be more appropriate. > > > > The problem with a task flag is that a root process might not want to > > give permission for _all_ user-mountable filesystem, but only certain > > ones. So the idea is that the process should be able to specify > > specific mountpoints as being "ok" for the process to access. > > OK, so you're saying, it should be a per-mountpoint flag, and not a > per-process one. No, I'm saying that it should be both per-mountpoint and per-process. Hence something like fnctl. Problem is though fnctl() works on a file descriptor, and with FUSE running you wouldn't be able to even get a file descriptor opened on the mountpoint. > Then the solution is simple I think. Root can just remount the > filesystem with the 'allow_other' flag (that's currently not possible, > because the remount_fs operation is not yet implemented). Globally remounting the filesystem allow_other isn't the right thing, since applications that don't know what to expect will be taken by surprise. Hence my suggestion to do something which is both per-mountpoint and per-process. - Ted ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 6:39 ` Andrew Morton 2005-06-22 7:16 ` Miklos Szeredi @ 2005-06-24 16:00 ` Gábor Lénárt 1 sibling, 0 replies; 620+ messages in thread From: Gábor Lénárt @ 2005-06-24 16:00 UTC (permalink / raw) To: linux-kernel EHLO, On Tue, Jun 21, 2005 at 11:39:14PM -0700, Andrew Morton wrote: > > > System where users can mount their own filesystems should not be > > > called "Unix" any more. > > > > It's not. It's "Linux". > > It would be helpful if we could have a brief description of the feature > which you're discussing here. We discussed this a couple of months back, > but I've forgotten most of it and it was off-list I think. <offtopic> Excuse me, I'm far from being a filesystem/vfs expert ... However I've got the idea about the merging fuse/reiser4 that some guys keep complaining about the quite strange behaviour of these stuffs: when I write 'strange' I mean strange from the view point of some standard unix ideas about filesystems (and anything related to filesystems like permission checking, namespaces etc) and how they should be implemented and handled. This reminds me articles about comparing Linux and BSDs. BSD guys claims that BSD distros _ARE_ unices but Linux is not. It's out of scope to waste mails about these flames like this (it's question of view point as almost always) however there IS some lesson here. BSD systems are somewhat (well, not counting the interesting ideas of DragonFly BSD) conservative to implement new stuffs. I'm about stuffs like filesystem transactions, API exported to the user space to be able to do things like deleting data from the begining of the file (there is API call to truncate - from the end of the file ...) and such (what a quite braindead idea to rewrite eg a 10Gbyte long file just for inserting one byte to somewhere in the middle of the file - in 2005 ...). The only thing explains where the later is not present in most OSes is because of historical reasons and not technical ones. And if even Linux does not want to open toward extended filesystem abilities which common open source system will? I guess none. I can inmagine that vendors of some closed source systems will exploit the hole in the area of outdated filesystem concept of our current world and when it becomes reality it's to late. Maybe. Please forgive for my possible offtopic mail here but I could not resist :) </offtopic> -- - Gábor ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 6:20 ` Miklos Szeredi 2005-06-22 6:39 ` Andrew Morton @ 2005-06-22 8:53 ` Pavel Machek 1 sibling, 0 replies; 620+ messages in thread From: Pavel Machek @ 2005-06-22 8:53 UTC (permalink / raw) To: Miklos Szeredi; +Cc: akpm, linux-kernel Hi! > > > You have a choice of: 1) believe me that the current solution is > > > fine > > > > > 2) get down and try to understand the damn thing, and then come up > > > with technical arguments for/against it > > > > Argument is "it is **** ugly". > > Yeah, that's your opinion. Mine is that it's f****** beautiful ;). > > There are plenty of ugly things in Unix/Linux that you've become so > accustomed to, that they no longer seem ugly. Think about the sticky > bit on directories for example. That one was breaking assumptions > left and right when it got introduced, but people came to accept it, > because it's useful. Just for the record, I still consider sticky bit "slightly" ugly and nfs root squash "very" ugly. > > Your fuse.txt explains why it is not security hole. It does not > > explain why your interface is the best possible, and what alternative > > ways of "not security hole" exist. > > That's because I don't see any alternative. The "preventing user from > tracing root" and "preventing access to user's filesysem by root" must > come together. There's doesn't seem to be any other way. It is clear that we can't allow root (or anyone else) to access that filesystem. Infinite namespace is nice trap. > BTW, thanks for reading through fuse.txt :) You are welcome ;-). Pavel -- teflon -- maybe it is a trademark, but it should not be. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 22:06 ` Pavel Machek 2005-06-22 6:20 ` Miklos Szeredi @ 2005-06-22 12:57 ` Matthias Urlichs 2005-06-22 14:43 ` Eric Van Hensbergen 2 siblings, 0 replies; 620+ messages in thread From: Matthias Urlichs @ 2005-06-22 12:57 UTC (permalink / raw) To: linux-kernel Hi, Pavel Machek wrote: > How is it going > to interact with backup tools? I admit to severe dislike for any backup tool which crosses file system borders without permission, and therefore am not going to cry if something breaks because of this. Ditto updatedb/locate -- it's a remote file system. Tools should treat it as such. NB: I am assuming that root can stat() the mountpoint and that it's not possible to delay this call arbitrarily. If not, the debate IMHO should be whether to special-case this situation. -- Matthias Urlichs | {M:U} IT Design @ m-u-it.de | smurf@smurf.noris.de Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de - - The early worm has a death wish. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 22:06 ` Pavel Machek 2005-06-22 6:20 ` Miklos Szeredi 2005-06-22 12:57 ` Matthias Urlichs @ 2005-06-22 14:43 ` Eric Van Hensbergen 2005-06-22 15:08 ` Pavel Machek 2 siblings, 1 reply; 620+ messages in thread From: Eric Van Hensbergen @ 2005-06-22 14:43 UTC (permalink / raw) To: Pavel Machek; +Cc: Miklos Szeredi, akpm, linux-kernel On 6/21/05, Pavel Machek <pavel@ucw.cz> wrote: > > > > An emotional argument again. What's "strange" about it? > > Not so emotional argument... > > System where users can mount their own filesystems should not be > called "Unix" any more. It introduces new mechanism, similar to > ptrace. I think that's a rather severe statement. I don't see allowing user mounts damaging standard UNIX system semantics as long as certain rules are followed. After all, user-mounts and private name spaces are what the original authors of UNIX went on to develop. > It restricts root in ways not seen before. How is > updatedb/locate supposed to work on system with this? How is it going > to interact with backup tools? > I think requiring user-mounts to be in private name spaces solves many of these issues -- you don't have to worry about system daemons and/or other users wandering into synthetic file systems. -eric ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 14:43 ` Eric Van Hensbergen @ 2005-06-22 15:08 ` Pavel Machek 2005-06-22 15:50 ` Eric Van Hensbergen 0 siblings, 1 reply; 620+ messages in thread From: Pavel Machek @ 2005-06-22 15:08 UTC (permalink / raw) To: Eric Van Hensbergen; +Cc: Miklos Szeredi, akpm, linux-kernel Hi! > > > An emotional argument again. What's "strange" about it? > > > > Not so emotional argument... > > > > System where users can mount their own filesystems should not be > > called "Unix" any more. It introduces new mechanism, similar to > > ptrace. > > I think that's a rather severe statement. I don't see allowing user > mounts damaging standard UNIX system semantics as long as certain > rules are followed. After all, user-mounts and private name spaces > are what the original authors of UNIX went on to develop. Well, but notice how it is called "Plan 9", not "Unix" :-). The "certain rules" are rather tricky to enforce... btw any ideas how Plan 9 solves problems around user-mounts? Does it allow it at all? Pavel -- teflon -- maybe it is a trademark, but it should not be. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 15:08 ` Pavel Machek @ 2005-06-22 15:50 ` Eric Van Hensbergen 2005-06-22 16:34 ` Eric Van Hensbergen 2005-06-22 16:43 ` Alan Cox 0 siblings, 2 replies; 620+ messages in thread From: Eric Van Hensbergen @ 2005-06-22 15:50 UTC (permalink / raw) To: Pavel Machek; +Cc: Miklos Szeredi, akpm, linux-kernel On 6/22/05, Pavel Machek <pavel@ucw.cz> wrote: > > > > I think that's a rather severe statement. I don't see allowing user > > mounts damaging standard UNIX system semantics as long as certain > > rules are followed. After all, user-mounts and private name spaces > > are what the original authors of UNIX went on to develop. > > Well, but notice how it is called "Plan 9", not "Unix" :-). The > "certain rules" are rather tricky to enforce... btw any ideas how > Plan 9 solves problems around user-mounts? Does it allow it at all? > I agree the rules can be tricky, but I think we can start with something ultra-conservative (but not as ultra-conservative as no user mounts) and work from there. Plan 9 does indeed allow (in fact in many ways it depends on) user mounts. There are a number of answers for how it gets around some of the sticky issues -- some of those may be applicable in UNIX-style systems, while others most likely don't apply. The ones that don't apply: (a) Plan 9 has no root user, nor does it have a concept of setuid/setgid -- this solves a lot of problems, but isn't practical in UNIX environments. (b) Plan 9 cluster environments typically have user-controller terminals and shared centralized resources (cpu servers, file servers, etc.) Users have more or less complete control over their local workstation (which includes their name space) -- and they can import resources/services from shared servers (where they have less/no control). Critical services, such as authentication, are hosted on a remote server protected from normal users. While this sort of environment can be set up using UNIX systems -- its probably not practical for most folks. The ones that do apply: (c) Private Name Space - by default, every new process in Plan 9 gets a private name space inherited from its parent. This means that any user mounts will be in their own private name space and not interfere with system tasks or other users. As I've stated, I think this has direct application in UNIX environments. We may not go to the extreme of making CLONENS the default for new threads, but I think it should be easy enough to enforce unprivileged mounts be in a non-global name space. (d) For situations in which sharing a name space is desirable, the name space is re-exported through a global handle (/srv in Plan 9's case) so that a user can mount instances of his private name space in other threads -- or he can set permissions on the handle so it may be used by other users. This re-export is done using the 9P protocol (although if re-exporting to the same system it short-circuits to just function calls within the OS). The 9P layer adds a certain level of security to just allowing name spaces to be accessed/mounted directly - of course it also adds a bit of overhead. We are currently looking at this sort of implementation in the v9fs 2.1 release. Given that (a) & (b) aren't practical in UNIX environments, I think a reasonable set of restrictions (some of which were suggested by Miklos) can be used to maintain integrity: 1) only allow user's to mount/bind on directories/files where they have unconditional write access. 2) enforce NOSUID mount options on user-mounts If you combine these two restrictions with only allowing unprivileged mounts in private name space I think you get 90% there. The only thing left to resolve is the best way to allow sharing private name spaces between threads/users -- and I still view this as more of extended functionality than a hard-requirement. I'm giving a talk at OLS overviewing my perspective on the barriers and solutions. My motivation was to try and get some dialog/compromise/closure on some of the user-space mount and private name space issues. Hopefully some of the stake-holders will attend. -eric ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 15:50 ` Eric Van Hensbergen @ 2005-06-22 16:34 ` Eric Van Hensbergen 2005-06-22 16:43 ` Alan Cox 1 sibling, 0 replies; 620+ messages in thread From: Eric Van Hensbergen @ 2005-06-22 16:34 UTC (permalink / raw) To: Pavel Machek; +Cc: Miklos Szeredi, akpm, linux-kernel On 6/22/05, Eric Van Hensbergen <ericvh@gmail.com> wrote: ... > > If you combine these two restrictions with only allowing unprivileged > mounts in private name space I think you get 90% there. The only > thing left to resolve is the best way to allow sharing private name > spaces between threads/users -- and I still view this as more of > extended functionality than a hard-requirement. > Reviewing my notes, there were a few subtle restrictions I forgot (most of which originally suggested by Miklos): (a) User's can't mount file system types not deemed "safe" (via flag in the file system type) -- this should help mitigate user's exploiting bugs in existing file systems to interfere with the system. Judging when a file system type is "safe" is a nasty kettle of fish though... (b) Enforce NODEV along with NOSUID so that user-based synthetics can't have device inodes with compromised permissions, etc. -eric ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 15:50 ` Eric Van Hensbergen 2005-06-22 16:34 ` Eric Van Hensbergen @ 2005-06-22 16:43 ` Alan Cox 2005-06-22 16:56 ` Eric Van Hensbergen 1 sibling, 1 reply; 620+ messages in thread From: Alan Cox @ 2005-06-22 16:43 UTC (permalink / raw) To: Eric Van Hensbergen Cc: Pavel Machek, Miklos Szeredi, akpm, Linux Kernel Mailing List > 1) only allow user's to mount/bind on directories/files where they > have unconditional write access. Like say /tmp. Build a bizarre behaving /tmp and I can do funky stuff with some third party suid apps. Its a good start but you probably want a stronger policy and one enforced by the user space side not kernel (eg "Below ~") > 2) enforce NOSUID mount options on user-mounts 2 is unneccessarily crude. Just enforce suid owner/owner group. Alan ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-22 16:43 ` Alan Cox @ 2005-06-22 16:56 ` Eric Van Hensbergen 0 siblings, 0 replies; 620+ messages in thread From: Eric Van Hensbergen @ 2005-06-22 16:56 UTC (permalink / raw) To: Alan Cox; +Cc: Pavel Machek, Miklos Szeredi, akpm, Linux Kernel Mailing List On 6/22/05, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote: > > 1) only allow user's to mount/bind on directories/files where they > > have unconditional write access. > > Like say /tmp. Build a bizarre behaving /tmp and I can do funky stuff > with some third party suid apps. Its a good start but you probably want > a stronger policy and one enforced by the user space side not kernel (eg > "Below ~") > Well in the original discussions Miklos had classified directories that had the sticky bit set (such as /tmp) as out-of-bounds for user-mounts. However, its a point well taken. I had originally proposed having some sort of a policy file (sort of like an extended fstab with regular expressions) to give more granular control over where users could and couldn't mount (along with what types of devices, network servers, and file systems they could mount from). However, this leans more towards the "super-mount" suid-application which I think many found undesirable. An alternative would be some way for the kernel to consult with an application about different mount policies. I don't know what the right thing is here. > > 2) enforce NOSUID mount options on user-mounts > > 2 is unneccessarily crude. Just enforce suid owner/owner group. > I'm dense this morning, not sure what you mean here. -eric ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 10:22 ` -mm -> 2.6.13 merge status (fuse) Miklos Szeredi 2005-06-21 12:39 ` Alan Cox 2005-06-21 14:28 ` Pavel Machek @ 2005-06-21 15:26 ` Avuton Olrich 2005-06-21 15:36 ` Miklos Szeredi 2 siblings, 1 reply; 620+ messages in thread From: Avuton Olrich @ 2005-06-21 15:26 UTC (permalink / raw) To: Miklos Szeredi; +Cc: akpm, linux-kernel On 6/21/05, Miklos Szeredi <miklos@szeredi.hu> wrote: > I won't shed many tears if you drop fuse-nfs-export.patch. It would > at least give the userspace solution some boost. > > However the patch is pretty small, and despite it's flaws, I know it's > used by a number of people. Why not leave it up to the user as an option, for the time being at least. Does this somehow break things? thanks, avuton -- Anyone who quotes me in their sig is an idiot. -- Rusty Russell. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (fuse) 2005-06-21 15:26 ` Avuton Olrich @ 2005-06-21 15:36 ` Miklos Szeredi 0 siblings, 0 replies; 620+ messages in thread From: Miklos Szeredi @ 2005-06-21 15:36 UTC (permalink / raw) To: avuton; +Cc: akpm, linux-kernel > > I won't shed many tears if you drop fuse-nfs-export.patch. It would > > at least give the userspace solution some boost. > > > > However the patch is pretty small, and despite it's flaws, I know it's > > used by a number of people. > > Why not leave it up to the user as an option, for the time being at > least. Making it a config option could make sense, yes. > Does this somehow break things? You mean outside NFS export? No, it's completely harmless. NFS export itself is slightly broken (random ESTALE errors), but it's still useful. Thanks, Miklos ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton 2005-06-21 10:22 ` -mm -> 2.6.13 merge status (fuse) Miklos Szeredi @ 2005-06-21 12:01 ` Andrey Panin 2005-06-21 12:35 ` Alan Cox ` (16 subsequent siblings) 18 siblings, 0 replies; 620+ messages in thread From: Andrey Panin @ 2005-06-21 12:01 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel [-- Attachment #1: Type: text/plain, Size: 6529 bytes --] On 171, 06 20, 2005 at 11:54:58 -0700, Andrew Morton wrote: > > This summarises my current thinking on various patches which are presently > in -mm. I cover large things and small-but-controversial things. Anything > which isn't covered here (and that's a lot of material) is probably a "will > merge", unless it obviously isn't. > > (If you reply to this email it would be a good idea to alter the Subject: > to reflect which feature you are discussing) > > > > git-ocfs > > The OCFS2 filesystem. OK by me, although I'm not sure it's had enough > review. > > sparsemem > > OK by me for a merge. Need to poke arch maintainers first, check that > they've looked at it sufficiently closely. > > vm-early-zone-reclaim > > Needs some convincing benchmark numbers to back it up. Otherwise OK. > > avoiding-mmap-fragmentation > > Tricky. Addresses vm area fragmentation issues due to recent > optimisations to the free-area lookup code. Will merge. > > periodically-drain-non-local-pagesets > > Will merge > > pcibus_to_node and users > > Will merge > > CONFIG_HZ for x86 and ia64: changes default HZ to 250, make HZ Kconfigurable. > > Will merge (will switch default to 1000 Hz later if that seems necessary) > > dmi-*.patch > > Will merge. I have a comment "The below break x440". Maybe it got > fixed. We'll doubtless hear if not. Fixed, patch merged in -mm as dmi-move-acpi-sleep-quirk-fix.patch http://marc.theaimsgroup.com/?l=linux-kernel&m=111829134708641&w=2 http://marc.theaimsgroup.com/?l=linux-kernel&m=111832375203467&w=2 > xen-*.patch > > These are little cleanups and abstractions which make a Xen merge > easier. May as well merge them. > > CPU hotplug for x86 and x86_64 > > Not really useful on current hardware, but these provide > infrastructure which some power management patches need, and it seems > sensible to make the reference architecture support hotplug. Will merge. > > swsusp-on-SMP > > Will merge. > > cfq version 3 > > Not sure. Jens seems to be setting up a few git trees. On hold. > > RCUification of the key management code > > Don't know - dhowells seemed diffident last time we discussed this. > > timers-fixes-improvements.patch > > SMP speedups for the core timer code. It was bumpy, but this seems > stable now. Will merge. > > kprobes-* > > Will merge > > rapidio-* > > Will merge. > > namespace*.patch > > Awaiting viro ack. > > xtensa architecture > > Is xtensa now, or will it be in the future a sufficiently popular > architecture to justify the cost of having this code in the tree? > > Heaven knows. Will merge. > > dlm-*.patch: Red Hat distributed lock manager > > Hard. Right now it seems that no in-kernel projects will use this and > only one out-of-kernel project will use it. Shelve the problem until > after Kernel Summit, where some light may be shed. > > Opinions are sought... > > connector.patch > > Nice idea IMO, but there are still questions around the > implementation. More dialogue needed ;) > > connector-add-a-fork-connector.patch > > OK, but needs connector. > > inotify > > There are still concerns about the userspace API and internal > implementation details. More slogging needed. > > pcmcia-*.patch > > Makes the pcmcia layer generate hotplug events and deprecates cardmgr. > Will merge. > > NUMA-aware slab allocator > > Seems stable now, but it needs some ifdef reduction work before > merging, please. > > CPU scheduler > > Will merge some of these patches. We're still discussing which ones. > > perfctr > > Not yet, but getting closer. The PPC64 guys still need to sort out a > few interface issues with Mikael. We might be able to fit this into > 2.6.13 if people get a move on. > > cachefs > > This is a ton of code which knows rather a lot about pagecache > internals. It allows the AFS client to cache file contents on a local > blockdev. > > I don't think it's a justified addition for only AFS and I'd prefer to > see it proven for NFS as well. > > Issues around add-page-becoming-writable-notification.patch need to > be resolved. > > cachefs-for-nfs > > A recent addition. Needs review from NFS developers and considerably > more testing. > > These things aren't looking likely for 2.6.13. > > kexec and kdump > > I guess we should merge these. > > I'm still concerned that the various device shutdown problems will > mean that the success rate for crashing kernels is not high enough for > kdump to be considered a success. In which case in six months time we'll > hear rumours about vendors shipping wholly different crashdump > implementations, which would be quite bad. > > But I think this has gone as far as it can go in -mm, so it's a bit of > a punt. > > reiser4 > > Merge it, I guess. > > The patches still contain all the reiser4-specific namespace > enhancements, only it is disabled, so it is effectively dead code. Maybe > we should ask that it actually be removed? > > v9fs > > I'm not sure that this has a sufficiently high > usefulness-to-maintenance-cost ratio. > > fuse > > This is useful, but there are, AFAIK, two issues: > > - We're still deadlocked over some permission-checking hacks in there > > - It has an NFS server implementation which only works if the > to-be-served file happens to be in dcache. > > It has been said that a userspace NFS server can be used to get > full NFS server functionality with FUSE. I think the half-assed kernel > implementation should be done away with. > > execute-in-place > > Will merge. Have the embedded guys commented on the usefulness of > this for execute-out-of-ROM? > > > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > -- Andrey Panin | Linux and UNIX system administrator pazke@donpac.ru | PGP key: wwwkeys.pgp.net [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton 2005-06-21 10:22 ` -mm -> 2.6.13 merge status (fuse) Miklos Szeredi 2005-06-21 12:01 ` -mm -> 2.6.13 merge status Andrey Panin @ 2005-06-21 12:35 ` Alan Cox 2005-06-21 13:07 ` Arjan van de Ven 2005-06-21 12:43 ` Carsten Otte ` (15 subsequent siblings) 18 siblings, 1 reply; 620+ messages in thread From: Alan Cox @ 2005-06-21 12:35 UTC (permalink / raw) To: Andrew Morton; +Cc: Linux Kernel Mailing List On Maw, 2005-06-21 at 07:54, Andrew Morton wrote: > CONFIG_HZ for x86 and ia64: changes default HZ to 250, make HZ Kconfigurable. > Will merge (will switch default to 1000 Hz later if that seems necessary) This has been in Fedora for a while. DaveJ can probably give you more info. From own testing 100Hz is how far down you want to go to avoid random clock slew on laptops and to see power improvements. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 12:35 ` Alan Cox @ 2005-06-21 13:07 ` Arjan van de Ven 2005-06-22 10:50 ` Erik Slagter 0 siblings, 1 reply; 620+ messages in thread From: Arjan van de Ven @ 2005-06-21 13:07 UTC (permalink / raw) To: Alan Cox; +Cc: Linux Kernel Mailing List, Andrew Morton [-- Attachment #1: Type: text/plain, Size: 613 bytes --] On Tue, 2005-06-21 at 13:35 +0100, Alan Cox wrote: > On Maw, 2005-06-21 at 07:54, Andrew Morton wrote: > > CONFIG_HZ for x86 and ia64: changes default HZ to 250, make HZ Kconfigurable. > > Will merge (will switch default to 1000 Hz later if that seems necessary) > > This has been in Fedora for a while. DaveJ can probably give you more > info. From own testing 100Hz is how far down you want to go to avoid > random clock slew on laptops and to see power improvements. actually 250Hz is a not so fun value. 300 is a lot nicer (multiple of both 50Hz and 60Hz and thus covers most TV standards) [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 13:07 ` Arjan van de Ven @ 2005-06-22 10:50 ` Erik Slagter 0 siblings, 0 replies; 620+ messages in thread From: Erik Slagter @ 2005-06-22 10:50 UTC (permalink / raw) To: Linux Kernel Mailing List On Tue, 2005-06-21 at 09:07 -0400, Arjan van de Ven wrote: > On Tue, 2005-06-21 at 13:35 +0100, Alan Cox wrote: > > On Maw, 2005-06-21 at 07:54, Andrew Morton wrote: > > > CONFIG_HZ for x86 and ia64: changes default HZ to 250, make HZ Kconfigurable. > > > Will merge (will switch default to 1000 Hz later if that seems necessary) > > > > This has been in Fedora for a while. DaveJ can probably give you more > > info. From own testing 100Hz is how far down you want to go to avoid > > random clock slew on laptops and to see power improvements. > > actually 250Hz is a not so fun value. 300 is a lot nicer (multiple of > both 50Hz and 60Hz and thus covers most TV standards) Sorry, the ITU-R "M" standard specifies 30000/1001 frames (60000/1001 fields) per second, that's close to 30/60 but not the same. Now please please don't make use run HZ = (60000/1001 * 5) or similar ;-) BTW it seems to me that power management using C2/C3 states will work much more efficiently with a lower HZ because the chipset/processor will be a larger percentage of the time in c2/c3 mode, right? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (2 preceding siblings ...) 2005-06-21 12:35 ` Alan Cox @ 2005-06-21 12:43 ` Carsten Otte 2005-06-21 12:58 ` Jörn Engel ` (14 subsequent siblings) 18 siblings, 0 replies; 620+ messages in thread From: Carsten Otte @ 2005-06-21 12:43 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel On 6/21/05, Andrew Morton <akpm@osdl.org> wrote: > execute-in-place > > Will merge. Have the embedded guys commented on the usefulness of > this for execute-out-of-ROM? Allright. Going to push our test-team to run their tests on the xip code that is in -mm. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (3 preceding siblings ...) 2005-06-21 12:43 ` Carsten Otte @ 2005-06-21 12:58 ` Jörn Engel 2005-06-21 13:08 ` -mm -> 2.6.13 merge status (Suspend-to-disk) Nigel Cunningham ` (13 subsequent siblings) 18 siblings, 0 replies; 620+ messages in thread From: Jörn Engel @ 2005-06-21 12:58 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel On Mon, 20 June 2005 23:54:58 -0700, Andrew Morton wrote: > > execute-in-place > > Will merge. Have the embedded guys commented on the usefulness of > this for execute-out-of-ROM? It looks fairly useful, but XIP for NOR flashes still needs additional work. No objections from my side. Jörn -- Optimizations always bust things, because all optimizations are, in the long haul, a form of cheating, and cheaters eventually get caught. -- Larry Wall ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (Suspend-to-disk) 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (4 preceding siblings ...) 2005-06-21 12:58 ` Jörn Engel @ 2005-06-21 13:08 ` Nigel Cunningham 2005-06-21 9:44 ` Marcelo Tosatti 2005-06-21 14:16 ` Pavel Machek 2005-06-21 13:51 ` v9fs (-mm -> 2.6.13 merge status) Eric Van Hensbergen ` (12 subsequent siblings) 18 siblings, 2 replies; 620+ messages in thread From: Nigel Cunningham @ 2005-06-21 13:08 UTC (permalink / raw) To: Andrew Morton; +Cc: Linux Kernel Mailing List, Marcelo Tosatti Hi. (Marcelo: Copied for issue at the bottom). On Tue, 2005-06-21 at 16:54, Andrew Morton wrote: > CPU hotplug for x86 and x86_64 > > Not really useful on current hardware, but these provide > infrastructure which some power management patches need, and it seems > sensible to make the reference architecture support hotplug. Will merge. Yay. I'm not going to use it yet, but know Pavel wants it for the next one. > swsusp-on-SMP > > Will merge. > > kexec and kdump > > I guess we should merge these. > > I'm still concerned that the various device shutdown problems will > mean that the success rate for crashing kernels is not high enough for > kdump to be considered a success. In which case in six months time we'll > hear rumours about vendors shipping wholly different crashdump > implementations, which would be quite bad. > > But I think this has gone as far as it can go in -mm, so it's a bit of > a punt. No potential clashes with suspend code, I assume? > execute-in-place > > Will merge. Have the embedded guys commented on the usefulness of > this for execute-out-of-ROM? Switch roles for a mo and put my Cyclades hat on. Probably not useful to us at the moment, at least in the case of the products I work on. Marcelo? Regards, Nigel ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (Suspend-to-disk) 2005-06-21 13:08 ` -mm -> 2.6.13 merge status (Suspend-to-disk) Nigel Cunningham @ 2005-06-21 9:44 ` Marcelo Tosatti 2005-06-21 14:16 ` Pavel Machek 1 sibling, 0 replies; 620+ messages in thread From: Marcelo Tosatti @ 2005-06-21 9:44 UTC (permalink / raw) To: Nigel Cunningham Cc: Andrew Morton, Linux Kernel Mailing List, Tim Bird, Russell King > > execute-in-place > > > > Will merge. Have the embedded guys commented on the usefulness of > > this for execute-out-of-ROM? > > Switch roles for a mo and put my Cyclades hat on. Probably not useful to > us at the moment, at least in the case of the products I work on. > Marcelo? Well yes, its definately very useful for embedded folks where RAM is a precious resource (not our case at the moment). I'm not aware of any users of this XIP implementation, maybe Tim Bird or Russell have reviewed/tested it? It went through filesystem folks reviewing (and I'm pretty sure akpm knows about that already)... Hope to be helpful. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (Suspend-to-disk) 2005-06-21 13:08 ` -mm -> 2.6.13 merge status (Suspend-to-disk) Nigel Cunningham 2005-06-21 9:44 ` Marcelo Tosatti @ 2005-06-21 14:16 ` Pavel Machek 1 sibling, 0 replies; 620+ messages in thread From: Pavel Machek @ 2005-06-21 14:16 UTC (permalink / raw) To: Nigel Cunningham Cc: Andrew Morton, Linux Kernel Mailing List, Marcelo Tosatti Hi! > > kexec and kdump > > > > I guess we should merge these. > > ... > No potential clashes with suspend code, I assume? > I test suspend in -mm series from time to time, and it seems ok; so this one should be safe. Pavel -- 64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: v9fs (-mm -> 2.6.13 merge status) 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (5 preceding siblings ...) 2005-06-21 13:08 ` -mm -> 2.6.13 merge status (Suspend-to-disk) Nigel Cunningham @ 2005-06-21 13:51 ` Eric Van Hensbergen 2005-06-21 15:35 ` Uriel [not found] ` <a4e6962a05062112206e823c0a@mail.gmail.com> 2005-06-21 14:08 ` -mm -> 2.6.13 merge status Martin Hicks ` (11 subsequent siblings) 18 siblings, 2 replies; 620+ messages in thread From: Eric Van Hensbergen @ 2005-06-21 13:51 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel On 6/21/05, Andrew Morton <akpm@osdl.org> wrote: > > v9fs > > I'm not sure that this has a sufficiently high > usefulness-to-maintenance-cost ratio. > I think v9fs/9P has some unique aspects which differentiate it from the other distributed system protocols integrated into Linux: a) it presents a unified distributed resource sharing protocol. It will be able to distribute devices, file systems, system services, and application interfaces. b) it provides non-caching RPC-style access to synthetic file systems which could be used with in-kernel file systems such as sysfs or with user-space synthetics such as those provided by FUSE c) its implementation supports transport independence enabling easy support for different interconnects (shared memory, Xen device channels, RDMA, Infiniband, etc.) v9fs-2.0 has a somewhat limited audience at the moment - but now that the initial implementation is more or less complete we are working to build applications on top of it (and provide a better server). It's being integrated into cluster projects at LANL and being looked at wrt virtualization I/O at IBM. Its our hope that these improvements and cluster applications will motivate more wide-spread use of the v9fs module. -eric ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: v9fs (-mm -> 2.6.13 merge status) 2005-06-21 13:51 ` v9fs (-mm -> 2.6.13 merge status) Eric Van Hensbergen @ 2005-06-21 15:35 ` Uriel 2005-06-22 5:13 ` Martin Atkins [not found] ` <a4e6962a05062112206e823c0a@mail.gmail.com> 1 sibling, 1 reply; 620+ messages in thread From: Uriel @ 2005-06-21 15:35 UTC (permalink / raw) To: linux-kernel On Tue, Jun 21, 2005 at 08:51:27AM -0500, Eric Van Hensbergen wrote: > On 6/21/05, Andrew Morton <akpm@osdl.org> wrote: > > > > v9fs > > > > I'm not sure that this has a sufficiently high > > usefulness-to-maintenance-cost ratio. The 9P protocol implemented by v9fs is the result of over a decade of research in distributed systems at Bell Labs by the original Unix team, and it has various implementations for other operating systems that have been used in production systems for many years. 9P is designed to be portable across systems and transport protocols, it's network transparent, and it gives us interoperativity with Inferno(which can run hosted under Linux already), Plan 9, and p9p, and implementations for *BSD and other systems are in the works. 9P has the potential to become the standard protocol for distributed resources and I don't think any of the alternatives come anywhere near being as well designed, well proven and encompassing. uriel ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: v9fs (-mm -> 2.6.13 merge status) 2005-06-21 15:35 ` Uriel @ 2005-06-22 5:13 ` Martin Atkins 0 siblings, 0 replies; 620+ messages in thread From: Martin Atkins @ 2005-06-22 5:13 UTC (permalink / raw) To: linux-kernel Uriel <uriell <at> binarydream.org> writes: >.. > The 9P protocol implemented by v9fs is the result of over a decade of > research in distributed systems at Bell Labs by the original Unix team, I would second that. There are several other filesystems already merged or under discussion that are superficially similar - fuse, coda, and even nfs! However, v9fs is the only one that is (all of) 1) suitable for synthetic filesystems (resources), as well as 'normal' filesystems 2) equally good for local and remote filesystems 3) OS/machine independent "on-the-wire" 4) (network) transport independent 5) has a significant history of deployment If there are few Linux applications using v9fs as yet (but there is plan9ports), then one must admit that there is something of a chicken-and-eggs situation involved! Martin ^ permalink raw reply [flat|nested] 620+ messages in thread
[parent not found: <a4e6962a05062112206e823c0a@mail.gmail.com>]
* Re: Fwd: v9fs (-mm -> 2.6.13 merge status) [not found] ` <a4e6962a05062112206e823c0a@mail.gmail.com> @ 2005-06-21 20:52 ` Ronald G. Minnich 0 siblings, 0 replies; 620+ messages in thread From: Ronald G. Minnich @ 2005-06-21 20:52 UTC (permalink / raw) To: linux-kernel; +Cc: Eric Van Hensbergen On Tue, 21 Jun 2005, Eric Van Hensbergen wrote: > On 6/21/05, Andrew Morton <akpm@osdl.org> wrote: > > > > v9fs > > > > I'm not sure that this has a sufficiently high > > usefulness-to-maintenance-cost ratio. > > I got pointed at this discussion. Here are my $.02 on why we at LANL are interested in v9fs. We build clusters on the order of 2000 machines at present, with larger systems coming along. The system which we use to run these clusters is bproc. While bproc has proven to be very powerful to date, it does have its limits: - requires homogenous system - the network protocols it uses, while simple, are somewhat ad-hoc (as is common in this type of system) - if you are on a bproc system as user x, using 25% of the system, you still see 100% of the processes. This is a bit of a security issue. We have a desire to build single-system-image looking clusters along the bproc model, but at the same time compose those clusters of, e.g., Opterons and G5s. This mixing is highly desirable for compoutations that have phases, some of which belong on one type of a machine, and some on another. We are going to use v9fs as the glue for our next-generation cluster software, called 'xcpu'. Xcpu has been implemented on Plan 9 and works there. I have ported xcpu to Linux, using v9fs as the client side and Russ Cox's plan9ports server to write servers. xcpu presents a remote execution service as a 9p server. xcpu has been tested across architectures and it works very well. By summer 2006, we hope to have cut over our bproc systems to xcpu. That's one use for v9fs. We also plan to use v9fs to provide us with servers for global /proc, monitoring, and control systems for our clusters. The global /proc is interesting. bproc provides a global /proc, but it is incomplete; entries for, e.g., exe and maps are not filled in. bproc also caches part of the /proc, but the rules about what is cached and what the timeouts are, are set in the kernel module and not easily changed. We are going to have an "aggregating" user level 9p server based on Mirtchovskis's aggrfs, which will both aggregate all the cluster nodes, and have caching rules that make sense in clusters of 1000s of node (for example, it is ok to cache /proc/x/status; there is no need to cache /proc/x/maps, and you probably don't want to anyway). A neat capability is that if we give a user, e.g., 25% of the cluster, we can tailor that user's name space so that they only see their procs and the 25% of the cluster they own. This is good for security, but also good for convenience: most users don't really care that some other user is on 75% of the cluster. Global pid spaces are neat in theory, messy in practice at large scale. I want my global pid space to be global to *me*, meaning I see the global space of the nodes I care about. The sysadmin, of course, wants to see everything. All this is possible. V9fs, along with Linux private name spaces, will allow us to provide this model: users can see some or all of the global pid space, depending on need; users can be constrained to only see part of the global pid space, depending on other issues. 9p will also replace the Supermon protocol, allowing people to easily view status information in a file system. In addition to the cluster usage, there is also grid usage. The 9grid, composed of plan 9 systems, is connected by 9p servers. Linux systems can join the 9grid with no problem, once Linux has v9fs. Were v9fs just a file system, I would not really be interested in it one way or another; we have NFS, after all. But v9fs is really the key piece of a new model of cluster services we are building at LANL. 9p will be the glue, and v9fs will be the needed client side for hooking 9p servers into the file system name space. I'm hoping we can see v9fs in the kernel someday. thanks ron ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (6 preceding siblings ...) 2005-06-21 13:51 ` v9fs (-mm -> 2.6.13 merge status) Eric Van Hensbergen @ 2005-06-21 14:08 ` Martin Hicks 2005-06-21 19:54 ` Andrew Morton 2005-06-21 14:19 ` -mm -> 2.6.13 merge status (wireless) Pavel Machek ` (10 subsequent siblings) 18 siblings, 1 reply; 620+ messages in thread From: Martin Hicks @ 2005-06-21 14:08 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel On Mon, Jun 20, 2005 at 11:54:58PM -0700, Andrew Morton wrote: > > vm-early-zone-reclaim > > Needs some convincing benchmark numbers to back it up. Otherwise OK. The only benchmarks I have for this were included in my last mail to linux-mm: http://marc.theaimsgroup.com/?l=linux-mm&m=111763597218177&w=2 Are they convincing? Well, the patch doesn't seem to make the memory thrashing case much worse ("make -j" kernbench run) which is a good thing since the VM is trying to reclaim much earlier. In the same e-mail I mention that there is a fairly good performance gain in the optimal case, where processes are tied to a single node and the node's memory is filled with page cache. With zone reclaim turned on the "make -j8" kernel build runs in 700 seconds; 735 seconds with no reclaim. mh -- Martin Hicks Wild Open Source Inc. mort@wildopensource.com 613-266-2296 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 14:08 ` -mm -> 2.6.13 merge status Martin Hicks @ 2005-06-21 19:54 ` Andrew Morton 2005-06-21 20:00 ` Martin Hicks 0 siblings, 1 reply; 620+ messages in thread From: Andrew Morton @ 2005-06-21 19:54 UTC (permalink / raw) To: Martin Hicks; +Cc: linux-kernel Martin Hicks <mort@wildopensource.com> wrote: > > On Mon, Jun 20, 2005 at 11:54:58PM -0700, Andrew Morton wrote: > > > > vm-early-zone-reclaim > > > > Needs some convincing benchmark numbers to back it up. Otherwise OK. > > The only benchmarks I have for this were included in my last mail to > linux-mm: > > http://marc.theaimsgroup.com/?l=linux-mm&m=111763597218177&w=2 > > Are they convincing? Well, the patch doesn't seem to make the memory > thrashing case much worse ("make -j" kernbench run) which is a good > thing since the VM is trying to reclaim much earlier. > > In the same e-mail I mention that there is a fairly good performance > gain in the optimal case, where processes are tied to a single node and > the node's memory is filled with page cache. With zone reclaim turned > on the "make -j8" kernel build runs in 700 seconds; 735 seconds with > no reclaim. Ah, OK, I failed to capture that info. (I always have to move the info in the [patch 0/n] email into the first real patch, and this time I didn't) Thanks. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 19:54 ` Andrew Morton @ 2005-06-21 20:00 ` Martin Hicks 0 siblings, 0 replies; 620+ messages in thread From: Martin Hicks @ 2005-06-21 20:00 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel On Tue, Jun 21, 2005 at 12:54:57PM -0700, Andrew Morton wrote: > > Ah, OK, I failed to capture that info. (I always have to move the info in > the [patch 0/n] email into the first real patch, and this time I didn't) Oops. I'll try to remember to stick the benchmark info into one of the real patches next time. mh -- Martin Hicks Wild Open Source Inc. mort@wildopensource.com 613-266-2296 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (wireless) 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (7 preceding siblings ...) 2005-06-21 14:08 ` -mm -> 2.6.13 merge status Martin Hicks @ 2005-06-21 14:19 ` Pavel Machek 2005-06-21 20:02 ` Andrew Morton 2005-06-21 15:26 ` -mm -> 2.6.13 merge status Jeff Garzik ` (9 subsequent siblings) 18 siblings, 1 reply; 620+ messages in thread From: Pavel Machek @ 2005-06-21 14:19 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel, jbenc Hi! > This summarises my current thinking on various patches which are presently > in -mm. I cover large things and small-but-controversial things. Anything > which isn't covered here (and that's a lot of material) is probably a "will > merge", unless it obviously isn't. I'd like to ask about 802.11 stack and ipw2100 in particular... Is it "small enough that it did not need mentioning"? Working wireless in mainline would be great... Pavel -- 64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (wireless) 2005-06-21 14:19 ` -mm -> 2.6.13 merge status (wireless) Pavel Machek @ 2005-06-21 20:02 ` Andrew Morton 2005-06-22 3:26 ` Jeff Garzik 0 siblings, 1 reply; 620+ messages in thread From: Andrew Morton @ 2005-06-21 20:02 UTC (permalink / raw) To: Pavel Machek; +Cc: linux-kernel, jbenc, Jeff Garzik Pavel Machek <pavel@ucw.cz> wrote: > > Hi! > > > This summarises my current thinking on various patches which are presently > > in -mm. I cover large things and small-but-controversial things. Anything > > which isn't covered here (and that's a lot of material) is probably a "will > > merge", unless it obviously isn't. > > I'd like to ask about 802.11 stack and ipw2100 in particular... Is it > "small enough that it did not need mentioning"? > Working wireless in mainline would be great... That's up to Jeff. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (wireless) 2005-06-21 20:02 ` Andrew Morton @ 2005-06-22 3:26 ` Jeff Garzik 0 siblings, 0 replies; 620+ messages in thread From: Jeff Garzik @ 2005-06-22 3:26 UTC (permalink / raw) To: Andrew Morton; +Cc: Pavel Machek, linux-kernel, jbenc Andrew Morton wrote: > Pavel Machek <pavel@ucw.cz> wrote: > >>Hi! >> >> >>>This summarises my current thinking on various patches which are presently >>>in -mm. I cover large things and small-but-controversial things. Anything >>>which isn't covered here (and that's a lot of material) is probably a "will >>>merge", unless it obviously isn't. >> >>I'd like to ask about 802.11 stack and ipw2100 in particular... Is it >>"small enough that it did not need mentioning"? >>Working wireless in mainline would be great... > > > That's up to Jeff. 802.11 stack is still too ipw-specific. Someone needs to get together another driver using 802.11 stack (such as HostAP, the original location of much of the code). So, the merge criteria is: something other than ipw uses it. Otherwise, it'll never be generic... Jeff, who has several SuSE wireless patches to merge still ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (8 preceding siblings ...) 2005-06-21 14:19 ` -mm -> 2.6.13 merge status (wireless) Pavel Machek @ 2005-06-21 15:26 ` Jeff Garzik 2005-06-21 15:39 ` Robert Love ` (4 more replies) 2005-06-21 15:50 ` Lee Revell ` (8 subsequent siblings) 18 siblings, 5 replies; 620+ messages in thread From: Jeff Garzik @ 2005-06-21 15:26 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel Andrew Morton wrote: > git-ocfs > > The OCFS2 filesystem. OK by me, although I'm not sure it's had enough > review. Every time I come up with a complaint about ocfs2, the Oracle guys manage to shoot me down. I think it's OK to merge. > sparsemem > > OK by me for a merge. Need to poke arch maintainers first, check that > they've looked at it sufficiently closely. seems sane, though there are some whitespace niggles that should be cleaned up > rapidio-* > > Will merge. send through netdev, as is proper > dlm-*.patch: Red Hat distributed lock manager > > Hard. Right now it seems that no in-kernel projects will use this and > only one out-of-kernel project will use it. Shelve the problem until > after Kernel Summit, where some light may be shed. > > Opinions are sought... I hate to say it, since its my own employer, but I agree: wait for in-kernel users, at the very least. > connector.patch > > Nice idea IMO, but there are still questions around the > implementation. More dialogue needed ;) > > connector-add-a-fork-connector.patch > > OK, but needs connector. I don't like connector > inotify > > There are still concerns about the userspace API and internal > implementation details. More slogging needed. We should ask hpa what he needs for kernel.org. Ideally kernel.org probably wants <something> that facilitates listening to <something> for a list of files being changed. That would greatly speed up the robots, and possibly rsync-like activities too. > pcmcia-*.patch > > Makes the pcmcia layer generate hotplug events and deprecates cardmgr. > Will merge. Testing? The goal behind the patch is certainly good, but I worry about exposure. > cachefs > > This is a ton of code which knows rather a lot about pagecache > internals. It allows the AFS client to cache file contents on a local > blockdev. > > I don't think it's a justified addition for only AFS and I'd prefer to > see it proven for NFS as well. > > Issues around add-page-becoming-writable-notification.patch need to > be resolved. > > cachefs-for-nfs > > A recent addition. Needs review from NFS developers and considerably > more testing. > > These things aren't looking likely for 2.6.13. If I could vote more than once, I would! I really like cachefs, and have been pushing for its inclusion for a while. > kexec and kdump > > I guess we should merge these. > > I'm still concerned that the various device shutdown problems will > mean that the success rate for crashing kernels is not high enough for > kdump to be considered a success. In which case in six months time we'll > hear rumours about vendors shipping wholly different crashdump > implementations, which would be quite bad. > > But I think this has gone as far as it can go in -mm, so it's a bit of > a punt. I'm not particularly pleased with these, and indeed vendors ARE shipping other crashdump methods. > reiser4 > > Merge it, I guess. > > The patches still contain all the reiser4-specific namespace > enhancements, only it is disabled, so it is effectively dead code. Maybe > we should ask that it actually be removed? The plugin stuff is crap. This is not a filesystem but a filesystem + new layer. IMO considered in that light, it duplicates functionality elsewhere. > v9fs > > I'm not sure that this has a sufficiently high > usefulness-to-maintenance-cost ratio. agreed (though I think 9P is neat) > It has been said that a userspace NFS server can be used to get > full NFS server functionality with FUSE. I think the half-assed kernel > implementation should be done away with. "It has been said" -- its true. A userspace NFS server can do 100% of userspace FS functionality. Jeff ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 15:26 ` -mm -> 2.6.13 merge status Jeff Garzik @ 2005-06-21 15:39 ` Robert Love 2005-06-21 19:22 ` Christoph Lameter 2005-06-21 15:43 ` Matt Porter ` (3 subsequent siblings) 4 siblings, 1 reply; 620+ messages in thread From: Robert Love @ 2005-06-21 15:39 UTC (permalink / raw) To: Jeff Garzik; +Cc: Andrew Morton, linux-kernel On Tue, 2005-06-21 at 11:26 -0400, Jeff Garzik wrote: > > inotify > > > > There are still concerns about the userspace API and internal > > implementation details. More slogging needed. > > We should ask hpa what he needs for kernel.org. Ideally kernel.org > probably wants <something> that facilitates listening to <something> for > a list of files being changed. That would greatly speed up the robots, > and possibly rsync-like activities too. I've talked to some people who've hooked inotify into rsync successfully. Cool hack. Robert Love ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 15:39 ` Robert Love @ 2005-06-21 19:22 ` Christoph Lameter 2005-06-21 19:38 ` Robert Love 2005-06-21 22:45 ` Jeff Garzik 0 siblings, 2 replies; 620+ messages in thread From: Christoph Lameter @ 2005-06-21 19:22 UTC (permalink / raw) To: Robert Love; +Cc: Jeff Garzik, Andrew Morton, linux-kernel On Tue, 21 Jun 2005, Robert Love wrote: > > We should ask hpa what he needs for kernel.org. Ideally kernel.org > > probably wants <something> that facilitates listening to <something> for > > a list of files being changed. That would greatly speed up the robots, > > and possibly rsync-like activities too. > > I've talked to some people who've hooked inotify into rsync > successfully. Cool hack. I noticed that select() is not working on real files. Could inotify be used to fix select()? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 19:22 ` Christoph Lameter @ 2005-06-21 19:38 ` Robert Love 2005-06-21 19:44 ` Christoph Lameter 2005-06-21 20:02 ` Zan Lynx 2005-06-21 22:45 ` Jeff Garzik 1 sibling, 2 replies; 620+ messages in thread From: Robert Love @ 2005-06-21 19:38 UTC (permalink / raw) To: Christoph Lameter; +Cc: Jeff Garzik, Andrew Morton, linux-kernel On Tue, 2005-06-21 at 12:22 -0700, Christoph Lameter wrote: > I noticed that select() is not working on real files. Could inotify > be used to fix select()? Select the system call? It should work fine. ;-) Who is confused? Robert Love ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 19:38 ` Robert Love @ 2005-06-21 19:44 ` Christoph Lameter 2005-06-21 20:02 ` Zan Lynx 1 sibling, 0 replies; 620+ messages in thread From: Christoph Lameter @ 2005-06-21 19:44 UTC (permalink / raw) To: Robert Love; +Cc: Jeff Garzik, Andrew Morton, linux-kernel On Tue, 21 Jun 2005, Robert Love wrote: > On Tue, 2005-06-21 at 12:22 -0700, Christoph Lameter wrote: > > > I noticed that select() is not working on real files. Could inotify > > be used to fix select()? > > Select the system call? It should work fine. ;-) Hmmm. I just wrote an app that uses select to do essentially a "tail" waiting for new content in a log file. The file descriptors for real disk files are always ready even if there is no content available for the application. The file is positioned at the end of the file after open via lseek. select tells me that data is available but the read() returns zero bytes. The current fix on the app level is to checking if useful work was done as a result of "READY" file descriptors. If the read() operations do not return any data then the app will simply sleep for a couple of seconds. So the app degenerates to a kind of poll mode if disk files are used. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 19:38 ` Robert Love 2005-06-21 19:44 ` Christoph Lameter @ 2005-06-21 20:02 ` Zan Lynx 2005-06-21 20:06 ` Christoph Lameter 2005-06-21 20:52 ` Stephen Hemminger 1 sibling, 2 replies; 620+ messages in thread From: Zan Lynx @ 2005-06-21 20:02 UTC (permalink / raw) To: Robert Love; +Cc: Christoph Lameter, Jeff Garzik, Andrew Morton, linux-kernel [-- Attachment #1: Type: text/plain, Size: 631 bytes --] On Tue, 2005-06-21 at 15:38 -0400, Robert Love wrote: > On Tue, 2005-06-21 at 12:22 -0700, Christoph Lameter wrote: > > > I noticed that select() is not working on real files. Could inotify > > be used to fix select()? > > Select the system call? It should work fine. ;-) > > Who is confused? > > Robert Love Sounds interesting. tail -f could use it. Instead of sleep 1, seek to current position, read to eof; just select() for read on the file and sleep in select() until someone else writes to that file. I've never tried doing that. It might work, for all I know. -- Zan Lynx <zlynx@acm.org> [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:02 ` Zan Lynx @ 2005-06-21 20:06 ` Christoph Lameter 2005-06-21 20:07 ` Robert Love 2005-06-21 22:54 ` Alan Cox 2005-06-21 20:52 ` Stephen Hemminger 1 sibling, 2 replies; 620+ messages in thread From: Christoph Lameter @ 2005-06-21 20:06 UTC (permalink / raw) To: Zan Lynx; +Cc: Robert Love, Jeff Garzik, Andrew Morton, linux-kernel On Tue, 21 Jun 2005, Zan Lynx wrote: > I've never tried doing that. It might work, for all I know. I was told that Linux cannot do this. It always returns the filehandles as ready for disk files. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:06 ` Christoph Lameter @ 2005-06-21 20:07 ` Robert Love 2005-06-21 20:10 ` Christoph Lameter 2005-06-21 22:54 ` Alan Cox 1 sibling, 1 reply; 620+ messages in thread From: Robert Love @ 2005-06-21 20:07 UTC (permalink / raw) To: Christoph Lameter; +Cc: Zan Lynx, Jeff Garzik, Andrew Morton, linux-kernel On Tue, 2005-06-21 at 13:06 -0700, Christoph Lameter wrote: > On Tue, 21 Jun 2005, Zan Lynx wrote: > > > I've never tried doing that. It might work, for all I know. > > I was told that Linux cannot do this. It always returns the filehandles as > ready for disk files. Inotify would definitely work. Robert Love ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:07 ` Robert Love @ 2005-06-21 20:10 ` Christoph Lameter 2005-06-21 20:15 ` Zan Lynx 2005-06-22 5:53 ` Matthias Urlichs 0 siblings, 2 replies; 620+ messages in thread From: Christoph Lameter @ 2005-06-21 20:10 UTC (permalink / raw) To: Robert Love; +Cc: Zan Lynx, Jeff Garzik, Andrew Morton, linux-kernel On Tue, 21 Jun 2005, Robert Love wrote: > > I was told that Linux cannot do this. It always returns the filehandles as > > ready for disk files. > > Inotify would definitely work. Well we could use it in kernel to make select() work correctly. For disk files set up a notification for write and then only return from select if new data is available. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:10 ` Christoph Lameter @ 2005-06-21 20:15 ` Zan Lynx 2005-06-22 5:53 ` Matthias Urlichs 1 sibling, 0 replies; 620+ messages in thread From: Zan Lynx @ 2005-06-21 20:15 UTC (permalink / raw) To: Christoph Lameter; +Cc: Robert Love, Jeff Garzik, Andrew Morton, linux-kernel [-- Attachment #1: Type: text/plain, Size: 511 bytes --] On Tue, 2005-06-21 at 13:10 -0700, Christoph Lameter wrote: > On Tue, 21 Jun 2005, Robert Love wrote: > > > > I was told that Linux cannot do this. It always returns the filehandles as > > > ready for disk files. > > > > Inotify would definitely work. > > Well we could use it in kernel to make select() work correctly. For disk > files set up a notification for write and then only return from select if > new data is available. You could do it inside glibc. -- Zan Lynx <zlynx@acm.org> [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:10 ` Christoph Lameter 2005-06-21 20:15 ` Zan Lynx @ 2005-06-22 5:53 ` Matthias Urlichs 1 sibling, 0 replies; 620+ messages in thread From: Matthias Urlichs @ 2005-06-22 5:53 UTC (permalink / raw) To: linux-kernel Hi, Christoph Lameter wrote: > Well we could use it in kernel to make select() work correctly. select() already works correctly. It answers the "will I not block if I call read()/write() on this" question, and since you never block on files (assuming infinite disk speed ;-) select() will always return True on it. You can't change this, it's in POSIX. ... or maybe I misunderstood your comment. -- Matthias Urlichs | {M:U} IT Design @ m-u-it.de | smurf@smurf.noris.de Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de - - "I don't really miss God but i sure miss Santa Claus!" [Courtney Love] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:06 ` Christoph Lameter 2005-06-21 20:07 ` Robert Love @ 2005-06-21 22:54 ` Alan Cox 1 sibling, 0 replies; 620+ messages in thread From: Alan Cox @ 2005-06-21 22:54 UTC (permalink / raw) To: Christoph Lameter Cc: Zan Lynx, Robert Love, Jeff Garzik, Andrew Morton, Linux Kernel Mailing List On Maw, 2005-06-21 at 21:06, Christoph Lameter wrote: > On Tue, 21 Jun 2005, Zan Lynx wrote: > > > I've never tried doing that. It might work, for all I know. > > I was told that Linux cannot do this. It always returns the filehandles as > ready for disk files. That is because disk files are always ready - select/poll are for waits for data (or space) to become available not for events in the sense of inotify. That said there *is* scope in the poll() API [but not select()] to add a new kind of poll notification type. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:02 ` Zan Lynx 2005-06-21 20:06 ` Christoph Lameter @ 2005-06-21 20:52 ` Stephen Hemminger 1 sibling, 0 replies; 620+ messages in thread From: Stephen Hemminger @ 2005-06-21 20:52 UTC (permalink / raw) To: linux-kernel On Tue, 21 Jun 2005 14:02:09 -0600 Zan Lynx <zlynx@acm.org> wrote: > On Tue, 2005-06-21 at 15:38 -0400, Robert Love wrote: > > On Tue, 2005-06-21 at 12:22 -0700, Christoph Lameter wrote: > > > > > I noticed that select() is not working on real files. Could inotify > > > be used to fix select()? > > > > Select the system call? It should work fine. ;-) > > > > Who is confused? > > > > Robert Love > > Sounds interesting. tail -f could use it. Instead of sleep 1, seek to > current position, read to eof; just select() for read on the file and > sleep in select() until someone else writes to that file. > > I've never tried doing that. It might work, for all I know. > -- > Zan Lynx <zlynx@acm.org> Posix requires select() of regular files always return true: http://www.opengroup.org/onlinepubs/009695399/functions/select.html File descriptors associated with regular files shall always select true for ready to read, ready to write, and error conditions. -- Stephen Hemminger <shemminger@osdl.org> ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 19:22 ` Christoph Lameter 2005-06-21 19:38 ` Robert Love @ 2005-06-21 22:45 ` Jeff Garzik 2005-06-21 23:30 ` Christoph Lameter 1 sibling, 1 reply; 620+ messages in thread From: Jeff Garzik @ 2005-06-21 22:45 UTC (permalink / raw) To: Christoph Lameter; +Cc: Robert Love, Andrew Morton, linux-kernel Christoph Lameter wrote: > On Tue, 21 Jun 2005, Robert Love wrote: > > >>>We should ask hpa what he needs for kernel.org. Ideally kernel.org >>>probably wants <something> that facilitates listening to <something> for >>>a list of files being changed. That would greatly speed up the robots, >>>and possibly rsync-like activities too. >> >>I've talked to some people who've hooked inotify into rsync >>successfully. Cool hack. > > > I noticed that select() is not working on real files. Could inotify > be used to fix select()? Non-blocking file I/O is an open issue. AIO is probably a better path. Jeff ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 22:45 ` Jeff Garzik @ 2005-06-21 23:30 ` Christoph Lameter 2005-06-21 23:39 ` Jeff Garzik 0 siblings, 1 reply; 620+ messages in thread From: Christoph Lameter @ 2005-06-21 23:30 UTC (permalink / raw) To: Jeff Garzik; +Cc: Robert Love, Andrew Morton, linux-kernel On Tue, 21 Jun 2005, Jeff Garzik wrote: > Non-blocking file I/O is an open issue. > > AIO is probably a better path. AIO is requiring you to poll and check if I/O is complete. select() does not require any polling and just needs to be made to work the way it was intended to. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 23:30 ` Christoph Lameter @ 2005-06-21 23:39 ` Jeff Garzik 2005-06-22 6:19 ` Christoph Lameter 0 siblings, 1 reply; 620+ messages in thread From: Jeff Garzik @ 2005-06-21 23:39 UTC (permalink / raw) To: Christoph Lameter; +Cc: Robert Love, Andrew Morton, linux-kernel Christoph Lameter wrote: > On Tue, 21 Jun 2005, Jeff Garzik wrote: > > >>Non-blocking file I/O is an open issue. >> >>AIO is probably a better path. > > > AIO is requiring you to poll and check if I/O is complete. select() does Incorrect. The entire point of AIO is that its an async callback system, when the I/O is complete... just like the kernel's internal I/O request queue system. Jeff ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 23:39 ` Jeff Garzik @ 2005-06-22 6:19 ` Christoph Lameter 0 siblings, 0 replies; 620+ messages in thread From: Christoph Lameter @ 2005-06-22 6:19 UTC (permalink / raw) To: Jeff Garzik; +Cc: Robert Love, Andrew Morton, linux-kernel On Tue, 21 Jun 2005, Jeff Garzik wrote: > > AIO is requiring you to poll and check if I/O is complete. select() does > > Incorrect. The entire point of AIO is that its an async callback system, when > the I/O is complete... just like the kernel's internal I/O request queue > system. Hmmm.. Okay it may work like dnotify. You get some signal and then its up to you to figure out what was going on. Traditionally select() does that all for you and tells you which stream got input. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 15:26 ` -mm -> 2.6.13 merge status Jeff Garzik 2005-06-21 15:39 ` Robert Love @ 2005-06-21 15:43 ` Matt Porter 2005-06-21 19:41 ` randy_dunlap ` (2 subsequent siblings) 4 siblings, 0 replies; 620+ messages in thread From: Matt Porter @ 2005-06-21 15:43 UTC (permalink / raw) To: Jeff Garzik; +Cc: Andrew Morton, linux-kernel On Tue, Jun 21, 2005 at 11:26:44AM -0400, Jeff Garzik wrote: > Andrew Morton wrote: > > rapidio-* > > > > Will merge. > > send through netdev, as is proper rapidio-support-net-driver.patch is the only netdev portion. -Matt ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 15:26 ` -mm -> 2.6.13 merge status Jeff Garzik 2005-06-21 15:39 ` Robert Love 2005-06-21 15:43 ` Matt Porter @ 2005-06-21 19:41 ` randy_dunlap 2005-06-21 20:05 ` Hans Reiser 2005-06-21 20:22 ` -mm -> 2.6.13 merge status Andrew Morton 4 siblings, 0 replies; 620+ messages in thread From: randy_dunlap @ 2005-06-21 19:41 UTC (permalink / raw) To: Jeff Garzik; +Cc: akpm, linux-kernel On Tue, 21 Jun 2005 11:26:44 -0400 Jeff Garzik wrote: | Andrew Morton wrote: | | > connector.patch | > | > Nice idea IMO, but there are still questions around the | > implementation. More dialogue needed ;) | > | > connector-add-a-fork-connector.patch | > | > OK, but needs connector. | | I don't like connector can you be more specific, like you did with reiser4? | > kexec and kdump | > | > I guess we should merge these. | > | > I'm still concerned that the various device shutdown problems will | > mean that the success rate for crashing kernels is not high enough for | > kdump to be considered a success. In which case in six months time we'll | > hear rumours about vendors shipping wholly different crashdump | > implementations, which would be quite bad. | > | > But I think this has gone as far as it can go in -mm, so it's a bit of | > a punt. | | I'm not particularly pleased with these, and indeed vendors ARE shipping | other crashdump methods. any specifics on the "not particularly pleased" part? | > reiser4 | > | > Merge it, I guess. | > | > The patches still contain all the reiser4-specific namespace | > enhancements, only it is disabled, so it is effectively dead code. Maybe | > we should ask that it actually be removed? | | The plugin stuff is crap. This is not a filesystem but a filesystem + | new layer. IMO considered in that light, it duplicates functionality | elsewhere. I don't think that r4 is just a filesystem either, but you know more about that than I do. thanks, --- ~Randy ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 15:26 ` -mm -> 2.6.13 merge status Jeff Garzik ` (2 preceding siblings ...) 2005-06-21 19:41 ` randy_dunlap @ 2005-06-21 20:05 ` Hans Reiser 2005-06-21 20:24 ` Christoph Hellwig 2005-06-21 20:22 ` -mm -> 2.6.13 merge status Andrew Morton 4 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-21 20:05 UTC (permalink / raw) To: Jeff Garzik; +Cc: Andrew Morton, linux-kernel Jeff Garzik wrote: > > >> reiser4 >> >> > > > The plugin stuff is crap. This is not a filesystem but a filesystem + > new layer. IMO considered in that light, it duplicates functionality > elsewhere. What functionality where? Please remember that this is per file, per item, per node, per attribute, per disk format, per bitmap, per super block, etc., abstracting, not per filesystem abstracting. Plugins allow a number of things: 1) They allow us to never pay the cost to change the disk format again, no matter how much we add in future years. This really matters: the prohibitive cost of disk format changes are the number one impediment to filesystem improvements, and the primary reason why most filesystems ossify after time has past. 2) They allow us to easily structure code for reuse. If we want to create a new kind of file that is like some other kind of file except for one thing, we just write the one thing, and then easily reuse all the other code, and create a new plugin id. The use of plugins forced all the programmers to think about reusability at every layer of design. V3 of reiserfs is way too hard to work on and modify. If you ask one of the team to code something for V3 instead of V4, they quietly groan at the thought. It is just so much easier to do in V4. When I asked one of our team to completely change the key assignment algorithm for V4 (which controls what things get packed near what in the tree), he complained that it would take 6 weeks to do it. Under V3 it would have taken 3 months. It took him 3 days, and now to change it again would take him 3 hours I think. Oh, by the way, the change boosted performance dramatically. If you take the time to become familiar with coding within our plugin layer, I think you will find yourself wanting the same to exist for other filesystems. Of course, no other filesystem needs to be impacted by our plugin layer, and there is no way reiser4 could easily be rewritten to exist without it (it would be like requiring that the kernel not use function calls and only use gotos). Reiser4's plugin layer has as its explicit objective making it possible for the weekend hacker to add something useful to reiser4 and send it in for inclusion. We want to democratize filesystem innovation so that random bright guys who usually work on something other than filesystems can act on their bright ideas without spending 3 years in the filesystem field to do it. I believe that most great filesystem innovations are envisioned by persons not working on filesystems, and go nowhere because of the especially high cost of entry into our field. I am not as bright as other filesystem developers, and so we must tinker with three ideas and keep one of them. The speed of tinkering is crucial to us, and the plugin layer increases that speed several fold. Please reconsider your view. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:05 ` Hans Reiser @ 2005-06-21 20:24 ` Christoph Hellwig 2005-06-22 1:07 ` reiser4 plugins Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-06-21 20:24 UTC (permalink / raw) To: Hans Reiser; +Cc: Jeff Garzik, Andrew Morton, linux-kernel Hans, we had this discussion before. And the consensus was pretty simple: the disk structure plugins are your business and fine to keep. The higher-level pluging that just add another useless abstraction below file_operation/inode_operation/etc. are not. keep the former and kill the latter and you're one step further. ^ permalink raw reply [flat|nested] 620+ messages in thread
* reiser4 plugins 2005-06-21 20:24 ` Christoph Hellwig @ 2005-06-22 1:07 ` Hans Reiser 2005-06-22 1:14 ` Jeff Garzik ` (2 more replies) 0 siblings, 3 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-22 1:07 UTC (permalink / raw) To: Christoph Hellwig; +Cc: Jeff Garzik, Andrew Morton, linux-kernel, ReiserFS List Christoph, Reiser4 users love the plugin concept, and all audiences which have listened to a presentation on plugins have been quite positive about it. Many users think it is the best thing about reiser4. Can you articulate why you are opposed to plugins in more detail? Perhaps you are simply not as familiar with it as the audiences I have presented to. Perhaps persons on our mailing list can comment..... In particular, what is wrong with having a plugin id associated with every file, storing the pluginid on disk in permanent storage in the stat data, and having that plugin id define the set of methods that implement the vfs operations associated with a particular file, rather than defining VFS methods only at filesystem granularity? What is wrong with having an encryption plugin implemented in this manner? What is wrong with being able to have some files implemented using a compression plugin, and others in the same filesystem not. What is wrong with having one file in the FS use a write only plugin, in which the encrypion key is changed with every append in a forward but not backward computable manner, and in order to read a file you must either have a key that is stored on another computer or be reading what was written after the moment of cracking root? What is wrong with having a set of critical data files use a CRC checking file plugin? What we have hurts no one but us. I have never seen an audience for one of my talks that thought it hurt us..... most audiences think it is great. Let us tinker with our FS, and you tinker with yours, and so long as what we do does not affect your FS, let the users choose. In the end, somebody will write a new fs that steals the good ideas from both of us, and obsoletes us both. They can only do this though if we are left to be both free to implement differing filesystem designs. I do not tell you how to design XFS, why are you making my life unpleasant? Christoph Hellwig wrote: >Hans, we had this discussion before. And the consensus was pretty simple: >the disk structure plugins are your business and fine to keep. The >higher-level pluging that just add another useless abstraction below >file_operation/inode_operation/etc. are not. keep the former and kill >the latter and you're one step further. > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 1:07 ` reiser4 plugins Hans Reiser @ 2005-06-22 1:14 ` Jeff Garzik 2005-06-22 4:25 ` David Masover 2005-06-22 14:24 ` Alexander Zarochentsev 2005-06-22 1:18 ` Andrew Morton 2005-06-22 5:34 ` Christoph Hellwig 2 siblings, 2 replies; 620+ messages in thread From: Jeff Garzik @ 2005-06-22 1:14 UTC (permalink / raw) To: Hans Reiser; +Cc: Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hans Reiser wrote: > Christoph, > > Reiser4 users love the plugin concept, and all audiences which have > listened to a presentation on plugins have been quite positive about > it. Many users think it is the best thing about reiser4. Can you > articulate why you are opposed to plugins in more detail? Perhaps you > are simply not as familiar with it as the audiences I have presented > to. Perhaps persons on our mailing list can comment..... > > In particular, what is wrong with having a plugin id associated with > every file, storing the pluginid on disk in permanent storage in the > stat data, and having that plugin id define the set of methods that > implement the vfs operations associated with a particular file, rather > than defining VFS methods only at filesystem granularity? You're basically implementing another VFS layer inside of reiser4, which is a big layering violation. This sort of feature should -not- be done at the low-level filesystem level. What happens if people decide plugins are a good idea, and they want them in ext3? We need massive surgery to extract the guts from reiser4. Jeff ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 1:14 ` Jeff Garzik @ 2005-06-22 4:25 ` David Masover 2005-06-22 5:11 ` Bedros Hanounik ` (4 more replies) 2005-06-22 14:24 ` Alexander Zarochentsev 1 sibling, 5 replies; 620+ messages in thread From: David Masover @ 2005-06-22 4:25 UTC (permalink / raw) To: Jeff Garzik Cc: Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jeff Garzik wrote: > Hans Reiser wrote: > >> Christoph, >> >> Reiser4 users love the plugin concept, and all audiences which have >> listened to a presentation on plugins have been quite positive about >> it. Many users think it is the best thing about reiser4. Can you >> articulate why you are opposed to plugins in more detail? Perhaps you >> are simply not as familiar with it as the audiences I have presented >> to. Perhaps persons on our mailing list can comment..... >> >> In particular, what is wrong with having a plugin id associated with >> every file, storing the pluginid on disk in permanent storage in the >> stat data, and having that plugin id define the set of methods that >> implement the vfs operations associated with a particular file, rather >> than defining VFS methods only at filesystem granularity? > > > You're basically implementing another VFS layer inside of reiser4, which > is a big layering violation. There's been sloppy code in the kernel before. I remember one bit in particular which was commented "Fuck me gently with a chainsaw." If I remember correctly, this had all of the PCI ids and the names and manufacturers of the corresponding devices -- in a data structure -- in C source code. It was something like one massive array definition, or maybe it was a structure. I don't remember exactly, but... The point is, this was in the kernel for quite awhile, and it was so ugly that someone would rather be fucked with a chainsaw. If something that bad can make it in the kernel and stay for awhile because it worked, and no one wanted to replace it -- nowdays, that database is kept in userland as some nice text format -- then I vote for putting Reiser4 in the kernel now, and ironing the sloppiness ("violation") out later. It runs now. > This sort of feature should -not- be done at the low-level filesystem > level. I agree there, too. In fact, some people have suggested that all "legacy" (read: non-reiser) filesystems should be implemented as Reiser4 plugins, effectively killing VFS.* So, Reiser4 may eventually take over VFS and be the only Linux filesystem, but if so, it will have to do it much more slowly. Take the good ideas -- things like plugins -- and make them at least look like incremental updates to the current VFS, and make them available to all filesystems. Eventually, this would result in a full merge of Reiser and Linux, such that the only thing left of "Reiser4" are a few plugins -- things like storage plugins and maybe some more exotic things like fibration that I don't really understand. > What happens if people decide plugins are a good idea, and they want > them in ext3? We need massive surgery to extract the guts from reiser4. And here is the crucial point. Reiser4 is usable and useful NOW, not after it has undergone massive surgery, and Namesys is bankrupt, and users have given up and moved on to XFS. But the massive surgery should happen eventually, partly to make all filesystems better (see below), and partly to make the transition easier and more palatable for those who don't work for Namesys. * Imagine ext3 as a storage-level plugin for reiser4. You get one benefit immediately -- lazy allocation. Lazy allocation is nice both for fragmentation and for busy systems writing and nuking a lot of temporary files. Imagine a file which is written and then deleted before it hits disk -- it should never touch the disk, regardless of the underlying storage layout. Other benefits are subtler but inevitable. Right now, it would be an understatement to say that there's duplication of effort between ext3, xfs, and reiser4. And yet, I don't think there are many core design decisions that influence my decision as to which I pick up. I just want it to be fast, stable, and have a stable on-disk format so I don't have to reformat too often. I honestly don't care about how XFS is segmenting the disk, or Reiser4 packs really well, or ext3 can be read as ext2 and flushes to disk every five seconds. These are the kinds of things which should be set to sane defaults, tunable for enterprise users, but are not really enough to warrant completely separate code bases. I would imagine that it wouldn't be too long after this before an uber-fs rose, something which combined enough of the strong points of all the existing Linux filesystems that no one would use anything else. But, Linux still needs support for FAT32, ISO9660, UDF, and all the other filesystems which won't go away as easily as XFS and ext3, and it would be nice if these could all share as much code as possible. I don't know if storage plugins really work that way, but they should. I think. I don't work here. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrjoNHgHNmZLgCUhAQIYYw/7BWZ0gVvy0ln5tRo405yUoRHJ/jVFBGyP 5pQ610ECMZORVWRO1qP/NXbvGZwDjEggM5iIeahsGqnBWzNGDsB6RslMUZAxzCYy iAovi0881zoARf3dmhKrDgbGkvNLPTx+/ypa20oRcHLnyI+NAjerUxNuvGc79PJn Ybm9JhX6tQsqGIKjZye9uNDHCifLqzA1gdxucPwWo9sz4ymzM9FgsMGvb+IxrU50 2a2G2K6AHcH+nkomEHw3xgY3PmUZUy0s2hQDSkJx6dCido7fGZwwykaSXm4IZs9s xZqBxKx32G/LDnDV3W8gpj8ZisUE+58kefRbIjo4Ml6IzgXvQqpRjaQOuz8JoKDX 9KUV43tcZkPpK+neIYPQYpXCrdMQqB5+ISpbNKuz/Z/abkcVw1042sy0EKM+/VnD n3Jr7PpSyk0lfCyADr+33qnqPQRFq02gQTohg9FheoMthhV01aaeGW5ExmTM/Wwa 6Dmv/qnn2oNImi+Ebz5u3Lbnz7lL3MVpRL4aoMmEOVUB3xhehnxesf//yacBmwj9 M/3KVae9epwX4K6yi8qQXzH4160IBFHpWUxBLc9LnOZlHQuZ+Fz3BIrbKQBvmaHT 7lrwi0Os6TmiBGMSFd6OUWHcYs4p97aMw30NG33fCL6e8X6oNVFwwnJXZpBPN1Mr +lrsVAywKEI= =rHdK -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 4:25 ` David Masover @ 2005-06-22 5:11 ` Bedros Hanounik 2005-06-22 5:18 ` Jeff Garzik ` (3 subsequent siblings) 4 siblings, 0 replies; 620+ messages in thread From: Bedros Hanounik @ 2005-06-22 5:11 UTC (permalink / raw) To: David Masover Cc: Jeff Garzik, Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 7801 bytes --] First of all, I'm HW engineer, and don't know much about implementation details of FS. I know some coding, I've coded for different levels from linux device drivers to GUI's, but I know a lot about integration between levels of hierarchy. I generally agree with the idea that high level functionality such as compressing, encrypting, ...etc should not be implemented in the FS. But, the term "high-level" is a relative term and three years down the road, what we consider high-level now becomes low-level. my suggestion is to keep everything as is. if Hans and the reiser4 guys want to implement high level stuff as plug-ins that's fine. If the VFS guys want to implement the same functionality in VFS on top of the system FS; they can do it too. BUT PLEASE MAKE THE API COMPATIBLE!!!! programmers on both sides should discuss the features and make sure the API is compatible. in such a scenario, if I want to compress a directory called /mystuff/data on a reiser4 FS, I could call the VFS API to do so; The VFS finds that the system is reiser4 and compression is supported, so it passes the call to reiser4. On another hand, if the system is ext3 and it doens't support compression, VFS performs the task and passes the file to ext3. My point is that the effect of gravity is upon us, high level stuff will become low-level in a short time. Moreover, at the low level in FS and especially when done by FS guys, functions such as compression can be done more efficiently than in VFS space. my worries are that the waste of effort by implementing the same functionality in two levels in the hierarchy, and incompatible API can cause serious headache. just my two cents. -Bedros On 6/21/05, David Masover <ninja@slaphack.com> wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Jeff Garzik wrote: > > Hans Reiser wrote: > > > >> Christoph, > >> > >> Reiser4 users love the plugin concept, and all audiences which have > >> listened to a presentation on plugins have been quite positive about > >> it. Many users think it is the best thing about reiser4. Can you > >> articulate why you are opposed to plugins in more detail? Perhaps you > >> are simply not as familiar with it as the audiences I have presented > >> to. Perhaps persons on our mailing list can comment..... > >> > >> In particular, what is wrong with having a plugin id associated with > >> every file, storing the pluginid on disk in permanent storage in the > >> stat data, and having that plugin id define the set of methods that > >> implement the vfs operations associated with a particular file, rather > >> than defining VFS methods only at filesystem granularity? > > > > > > You're basically implementing another VFS layer inside of reiser4, which > > is a big layering violation. > > There's been sloppy code in the kernel before. I remember one bit in > particular which was commented "Fuck me gently with a chainsaw." If I > remember correctly, this had all of the PCI ids and the names and > manufacturers of the corresponding devices -- in a data structure -- in > C source code. It was something like one massive array definition, or > maybe it was a structure. I don't remember exactly, but... > > The point is, this was in the kernel for quite awhile, and it was so > ugly that someone would rather be fucked with a chainsaw. If something > that bad can make it in the kernel and stay for awhile because it > worked, and no one wanted to replace it -- nowdays, that database is > kept in userland as some nice text format -- then I vote for putting > Reiser4 in the kernel now, and ironing the sloppiness ("violation") out > later. It runs now. > > > This sort of feature should -not- be done at the low-level filesystem > > level. > > I agree there, too. In fact, some people have suggested that all > "legacy" (read: non-reiser) filesystems should be implemented as Reiser4 > plugins, effectively killing VFS.* > > So, Reiser4 may eventually take over VFS and be the only Linux > filesystem, but if so, it will have to do it much more slowly. Take the > good ideas -- things like plugins -- and make them at least look like > incremental updates to the current VFS, and make them available to all > filesystems. > > Eventually, this would result in a full merge of Reiser and Linux, such > that the only thing left of "Reiser4" are a few plugins -- things like > storage plugins and maybe some more exotic things like fibration that I > don't really understand. > > > What happens if people decide plugins are a good idea, and they want > > them in ext3? We need massive surgery to extract the guts from reiser4. > > And here is the crucial point. Reiser4 is usable and useful NOW, not > after it has undergone massive surgery, and Namesys is bankrupt, and > users have given up and moved on to XFS. But the massive surgery should > happen eventually, partly to make all filesystems better (see below), > and partly to make the transition easier and more palatable for those > who don't work for Namesys. > > > > > > * Imagine ext3 as a storage-level plugin for reiser4. You get one > benefit immediately -- lazy allocation. Lazy allocation is nice both > for fragmentation and for busy systems writing and nuking a lot of > temporary files. Imagine a file which is written and then deleted > before it hits disk -- it should never touch the disk, regardless of the > underlying storage layout. > > Other benefits are subtler but inevitable. Right now, it would be an > understatement to say that there's duplication of effort between ext3, > xfs, and reiser4. And yet, I don't think there are many core design > decisions that influence my decision as to which I pick up. I just want > it to be fast, stable, and have a stable on-disk format so I don't have > to reformat too often. I honestly don't care about how XFS is > segmenting the disk, or Reiser4 packs really well, or ext3 can be read > as ext2 and flushes to disk every five seconds. These are the kinds of > things which should be set to sane defaults, tunable for enterprise > users, but are not really enough to warrant completely separate code > bases. > > I would imagine that it wouldn't be too long after this before an > uber-fs rose, something which combined enough of the strong points of > all the existing Linux filesystems that no one would use anything else. > But, Linux still needs support for FAT32, ISO9660, UDF, and all the > other filesystems which won't go away as easily as XFS and ext3, and it > would be nice if these could all share as much code as possible. > > > I don't know if storage plugins really work that way, but they should. > > I think. I don't work here. > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.1 (GNU/Linux) > Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org > > iQIVAwUBQrjoNHgHNmZLgCUhAQIYYw/7BWZ0gVvy0ln5tRo405yUoRHJ/jVFBGyP > 5pQ610ECMZORVWRO1qP/NXbvGZwDjEggM5iIeahsGqnBWzNGDsB6RslMUZAxzCYy > iAovi0881zoARf3dmhKrDgbGkvNLPTx+/ypa20oRcHLnyI+NAjerUxNuvGc79PJn > Ybm9JhX6tQsqGIKjZye9uNDHCifLqzA1gdxucPwWo9sz4ymzM9FgsMGvb+IxrU50 > 2a2G2K6AHcH+nkomEHw3xgY3PmUZUy0s2hQDSkJx6dCido7fGZwwykaSXm4IZs9s > xZqBxKx32G/LDnDV3W8gpj8ZisUE+58kefRbIjo4Ml6IzgXvQqpRjaQOuz8JoKDX > 9KUV43tcZkPpK+neIYPQYpXCrdMQqB5+ISpbNKuz/Z/abkcVw1042sy0EKM+/VnD > n3Jr7PpSyk0lfCyADr+33qnqPQRFq02gQTohg9FheoMthhV01aaeGW5ExmTM/Wwa > 6Dmv/qnn2oNImi+Ebz5u3Lbnz7lL3MVpRL4aoMmEOVUB3xhehnxesf//yacBmwj9 > M/3KVae9epwX4K6yi8qQXzH4160IBFHpWUxBLc9LnOZlHQuZ+Fz3BIrbKQBvmaHT > 7lrwi0Os6TmiBGMSFd6OUWHcYs4p97aMw30NG33fCL6e8X6oNVFwwnJXZpBPN1Mr > +lrsVAywKEI= > =rHdK > -----END PGP SIGNATURE----- > [-- Attachment #2: Type: text/html, Size: 8655 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 4:25 ` David Masover 2005-06-22 5:11 ` Bedros Hanounik @ 2005-06-22 5:18 ` Jeff Garzik 2005-06-22 8:27 ` Hans Reiser ` (2 more replies) 2005-06-22 5:36 ` reiser4 plugins Christoph Hellwig ` (2 subsequent siblings) 4 siblings, 3 replies; 620+ messages in thread From: Jeff Garzik @ 2005-06-22 5:18 UTC (permalink / raw) To: David Masover Cc: Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover wrote: > There's been sloppy code in the kernel before. I remember one bit in > particular which was commented "Fuck me gently with a chainsaw." If I > remember correctly, this had all of the PCI ids and the names and > manufacturers of the corresponding devices -- in a data structure -- in > C source code. It was something like one massive array definition, or > maybe it was a structure. I don't remember exactly, but... > > The point is, this was in the kernel for quite awhile, and it was so > ugly that someone would rather be fucked with a chainsaw. If something > that bad can make it in the kernel and stay for awhile because it > worked, and no one wanted to replace it -- nowdays, that database is > kept in userland as some nice text format -- then I vote for putting > Reiser4 in the kernel now, and ironing the sloppiness ("violation") out > later. It runs now. Existence of ugly code is not an excuse to add more. We have to maintain said ugly code for decades. Maintainability is a big deal when you deal with the timeframes we deal with. > So, Reiser4 may eventually take over VFS and be the only Linux > filesystem, but if so, it will have to do it much more slowly. Take the > good ideas -- things like plugins -- and make them at least look like > incremental updates to the current VFS, and make them available to all > filesystems. This is how all Linux development is done. The code evolves over time. You have just described the path ReiserFS needs to take, in order to get this stuff into the kernel, in fact. > And here is the crucial point. Reiser4 is usable and useful NOW, not > after it has undergone massive surgery, and Namesys is bankrupt, and > users have given up and moved on to XFS. But the massive surgery should > happen eventually, partly to make all filesystems better (see below), > and partly to make the transition easier and more palatable for those > who don't work for Namesys. We care about technical merit, not some random company's financial solvancy. Reiser4 has layering violations, and doesn't work with the current security layer. Those are two biggies. There is no entitlement to be merged in the upstream kernel. If people don't like how the Linux kernel is managed, they are free to maintain their own fork. If its better, people will use that instead. > I would imagine that it wouldn't be too long after this before an > uber-fs rose, something which combined enough of the strong points of > all the existing Linux filesystems that no one would use anything else. > But, Linux still needs support for FAT32, ISO9660, UDF, and all the > other filesystems which won't go away as easily as XFS and ext3, and it > would be nice if these could all share as much code as possible. > > > I don't know if storage plugins really work that way, but they should. No, they shouldn't. > I think. I don't work here. Obviously. Jeff ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 5:18 ` Jeff Garzik @ 2005-06-22 8:27 ` Hans Reiser 2005-06-22 9:08 ` David Masover 2005-06-23 5:58 ` Hans Reiser 2 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-22 8:27 UTC (permalink / raw) To: Jeff Garzik Cc: David Masover, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Jeff Garzik wrote: >> after it has undergone massive surgery, and Namesys is bankrupt, and >> users have given up and moved on to XFS. But the massive surgery should >> happen eventually, partly to make all filesystems better (see below), >> and partly to make the transition easier and more palatable for those >> who don't work for Namesys. > > And here is the crucial point. Reiser4 is usable and useful NOW, not > > We care about technical merit, not some random company's financial > solvancy. > > > Hmm, that must be why Daniel Robbins, the creator of the distro with the technically superior Gentoo approach, is still with us, yes? If you cared about technical merit, you would discuss benchmarks, yes? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 5:18 ` Jeff Garzik 2005-06-22 8:27 ` Hans Reiser @ 2005-06-22 9:08 ` David Masover 2005-06-22 14:28 ` Nikita Danilov 2005-06-26 17:02 ` Christoph Hellwig 2005-06-23 5:58 ` Hans Reiser 2 siblings, 2 replies; 620+ messages in thread From: David Masover @ 2005-06-22 9:08 UTC (permalink / raw) To: Jeff Garzik Cc: Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jeff Garzik wrote: > David Masover wrote: > >> There's been sloppy code in the kernel before. I remember one bit in >> particular which was commented "Fuck me gently with a chainsaw." If I >> remember correctly, this had all of the PCI ids and the names and >> manufacturers of the corresponding devices -- in a data structure -- in >> C source code. It was something like one massive array definition, or >> maybe it was a structure. I don't remember exactly, but... >> >> The point is, this was in the kernel for quite awhile, and it was so >> ugly that someone would rather be fucked with a chainsaw. If something >> that bad can make it in the kernel and stay for awhile because it >> worked, and no one wanted to replace it -- nowdays, that database is >> kept in userland as some nice text format -- then I vote for putting >> Reiser4 in the kernel now, and ironing the sloppiness ("violation") out >> later. It runs now. > > > Existence of ugly code is not an excuse to add more. Maybe not, but I'm making a point. I'm sure that, at the time, people wanted something that ran. They wanted lspci to work. It was generally assumed that it would be cleaned up later, and it was. Too much later, but it happened, eventually. I've been reading a bit of history, and the reason Linux got so popular in the first place was the tendency to include stuff that worked and provided a feature people wanted, even if it was ugly. The philosophy would be: choose a good implementation over an ugly one, but choose an ugly one over nothing at all. > We have to maintain said ugly code for decades. Maintainability is a > big deal when you deal with the timeframes we deal with. Maintainability is like optimization. The maintainability of a non-working program is irrelevant. You'd be right if we already had plugins-in-the-VFS. We don't. The most maintainable solution for plugins-in-the-FS that actually exists is Reiser4, exactly as it is now - -- because it is the _only_ one that actually exists right now. >> So, Reiser4 may eventually take over VFS and be the only Linux >> filesystem, but if so, it will have to do it much more slowly. Take the >> good ideas -- things like plugins -- and make them at least look like >> incremental updates to the current VFS, and make them available to all >> filesystems. > > > This is how all Linux development is done. > > The code evolves over time. > > You have just described the path ReiserFS needs to take, in order to get > this stuff into the kernel, in fact. This is the path it needs to take, long term, for the sanity of everyone. But I don't think it can get there without being included, short term. People will stomp all over any attempts to change the VFS as "unproven" and "unneccesary". Do you think it will be any easier to get this stuff into the kernel that way? Better, I think, to drop it in, as-is, and move stuff incrementally from the FS to the VFS. That way, there's at least something intermediate for people to use, test, and hammer/beat down, and for maintainers to get more comfortable with the idea and the logistics of this beast in their kernel. >> And here is the crucial point. Reiser4 is usable and useful NOW, not >> after it has undergone massive surgery, and Namesys is bankrupt, and >> users have given up and moved on to XFS. But the massive surgery should >> happen eventually, partly to make all filesystems better (see below), >> and partly to make the transition easier and more palatable for those >> who don't work for Namesys. > > > We care about technical merit, not some random company's financial > solvancy. Reiser4 has layering violations, and doesn't work with the > current security layer. Those are two biggies. Ah, so you mean to say, you care about how well something fits the current model. That has little to do with technical merit. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrkqoHgHNmZLgCUhAQIKixAAnTJTJJpuCLatFp8syccjNnE7WdHlO/zx G1bWuSthCnb7uaVb8buDeAlpArzttoguKKum0NE0khz/FjKw4YUXH4AEsVYGlZaO nBYpw0MVyypNP+hZhEuo1T826frEOb6Z40Y1WZCpYwAZCs9EQQm7TeYSMjhD17Ew PehYwUFUmnv1S7CpYNQvuboKh/1wuUQb6R2thjuCGJVkif8Mn2U20Fhk1/HIgnIr OHoCD8ZgwoqBDPKQ6V26dUX+ZHzQVJX1j/IgLnnitJ9E4quIHNs33lq4S99DWta6 uDS4hkHgFMRemh37sA0FUMeitFsrwNb2b2f0o/X8MpDJmwbdrdg9kxvwfHqqgaz+ Enj0rBXO8E+5a4STTk4L2LaSR2+knSEFdj53MYYq4ABL07hEbJp2cNFKh5AFXvg0 wg5WoHt4HhhOeuftIG9Twv5tHIC5qoM57ae9yZzj783G9ZnXy0xBefUmH+pWVQsp H1IpKIR4a0l8gV1AkJa6BUyAyzDDObFzmqcZ61W15Y2relD9Ow2qzVqMxroB78uJ O+on741BecGtohB5xdfth9rwDY6JPkDug3C6zHzxSAkGGEnWIn6O8CzcGrGqS0Ta EmB4LGw/fZqGcEYOOErqMC6GuImv2JbjtBOx8nAxM2OhGXFoDiD9FQaDaxWw9zjj ROODOhm0aTA= =ivqd -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 9:08 ` David Masover @ 2005-06-22 14:28 ` Nikita Danilov 2005-06-22 16:13 ` Vladimir Saveliev 2005-06-26 17:02 ` Christoph Hellwig 1 sibling, 1 reply; 620+ messages in thread From: Nikita Danilov @ 2005-06-22 14:28 UTC (permalink / raw) To: David Masover Cc: Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover writes: [...] > > Maintainability is like optimization. The maintainability of a > non-working program is irrelevant. You'd be right if we already had > plugins-in-the-VFS. We don't. The most maintainable solution for > plugins-in-the-FS that actually exists is Reiser4, exactly as it is now > - -- because it is the _only_ one that actually exists right now. But it is not so. There _are_ plugins-in-the-VFS. VFS operates on opaque objects (inodes, dentries, file system types) through interfaces: {inode,address_space,dentry,sb,etc.}_operations. Every file system back-end if free to implement whatever number of these interfaces. And the do this already: check the sources; even ext2 does this: e.g., ext2_fast_symlink_inode_operations and ext2_symlink_inode_operations. This is exactly what upper level reiser4 plugins are for. I guess that one of Christoph Hellwig's complaints is that reiser4 introduces another layer of abstraction to implement something that already exists. Nikita. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 14:28 ` Nikita Danilov @ 2005-06-22 16:13 ` Vladimir Saveliev 2005-06-22 16:59 ` Nikita Danilov 0 siblings, 1 reply; 620+ messages in thread From: Vladimir Saveliev @ 2005-06-22 16:13 UTC (permalink / raw) To: Nikita Danilov Cc: David Masover, Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hello On Wed, 2005-06-22 at 18:28, Nikita Danilov wrote: > David Masover writes: > > [...] > > > > > Maintainability is like optimization. The maintainability of a > > non-working program is irrelevant. You'd be right if we already had > > plugins-in-the-VFS. We don't. The most maintainable solution for > > plugins-in-the-FS that actually exists is Reiser4, exactly as it is now > > - -- because it is the _only_ one that actually exists right now. > > But it is not so. There _are_ plugins-in-the-VFS. VFS operates on opaque > objects (inodes, dentries, file system types) through interfaces: > {inode,address_space,dentry,sb,etc.}_operations. Every file system > back-end if free to implement whatever number of these interfaces. And > the do this already: check the sources; even ext2 does this: e.g., > ext2_fast_symlink_inode_operations and ext2_symlink_inode_operations. > imho, this is something different. Ext2 decides itself how to manage a symlink depending on length of string the symlink is to store. Reiser4 plugins are to allow a user to define himself which operations file is to be managed with. > This is exactly what upper level reiser4 plugins are for. > I guess that one of Christoph Hellwig's complaints is that reiser4 > introduces another layer of abstraction to implement something that > already exists. > I do not think that it exists already. You can have standart type of files and that is all. Linux filesystem is not supposed to provide anything but plain regular/directory/symlinks/sockets/block device/char device/fifo files. Existing file API does not allow create anything but that. Merging reiser4 with object plugins would make it necessary to modify VFS layer so that files of arbitrary types could be created. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 16:13 ` Vladimir Saveliev @ 2005-06-22 16:59 ` Nikita Danilov 0 siblings, 0 replies; 620+ messages in thread From: Nikita Danilov @ 2005-06-22 16:59 UTC (permalink / raw) To: Vladimir Saveliev Cc: David Masover, Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Vladimir Saveliev writes: > Hello > > On Wed, 2005-06-22 at 18:28, Nikita Danilov wrote: > > David Masover writes: > > > > [...] > > > > > > > > Maintainability is like optimization. The maintainability of a > > > non-working program is irrelevant. You'd be right if we already had > > > plugins-in-the-VFS. We don't. The most maintainable solution for > > > plugins-in-the-FS that actually exists is Reiser4, exactly as it is now > > > - -- because it is the _only_ one that actually exists right now. > > > > But it is not so. There _are_ plugins-in-the-VFS. VFS operates on opaque > > objects (inodes, dentries, file system types) through interfaces: > > {inode,address_space,dentry,sb,etc.}_operations. Every file system > > back-end if free to implement whatever number of these interfaces. And > > the do this already: check the sources; even ext2 does this: e.g., > > ext2_fast_symlink_inode_operations and ext2_symlink_inode_operations. > > > > imho, this is something different. Ext2 decides itself how to manage a > symlink depending on length of string the symlink is to store. > Reiser4 plugins are to allow a user to define himself which operations > file is to be managed with. I fail to see this as an important distinction: - ext2 decides what "plugin" to use based on parameters supplied to the file creation system call; - reiser4 decides what "plugin" to use based on parameters supplied to the file creation system call. > > > This is exactly what upper level reiser4 plugins are for. > > > I guess that one of Christoph Hellwig's complaints is that reiser4 > > introduces another layer of abstraction to implement something that > > already exists. > > > > I do not think that it exists already. > You can have standart type of files and that is all. > > Linux filesystem is not supposed to provide anything but plain > regular/directory/symlinks/sockets/block device/char device/fifo files. If you are talking about S_IFMT bits they are just a relic to keep user level happy (yes, VFS does few S_ISFOO checks here and there, but they all can be replaced with checks for appropriate operations being non-NULL). Nothing prevents file system from rolling forward whatever exotic objects it wants. > > Existing file API does not allow create anything but that. > Merging reiser4 with object plugins would make it necessary to modify > VFS layer so that files of arbitrary types could be created. > This is important, but it doesn't depend on whether file system has additional layer if indirection below VFS operations. If file system wants to support extended object types, additional parameters have to be somehow communicated from the user level during object creation. But: - this is necessary even if file system has no "plugins" below VFS operations; - this doesn't necessary means changing VFS. For example, these parameters can be set on parent directory (this is how ACLs were integrated into POSIX). > Nikita. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 9:08 ` David Masover 2005-06-22 14:28 ` Nikita Danilov @ 2005-06-26 17:02 ` Christoph Hellwig 2005-06-27 9:30 ` Alexander Zarochentsev 1 sibling, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-06-26 17:02 UTC (permalink / raw) To: David Masover Cc: Jeff Garzik, Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Wed, Jun 22, 2005 at 04:08:49AM -0500, David Masover wrote: > I've been reading a bit of history, and the reason Linux got so popular > in the first place was the tendency to include stuff that worked and > provided a feature people wanted, even if it was ugly. The philosophy > would be: choose a good implementation over an ugly one, but choose an > ugly one over nothing at all. And things change over time. Back in those days the linux codebase was small and it was easy to change things all over the place. These times our codebase is huge, and people that know enough parts of the kernel to do big changes are overloaded with work. Thus we have to set our acceptance criteria a lot higher now - else we'd be totally lost with the current size of the project already. > > We have to maintain said ugly code for decades. Maintainability is a > > big deal when you deal with the timeframes we deal with. > > Maintainability is like optimization. The maintainability of a > non-working program is irrelevant. You'd be right if we already had > plugins-in-the-VFS. We don't. The most maintainable solution for > plugins-in-the-FS that actually exists is Reiser4, exactly as it is now > - -- because it is the _only_ one that actually exists right now. We do have plugins in the VFS, every filesystem is a set of a few of them and some gluecode. <skipping a lot stuff> David and Hans, I've read through my backlog a lot now, and I must say it's pretty pointless - you're discussing lots of highlevel what if and don't actually care about something as boring as actual technical details. Hans has lots of very skillfull technical people like zam and vs, and maybe he should give them some freedom to sort out technical issues for a basic reiser4 merge, and one that is done we can turn back to discussion of advanced features and their implementation, hopefully with a few more arguments on both sides and a real technical discussion. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 17:02 ` Christoph Hellwig @ 2005-06-27 9:30 ` Alexander Zarochentsev 2005-06-27 9:42 ` Christoph Hellwig 2005-06-28 14:01 ` Horst von Brand 0 siblings, 2 replies; 620+ messages in thread From: Alexander Zarochentsev @ 2005-06-27 9:30 UTC (permalink / raw) To: Christoph Hellwig Cc: David Masover, Jeff Garzik, Hans Reiser, Andrew Morton, linux-kernel, ReiserFS List On Sunday 26 June 2005 21:02, Christoph Hellwig wrote: > On Wed, Jun 22, 2005 at 04:08:49AM -0500, David Masover wrote: > > I've been reading a bit of history, and the reason Linux got so popular > > in the first place was the tendency to include stuff that worked and > > provided a feature people wanted, even if it was ugly. The philosophy > > would be: choose a good implementation over an ugly one, but choose an > > ugly one over nothing at all. > > And things change over time. Back in those days the linux codebase was > small and it was easy to change things all over the place. These times > our codebase is huge, and people that know enough parts of the kernel to > do big changes are overloaded with work. Thus we have to set our > acceptance criteria a lot higher now - else we'd be totally lost with > the current size of the project already. > > > > We have to maintain said ugly code for decades. Maintainability is a > > > big deal when you deal with the timeframes we deal with. > > > > Maintainability is like optimization. The maintainability of a > > non-working program is irrelevant. You'd be right if we already had > > plugins-in-the-VFS. We don't. The most maintainable solution for > > plugins-in-the-FS that actually exists is Reiser4, exactly as it is now > > - -- because it is the _only_ one that actually exists right now. > > We do have plugins in the VFS, every filesystem is a set of a few of them > and some gluecode. > > <skipping a lot stuff> > > David and Hans, I've read through my backlog a lot now, and I must say > it's pretty pointless - you're discussing lots of highlevel what if and > don't actually care about something as boring as actual technical details. > > Hans has lots of very skillfull technical people like zam and vs, and maybe > he should give them some freedom to sort out technical issues for a basic > reiser4 merge, and one that is done we can turn back to discussion of > advanced features and their implementation, hopefully with a few more > arguments on both sides and a real technical discussion. Unfortunately, this is not only a technical discussion... it is about linux development model too. Well, about the plugins. We can clean reiser4<->VFS interface up by setting per-vfs-object inode/dentry/super ops instead using of our own dispatcher. So different reiser4 inodes/files will have different i_ops/f_ops. That would be only visible-in-VFS part of reiser4 object plugins. Would the help to solve "reiser4 plugins" question? It is just as other FS do -- procfs has seq_file and sysconfig interfaces below the VFS and l-k people do not complain each day about layering violation ;-) Procfs is taken as an example because it deals with objects of different types, actually anybody may create own procfs objects more or less general way. I don't belive that you want to see all reiser4-specific things as item plugins, disk format plugins in the VFS. Thanks, Alex. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 9:30 ` Alexander Zarochentsev @ 2005-06-27 9:42 ` Christoph Hellwig 2005-06-27 11:28 ` Alexander Zarochentsev 2005-06-28 14:01 ` Horst von Brand 1 sibling, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-06-27 9:42 UTC (permalink / raw) To: Alexander Zarochentsev Cc: David Masover, Jeff Garzik, Hans Reiser, Andrew Morton, linux-kernel, ReiserFS List On Mon, Jun 27, 2005 at 01:30:06PM +0400, Alexander Zarochentsev wrote: > -- procfs has seq_file and sysconfig interfaces below the VFS and l-k people > do not complain each day about layering violation ;-) Procfs is taken as an > example because it deals with objects of different types, actually anybody > may create own procfs objects more or less general way. seq_file actually works at the file_operations level, that's exactly what I'm telling you to do. The old sub-callbacks are on their way out. > I don't belive that you want to see all reiser4-specific things as item > plugins, disk format plugins in the VFS. If you'd read the previous discussions you'd see that no one complained about disk format plugins. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 9:42 ` Christoph Hellwig @ 2005-06-27 11:28 ` Alexander Zarochentsev 2005-06-27 19:22 ` Christoph Hellwig 0 siblings, 1 reply; 620+ messages in thread From: Alexander Zarochentsev @ 2005-06-27 11:28 UTC (permalink / raw) To: Christoph Hellwig; +Cc: reiserfs-list, Hans Reiser, linux-kernel On Monday 27 June 2005 13:42, Christoph Hellwig wrote: > On Mon, Jun 27, 2005 at 01:30:06PM +0400, Alexander Zarochentsev wrote: > > -- procfs has seq_file and sysconfig interfaces below the VFS and l-k > > people do not complain each day about layering violation ;-) Procfs is > > taken as an example because it deals with objects of different types, > > actually anybody may create own procfs objects more or less general way. > > seq_file actually works at the file_operations level, that's exactly > what I'm telling you to do. The old sub-callbacks are on their way out. not exactly. I meant that seq_file has its own VFS-like thing struct seq_operations. So I may assume that having own objects and their operations is allowed. The complains are about adding unnecessary level of indirection in the trivial reiser4 wrappers as reiser4_write(). > > I don't belive that you want to see all reiser4-specific things as item > > plugins, disk format plugins in the VFS. > > If you'd read the previous discussions you'd see that no one complained > about disk format plugins. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 11:28 ` Alexander Zarochentsev @ 2005-06-27 19:22 ` Christoph Hellwig 0 siblings, 0 replies; 620+ messages in thread From: Christoph Hellwig @ 2005-06-27 19:22 UTC (permalink / raw) To: Alexander Zarochentsev Cc: Christoph Hellwig, reiserfs-list, Hans Reiser, linux-kernel On Mon, Jun 27, 2005 at 03:28:49PM +0400, Alexander Zarochentsev wrote: > not exactly. I meant that seq_file has its own VFS-like thing struct > seq_operations. It's not that VFS-like, it's more a set of callback than actual methods. But yes, if you're actually doing work at a significant lower layer adding abstractions make sense. Note that seq_file.c while not beeing the VFS is also a generic library that you can use with any filesystem if you want to implement sequential synthetic files. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 9:30 ` Alexander Zarochentsev 2005-06-27 9:42 ` Christoph Hellwig @ 2005-06-28 14:01 ` Horst von Brand 2005-06-28 18:52 ` Sean 1 sibling, 1 reply; 620+ messages in thread From: Horst von Brand @ 2005-06-28 14:01 UTC (permalink / raw) To: Alexander Zarochentsev Cc: Christoph Hellwig, David Masover, Jeff Garzik, Hans Reiser, Andrew Morton, linux-kernel, ReiserFS List Alexander Zarochentsev <zam@namesys.com> wrote: > On Sunday 26 June 2005 21:02, Christoph Hellwig wrote: [...] > > David and Hans, I've read through my backlog a lot now, and I must say > > it's pretty pointless - you're discussing lots of highlevel what if and > > don't actually care about something as boring as actual technical details. > > > > Hans has lots of very skillfull technical people like zam and vs, and maybe > > he should give them some freedom to sort out technical issues for a basic > > reiser4 merge, and one that is done we can turn back to discussion of > > advanced features and their implementation, hopefully with a few more > > arguments on both sides and a real technical discussion. > Unfortunately, this is not only a technical discussion... it is about linux > development model too. Then better separate the two, keeping the technical discussion on LKML and taking the development model stuff elsewhere. > Well, about the plugins. First step: Axe them. They are (at most) /configuration options/, that will have to get fixed meanings, available to all ReiserFS filesystems purely for compatibility reasons. Sure, do get wild on new options in your own experimental trees, and migrate what survives into the standard version (and then it'll be ReiserFS 4.1, 4.2, ..., 4.25, ...). > We can clean reiser4<->VFS interface up by setting > per-vfs-object inode/dentry/super ops instead using of our own dispatcher. Sounds reasonable. > So different reiser4 inodes/files will have different i_ops/f_ops. That > would be only visible-in-VFS part of reiser4 object plugins. How would that work out from the userland (system call) perspective? How does that get handled from on-disk format? > Would the help to solve "reiser4 plugins" question? It is just as other > FS do -- procfs has seq_file and sysconfig interfaces below the VFS and > l-k people do not complain each day about layering violation ;-) Procfs > is taken as an example because it deals with objects of different types, > actually anybody may create own procfs objects more or less general way. But procfs /is/ quite special, as it is supposed to be a window into the kernel, not real files. Some layering violation is unavoidable there. > I don't belive that you want to see all reiser4-specific things as item > plugins, disk format plugins in the VFS. Only what makes sense. Plus many of those will probably have to go. Decide on /one/ way of doing things, even if not perfect for all uses. Everything else is useless bloat. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 14:01 ` Horst von Brand @ 2005-06-28 18:52 ` Sean 2005-06-28 19:28 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Sean @ 2005-06-28 18:52 UTC (permalink / raw) To: Horst von Brand Cc: Alexander Zarochentsev, Christoph Hellwig, David Masover, Jeff Garzik, Hans Reiser, Andrew Morton, linux-kernel, ReiserFS List On Tue, June 28, 2005 10:01 am, Horst von Brand said: >> I don't belive that you want to see all reiser4-specific things as item >> plugins, disk format plugins in the VFS. > > Only what makes sense. Plus many of those will probably have to go. Decide > on /one/ way of doing things, even if not perfect for all uses. Everything > else is useless bloat. It doesn't seem to be a problem as long as loadable plugins are GPL. How is it useless bloat? It doesn't seem much different from having loadable modules in the security system. Just don't enable Reiser4 in your kernel if you don't want to use it. Sean ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 18:52 ` Sean @ 2005-06-28 19:28 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-28 19:28 UTC (permalink / raw) To: Sean Cc: Horst von Brand, Alexander Zarochentsev, Christoph Hellwig, Jeff Garzik, Hans Reiser, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Sean wrote: > On Tue, June 28, 2005 10:01 am, Horst von Brand said: > >>>I don't belive that you want to see all reiser4-specific things as item >>>plugins, disk format plugins in the VFS. >> >>Only what makes sense. Plus many of those will probably have to go. Decide >>on /one/ way of doing things, even if not perfect for all uses. Everything >>else is useless bloat. > > > It doesn't seem to be a problem as long as loadable plugins are GPL. How "Loadable" how? reiser4 plugins must be compiled in, either directly to the kernel, or part of reiser4.ko The only thing "loadable" about them is that some can be turned on or off for individual files. > is it useless bloat? It doesn't seem much different from having loadable > modules in the security system. Just don't enable Reiser4 in your kernel > if you don't want to use it. This is not getting us anywhere. It seems to be an issue of overall design and neatness at this point. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsGkwXgHNmZLgCUhAQKUlw/6AxzaT+5R7lzhy4d/W55KY7m+8QJA90Ut 3m+E/hiSKykolERURKUoRPEK+jly8DUxLS+UpA5ZQwxzkR4YipMcoW5i3lHyJ9GB Hf9lBtGu0TwCF8W9Qh1Fk/7mwQcvyz6ATUYOYqvSA5Wxfr8BDFdcJUqTwMRCOxrA Fh5oOSBXEhqnSElfvurBkzMNurOmNnkpeZt8rM6qXqeYRRPrmIWmFGvkufoyb/q7 N2wdwtwoubWokl8FLWHQmH0f4pJqJUNXpDQvw67zKi24zOUq8ZZ7xH7EE8vNI9s9 ppVVsrLiAFE2/GeDRIdbQNSuClliq7XZV4AMHkorik7Qe0nIeR4hBr9hYI41l/Ie G3A9yP/69XsOodUWMZB/asxj3zMEhDynz26t98C2S858eRA7EJLcJ+IU1HnxHv7N KUcZLUyMP6DPDcm3r2FtjvwDwneQwk3zBqFE30DgAERZTgZ7RTglVuyBhETryk3m ZIf6bjuwnqn2/TFC7Yqwf9tzHshx91ywkfpOXwc/3IcrKKpfY74Mp4Cj/LkRemoz LnE6DdpjuNpMmwTwV0Vb5FKkfeO2nFkLZ96z8HphNmAAINh7gP3bpUMaODc8fLDG HtpzloYmBa4Yvn9JX+AwFuQStupPmScwa62Gcyv0Zm5EQ6fB1pqY1HdhpLHvXehT l0i+8OO74B0= =NI4M -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 5:18 ` Jeff Garzik 2005-06-22 8:27 ` Hans Reiser 2005-06-22 9:08 ` David Masover @ 2005-06-23 5:58 ` Hans Reiser 2005-06-23 6:26 ` Pekka Enberg ` (2 more replies) 2 siblings, 3 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-23 5:58 UTC (permalink / raw) To: Jeff Garzik Cc: David Masover, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Jeff Garzik wrote: > We have to maintain said ugly code for decades. No you don't, I do. >> filesystem, but if so, it will have to do it much more slowly. Take the >> good ideas -- things like plugins -- and make them at least look like >> incremental updates to the current VFS, and make them available to all >> filesystems. > > So, Reiser4 may eventually take over VFS and be the only Linux > > This is how all Linux development is done. > > The code evolves over time. > > You have just described the path ReiserFS needs to take, in order to > get this stuff into the kernel, in fact. You missed his point. He is saying ext3 code should migrate towards becoming reiser4 plugins, and reiser4 should be merged now so that the migration can get started. I don't really care whether ext3 uses our plugin model. (If it does, fine, please credit me for it though, and consider maybe a thank you. I am happy to thank the XFS team for the idea of delayed allocation....) I am entitled to get some advantage from being the first on the block. Other fs maintainers don't want me to have that entitlement, so they add delays to slow us down. It is odd how Hellwig no longer describes himself as XFS associated. Did SGI and him > There is no entitlement to be merged in the upstream kernel. True, but what is interesting is that you don't think the faster filesystem should be merged, and the others should catch up to it later if they can, instead you think it should be delayed until all of its benefits can be transmitted to the other filesystems first so that everyone can be equal. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 5:58 ` Hans Reiser @ 2005-06-23 6:26 ` Pekka Enberg 2005-06-23 14:11 ` David Masover 2005-06-23 21:41 ` reiser4 plugins Alan Cox 2 siblings, 0 replies; 620+ messages in thread From: Pekka Enberg @ 2005-06-23 6:26 UTC (permalink / raw) To: Hans Reiser Cc: Jeff Garzik, David Masover, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Pekka Enberg Hi Hans, Jeff Garzik wrote: > > We have to maintain said ugly code for decades. On 6/23/05, Hans Reiser <reiser@namesys.com> wrote: > No you don't, I do. Lots of people work on the kernel. If you wish to keep reiser4 maintenance to yourself, you probably need to keep it as a separate patch. Please consider submitting the non-controversial bits for merging first and then continue with the rest. It does not make a lot of sense reviewing the current patchkit as many parts of it won't be merged as is (e.g. the plugin stuff). Pekka ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 5:58 ` Hans Reiser 2005-06-23 6:26 ` Pekka Enberg @ 2005-06-23 14:11 ` David Masover 2005-06-23 19:24 ` Horst von Brand 2005-06-23 21:41 ` reiser4 plugins Alan Cox 2 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-23 14:11 UTC (permalink / raw) To: Hans Reiser Cc: Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hans Reiser wrote: > Jeff Garzik wrote: > > >>We have to maintain said ugly code for decades. > > > No you don't, I do. > > >>>filesystem, but if so, it will have to do it much more slowly. Take the >>>good ideas -- things like plugins -- and make them at least look like >>>incremental updates to the current VFS, and make them available to all >>>filesystems. >> >>So, Reiser4 may eventually take over VFS and be the only Linux >> >>This is how all Linux development is done. >> >>The code evolves over time. >> >>You have just described the path ReiserFS needs to take, in order to >>get this stuff into the kernel, in fact. > > > You missed his point. He is saying ext3 code should migrate towards > becoming reiser4 plugins, and reiser4 should be merged now so that the > migration can get started. Sort of. I think ext3 would be nice as a reiser4 plugin. Not everyone will want to reformat at once, but as the reiser4 code matures and proves itself (even more than it already has), the ext3 people may find themselves wanting some of the more generic optimizations. But, I don't think that will realistically happen at all. Instead, what will probably happen is that once Reiser4 is in the mainstream kernel, it will become more popular and noticable. Other FSes will take note. ext3 people may decide they want file-as-directory, and vfat people may decide they want cryptocompress, and so on. In order to do this, they can work with Namesys to port whatever feature they need at the time to the standard VFS. Eventually, with all the features ported, we end up with a situation where there may be no meaningful difference between a filesystem and a low-level reiser4 plugin. At that point, we both win. ext3 will effectively be reiser4 plugins, and reiser4 will replace VFS, but incrementally and sanely, without breaking old filesystems or depriving them of features. This is indeed the way to evolve the code slowly over time. This way also has the advantage of avoiding yet another, even more vicious flamefest if all the reiser4 functionality had to be ported to VFS all at once. Can any of you honestly say that you would be more willing to accept a brand-new, relatively unproven filesystem with tons of changes to the VFS than without? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrrDBHgHNmZLgCUhAQJN/A/6A4SqpsunKmj6NOKxqyMxkLVdMjxSuusa V9XZIL6K0dDsF1FmvxsSVnl3so8u93sRCkxrBs3WQs9JqsA3f8O5VUvsdfqas/Fb ZO+ht5n0r1BVfm1WJHHmR9xGm+jL2TMwVlVrcrrnBQLthq+9efggn5r9BXkOUPgz 57VMh6sjB9ioigmc2Wf6BsMlbBJpCz7EB91pcIrNzHdFOEWyuHm9mAP6ui2sppfK bGaDlZcU4BzNUM8beakMkFuy7JCTFQj/O7s3G42Tqpi/4Ol1Vk7F2tgPkpxB0lHm b/hCmCPbTHheRsDhT7gSHOzWrwA5pwKlT5nO4EWGq7DxZo2dgoYsa/2fP32uYPIX r3vUfygk7MR59DKMi4LLcDy2OzvrtbwTkzgOVrw82NCi5J5/pu/gIHH1aHzkXM6T YRd6G39+wmhN0n/MP3WAT7o3rbH4nuPiG8ZyZQyVTII9YRaOGwNLSjJKe1p9JFHY Y+6bqK3h7LQrX4MfTSIu8k3m0iIt806uHP3GlV+HHKx/N/4HSxt6ikikI3jURO4z XBr23/Uo3YNK+IuxQtPw2mKcdqqKhFq5OBmLkVFIzJyuh8KR7cfnFdl7luCTjFGL KjtgZ39SwtjOUWKWJslkcrNwESqplMTwAaB6GPHiJRKbilaCM9GXm/teI0ZRCl+o gBD0AdEBA9s= =NJM0 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 14:11 ` David Masover @ 2005-06-23 19:24 ` Horst von Brand 2005-06-23 20:12 ` Adrian Ulrich ` (2 more replies) 0 siblings, 3 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-23 19:24 UTC (permalink / raw) To: David Masover Cc: Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover <ninja@slaphack.com> wrote: > Hans Reiser wrote: > > Jeff Garzik wrote: [...] > > You missed his point. He is saying ext3 code should migrate towards > > becoming reiser4 plugins, and reiser4 should be merged now so that the > > migration can get started. > Sort of. > > I think ext3 would be nice as a reiser4 plugin. What for? It works just fine as it stands, AFAICS. > Not everyone will want > to reformat at once, but as the reiser4 code matures and proves itself > (even more than it already has), I for one have seen mainly people with wild claims that it will make their machines much faster, and coming back later asking how they can recover their thrashed partitions... > the ext3 people may find themselves > wanting some of the more generic optimizations. They'll filch them in due time, don't worry. > But, I don't think that will realistically happen at all. > > Instead, what will probably happen is that once Reiser4 is in the > mainstream kernel, it will become more popular and noticable. Other > FSes will take note. ext3 people may decide they want > file-as-directory, That idea is even much older than Linux itself, and no other (Unix) filesystem has implemented it. Ever. Wonder why... > and vfat people may decide they want cryptocompress, I'm sure they don't, because it is mostly for Windows and assorted devices (pendrives, digital cameras, ...) compatibility. > and so on. In order to do this, they can work with Namesys to port > whatever feature they need at the time to the standard VFS. I'm sure they are grateful for the offer. > Eventually, with all the features ported, we end up with a situation > where there may be no meaningful difference between a filesystem and a > low-level reiser4 plugin. Could very well take decades, if ever. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 19:24 ` Horst von Brand @ 2005-06-23 20:12 ` Adrian Ulrich 2005-06-23 22:04 ` David Masover 2005-06-28 13:54 ` cryptocompress [was Re: reiser4 plugins] Pavel Machek 2 siblings, 0 replies; 620+ messages in thread From: Adrian Ulrich @ 2005-06-23 20:12 UTC (permalink / raw) To: Horst von Brand Cc: ninja, reiser, jgarzik, hch, akpm, linux-kernel, reiserfs-list > > Not everyone will want > > to reformat at once, but as the reiser4 code matures and proves itself > > (even more than it already has), > > I for one have seen mainly people with wild claims that it will make their > machines much faster, and coming back later asking how they can recover > their thrashed partitions... Then please show us some Links/Message-IDs to such postings. I'd like to read them. >From my POV: I've been using Reiser4 for almost everything (Rootfs / External Harddrives) for about ~8 Months without any data loss.. Powerloss, unpluging the Disk while writing, full filesystem, heavy use : No problems with reiser4.. It *is* stable. -- Adrian ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-23 20:12 ` Adrian Ulrich 0 siblings, 0 replies; 620+ messages in thread From: Adrian Ulrich @ 2005-06-23 20:12 UTC (permalink / raw) To: Horst von Brand Cc: ninja, reiser, jgarzik, hch, akpm, linux-kernel, reiserfs-list > > Not everyone will want > > to reformat at once, but as the reiser4 code matures and proves itself > > (even more than it already has), > > I for one have seen mainly people with wild claims that it will make their > machines much faster, and coming back later asking how they can recover > their thrashed partitions... Then please show us some Links/Message-IDs to such postings. I'd like to read them. From my POV: I've been using Reiser4 for almost everything (Rootfs / External Harddrives) for about ~8 Months without any data loss.. Powerloss, unpluging the Disk while writing, full filesystem, heavy use : No problems with reiser4.. It *is* stable. -- Adrian ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 20:12 ` Adrian Ulrich (?) @ 2005-06-23 20:49 ` Michael Dreher 2005-06-23 21:54 ` M. ` (2 more replies) -1 siblings, 3 replies; 620+ messages in thread From: Michael Dreher @ 2005-06-23 20:49 UTC (permalink / raw) To: linux-kernel Cc: Adrian Ulrich, Horst von Brand, ninja, reiser, jgarzik, hch, akpm, reiserfs-list >>> Not everyone will want >>> to reformat at once, but as the reiser4 code matures and proves itself >>> (even more than it already has), >> >> I for one have seen mainly people with wild claims that it will make >> their machines much faster, and coming back later asking how they can >> recover their thrashed partitions... > > Then please show us some Links/Message-IDs to such postings. > I'd like to read them. Here you are.... The following happened to me with reiserfs as it was shipped with suse 9.1: ------------------------------------------------------------------ dreher@euler03:~/mytex/konstanz/wohnung> ls auto makler2.aux makler2.log makler2.tex makler.aux makler.log swk.eps unilogo.eps briefkpf.tex makler2.dvi makler2.ps makler3.tex makler.dvi makler.tex unikopf.tex dreher@euler03:~/mytex/konstanz/wohnung> rm *.aux *.log rm: cannot remove `makler2.log': No such file or directory dreher@euler03:~/mytex/konstanz/wohnung> ls auto briefkpf.tex makler2.dvi makler2.ps makler2.tex makler3.tex makler.dvi makler.tex swk.eps unikopf.tex unilogo.eps dreher@euler03:~/mytex/konstanz/wohnung> uname -a Linux euler03 2.6.5-7.108-smp #1 SMP Wed Aug 25 13:34:40 UTC 2004 i686 i686 i386 GNU/Linux dreher@euler03:~/mytex/konstanz/wohnung> date Tue Sep 21 13:15:45 CEST 2004 ---------------------------------------------------------- Note the line "rm: cannot remove `makler2.log': No such file or directory" There was no data loss, but such a bug should not happen. I never had similar experiences with ext3. Unfortunately, I cannot reproduce this behavior. > Powerloss, unpluging the Disk while writing, full filesystem, > heavy use : No problems with reiser4.. It *is* stable. My impression: reiser3 is not 100% stable, but quite stable, written by someone who asks for "review by benchmark". Michael ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 20:49 ` Michael Dreher @ 2005-06-23 21:54 ` M. 2005-06-23 23:37 ` Alan Cox 2005-06-23 23:57 ` Hans Reiser 2 siblings, 0 replies; 620+ messages in thread From: M. @ 2005-06-23 21:54 UTC (permalink / raw) To: linux-kernel; +Cc: reiserfs-list I think we are talking about reiser4, not reiser3.. Michele On 6/23/05, Michael Dreher <michael.dreher@uni-konstanz.de> wrote: > >>> Not everyone will want > >>> to reformat at once, but as the reiser4 code matures and proves itself > >>> (even more than it already has), > >> > >> I for one have seen mainly people with wild claims that it will make > >> their machines much faster, and coming back later asking how they can > >> recover their thrashed partitions... > > > > Then please show us some Links/Message-IDs to such postings. > > I'd like to read them. > > Here you are.... > > The following happened to me with reiserfs as it was shipped with > suse 9.1: > > ------------------------------------------------------------------ > dreher@euler03:~/mytex/konstanz/wohnung> ls > auto makler2.aux makler2.log makler2.tex makler.aux makler.log > swk.eps unilogo.eps > briefkpf.tex makler2.dvi makler2.ps makler3.tex makler.dvi makler.tex > unikopf.tex > dreher@euler03:~/mytex/konstanz/wohnung> rm *.aux *.log > rm: cannot remove `makler2.log': No such file or directory > dreher@euler03:~/mytex/konstanz/wohnung> ls > auto briefkpf.tex makler2.dvi makler2.ps makler2.tex makler3.tex > makler.dvi makler.tex swk.eps unikopf.tex unilogo.eps > dreher@euler03:~/mytex/konstanz/wohnung> uname -a > Linux euler03 2.6.5-7.108-smp #1 SMP Wed Aug 25 13:34:40 UTC 2004 i686 i686 > i386 GNU/Linux > dreher@euler03:~/mytex/konstanz/wohnung> date > Tue Sep 21 13:15:45 CEST 2004 > ---------------------------------------------------------- > > Note the line "rm: cannot remove `makler2.log': No such file or directory" > > There was no data loss, but such a bug should not happen. > I never had similar experiences with ext3. > > Unfortunately, I cannot reproduce this behavior. > > > Powerloss, unpluging the Disk while writing, full filesystem, > > heavy use : No problems with reiser4.. It *is* stable. > > My impression: reiser3 is not 100% stable, but quite stable, > written by someone who asks for "review by benchmark". > > Michael > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 20:49 ` Michael Dreher 2005-06-23 21:54 ` M. @ 2005-06-23 23:37 ` Alan Cox 2005-06-23 23:57 ` Hans Reiser 2 siblings, 0 replies; 620+ messages in thread From: Alan Cox @ 2005-06-23 23:37 UTC (permalink / raw) To: Michael Dreher Cc: Linux Kernel Mailing List, Adrian Ulrich, Horst von Brand, ninja, reiser, jgarzik, hch, akpm, reiserfs-list On Iau, 2005-06-23 at 21:49, Michael Dreher wrote: > My impression: reiser3 is not 100% stable, but quite stable, > written by someone who asks for "review by benchmark". Review by uniprocessor benchmark perhaps. However Reiser4 is new code. The original extfs on Linux was not very good either while ext2 was excellent. It seems inappropriate to technically review one fs based on the other. Looking at the authors maintenance record I think is important but every single star Linux kernel contributors first major contribution was generally not very good. Alan ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 20:49 ` Michael Dreher 2005-06-23 21:54 ` M. 2005-06-23 23:37 ` Alan Cox @ 2005-06-23 23:57 ` Hans Reiser 2005-06-24 0:16 ` Bernd Eckenfels 2005-06-28 11:14 ` Vitaly Fertman 2 siblings, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-23 23:57 UTC (permalink / raw) To: Michael Dreher, vitaly Cc: linux-kernel, Adrian Ulrich, Horst von Brand, ninja, jgarzik, hch, akpm, reiserfs-list One, you were using V3 not V4. Two, this bug you mention is probably not an fs bug. rm first creates a list of names, and then removes them. reiser@linux:~/scratch> touch fufu reiser@linux:~/scratch> touch fifu reiser@linux:~/scratch> rm *fu fi* rm: cannot remove `fifu': No such file or directory Your file either somehow got removed before rm got to it, or rm somehow got to it twice. Vitaly, can you look at the error handling by rm and see if it can get to things twice when it hits an error or if you can otherwise figure this out? If I remember right, I have hit this myself for non-reiserfs filesystems, and I never investigated it. Michael Dreher wrote: >>>> Not everyone will want >>>>to reformat at once, but as the reiser4 code matures and proves itself >>>>(even more than it already has), >>>> >>>> >>>I for one have seen mainly people with wild claims that it will make >>>their machines much faster, and coming back later asking how they can >>>recover their thrashed partitions... >>> >>> >>Then please show us some Links/Message-IDs to such postings. >>I'd like to read them. >> >> > >Here you are.... > >The following happened to me with reiserfs as it was shipped with >suse 9.1: > >------------------------------------------------------------------ >dreher@euler03:~/mytex/konstanz/wohnung> ls >auto makler2.aux makler2.log makler2.tex makler.aux makler.log >swk.eps unilogo.eps >briefkpf.tex makler2.dvi makler2.ps makler3.tex makler.dvi makler.tex >unikopf.tex >dreher@euler03:~/mytex/konstanz/wohnung> rm *.aux *.log >rm: cannot remove `makler2.log': No such file or directory >dreher@euler03:~/mytex/konstanz/wohnung> ls >auto briefkpf.tex makler2.dvi makler2.ps makler2.tex makler3.tex >makler.dvi makler.tex swk.eps unikopf.tex unilogo.eps >dreher@euler03:~/mytex/konstanz/wohnung> uname -a >Linux euler03 2.6.5-7.108-smp #1 SMP Wed Aug 25 13:34:40 UTC 2004 i686 i686 >i386 GNU/Linux >dreher@euler03:~/mytex/konstanz/wohnung> date >Tue Sep 21 13:15:45 CEST 2004 >---------------------------------------------------------- > >Note the line "rm: cannot remove `makler2.log': No such file or directory" > >There was no data loss, but such a bug should not happen. >I never had similar experiences with ext3. > >Unfortunately, I cannot reproduce this behavior. > > > >> Powerloss, unpluging the Disk while writing, full filesystem, >> heavy use : No problems with reiser4.. It *is* stable. >> >> > >My impression: reiser3 is not 100% stable, but quite stable, >written by someone who asks for "review by benchmark". > >Michael > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 23:57 ` Hans Reiser @ 2005-06-24 0:16 ` Bernd Eckenfels 2005-06-24 8:52 ` zhilla 2005-06-28 11:14 ` Vitaly Fertman 1 sibling, 1 reply; 620+ messages in thread From: Bernd Eckenfels @ 2005-06-24 0:16 UTC (permalink / raw) To: linux-kernel In article <42BB4C81.6070500@namesys.com> you wrote: > reiser@linux:~/scratch> touch fufu > reiser@linux:~/scratch> touch fifu > reiser@linux:~/scratch> rm *fu fi* > rm: cannot remove `fifu': No such file or directory # echo rm *fu fi* rm fifu fufu fifu shell magic :) Bernd ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 0:16 ` Bernd Eckenfels @ 2005-06-24 8:52 ` zhilla 0 siblings, 0 replies; 620+ messages in thread From: zhilla @ 2005-06-24 8:52 UTC (permalink / raw) To: linux-kernel Bernd Eckenfels wrote: >> reiser@linux:~/scratch> touch fufu >> reiser@linux:~/scratch> touch fifu >> reiser@linux:~/scratch> rm *fu fi* >> rm: cannot remove `fifu': No such file or directory > > # echo rm *fu fi* > rm fifu fufu fifu > shell magic :) may i add that you guys are a bunch of stoners :) ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 23:57 ` Hans Reiser 2005-06-24 0:16 ` Bernd Eckenfels @ 2005-06-28 11:14 ` Vitaly Fertman 1 sibling, 0 replies; 620+ messages in thread From: Vitaly Fertman @ 2005-06-28 11:14 UTC (permalink / raw) To: reiserfs-list Cc: Hans Reiser, Michael Dreher, vitaly, linux-kernel, Adrian Ulrich, Horst von Brand, ninja, jgarzik, hch, akpm On Friday 24 June 2005 03:57, Hans Reiser wrote: > One, you were using V3 not V4. > > Two, this bug you mention is probably not an fs bug. rm first creates a > list of names, and then removes them. shell creates a list of names, rm simply handles the given set of names and tries to remove a file twice if specified twice. the problem could be in the shell, or what is more likely in patterns that overlap each other, like in the example of 'rm *fu fi*'. we should probably investigate a reproducable example. > reiser@linux:~/scratch> touch fufu > reiser@linux:~/scratch> touch fifu > reiser@linux:~/scratch> rm *fu fi* > rm: cannot remove `fifu': No such file or directory > > Your file either somehow got removed before rm got to it, or rm somehow > got to it twice. Vitaly, can you look at the error handling by rm and > see if it can get to things twice when it hits an error or if you can once per every given name. > otherwise figure this out? If I remember right, I have hit this myself > for non-reiserfs filesystems, and I never investigated it. > > Michael Dreher wrote: > > >>>> Not everyone will want > >>>>to reformat at once, but as the reiser4 code matures and proves itself > >>>>(even more than it already has), > >>>> > >>>> > >>>I for one have seen mainly people with wild claims that it will make > >>>their machines much faster, and coming back later asking how they can > >>>recover their thrashed partitions... > >>> > >>> > >>Then please show us some Links/Message-IDs to such postings. > >>I'd like to read them. > >> > >> > > > >Here you are.... > > > >The following happened to me with reiserfs as it was shipped with > >suse 9.1: > > > >------------------------------------------------------------------ > >dreher@euler03:~/mytex/konstanz/wohnung> ls > >auto makler2.aux makler2.log makler2.tex makler.aux makler.log > >swk.eps unilogo.eps > >briefkpf.tex makler2.dvi makler2.ps makler3.tex makler.dvi makler.tex > >unikopf.tex > >dreher@euler03:~/mytex/konstanz/wohnung> rm *.aux *.log > >rm: cannot remove `makler2.log': No such file or directory > >dreher@euler03:~/mytex/konstanz/wohnung> ls > >auto briefkpf.tex makler2.dvi makler2.ps makler2.tex makler3.tex > >makler.dvi makler.tex swk.eps unikopf.tex unilogo.eps > >dreher@euler03:~/mytex/konstanz/wohnung> uname -a > >Linux euler03 2.6.5-7.108-smp #1 SMP Wed Aug 25 13:34:40 UTC 2004 i686 i686 > >i386 GNU/Linux > >dreher@euler03:~/mytex/konstanz/wohnung> date > >Tue Sep 21 13:15:45 CEST 2004 > >---------------------------------------------------------- > > > >Note the line "rm: cannot remove `makler2.log': No such file or directory" > > > >There was no data loss, but such a bug should not happen. > >I never had similar experiences with ext3. > > > >Unfortunately, I cannot reproduce this behavior. > > > > > > > >> Powerloss, unpluging the Disk while writing, full filesystem, > >> heavy use : No problems with reiser4.. It *is* stable. > >> > >> > > > >My impression: reiser3 is not 100% stable, but quite stable, > >written by someone who asks for "review by benchmark". > > > >Michael > > > > > > > > > > > -- Thanks, Vitaly Fertman ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 20:12 ` Adrian Ulrich (?) (?) @ 2005-06-23 21:29 ` Avuton Olrich 2005-06-23 21:36 ` Dan Oglesby 2005-06-24 1:19 ` Jim Crilly -1 siblings, 2 replies; 620+ messages in thread From: Avuton Olrich @ 2005-06-23 21:29 UTC (permalink / raw) To: Adrian Ulrich Cc: Horst von Brand, ninja, reiser, jgarzik, hch, akpm, linux-kernel, reiserfs-list On 6/23/05, Adrian Ulrich <reiser4@blinkenlights.ch> wrote: > From my POV: > I've been using Reiser4 for almost everything (Rootfs / External > Harddrives) for about ~8 Months without any data loss.. > > Powerloss, unpluging the Disk while writing, full filesystem, > heavy use : No problems with reiser4.. It *is* stable. *From users who use it* I have heard nothing but love for reiser4. It's amazing how quickly people seem to be dismissive about what reiser4 has to offer when they more than likely haven't taken it for a spin at all. All I hear about is 'we can't let something ugly go into the stable kernel' then in the same day I looked into some of the config options... CONFIG_WDC_ALI15X3: *snip* This allows for UltraDMA support for WDC drives that ignore CRC checking. You are a fool for enabling this option, but there have been requests. *snip* How many have requested that reiser4 make it into the kernel? I'd imagine many more then requested this IDE feature. And it's an *option*. Please work something out on this. avuton -- Anyone who quotes me in their sig is an idiot. -- Rusty Russell. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 21:29 ` Avuton Olrich @ 2005-06-23 21:36 ` Dan Oglesby 2005-06-24 1:19 ` Jim Crilly 1 sibling, 0 replies; 620+ messages in thread From: Dan Oglesby @ 2005-06-23 21:36 UTC (permalink / raw) To: Avuton Olrich Cc: Adrian Ulrich, Horst von Brand, ninja, reiser, jgarzik, hch, akpm, linux-kernel, reiserfs-list Avuton Olrich wrote: > On 6/23/05, Adrian Ulrich <reiser4@blinkenlights.ch> wrote: > >>From my POV: >> I've been using Reiser4 for almost everything (Rootfs / External >> Harddrives) for about ~8 Months without any data loss.. >> >> Powerloss, unpluging the Disk while writing, full filesystem, >> heavy use : No problems with reiser4.. It *is* stable. > > > *From users who use it* I have heard nothing but love for reiser4. > It's amazing how quickly people seem to be dismissive about what > reiser4 has to offer when they more than likely haven't taken it for a > spin at all. All I hear about is 'we can't let something ugly go into > the stable kernel' then in the same day I looked into some of the > config options... > > CONFIG_WDC_ALI15X3: > *snip* > This allows for UltraDMA support for WDC drives that ignore CRC > checking. You are a fool for enabling this option, but there have been > requests. > *snip* > > How many have requested that reiser4 make it into the kernel? I'd > imagine many more then requested this IDE feature. And it's an > *option*. Please work something out on this. > > avuton > As a user who uses ReiserFS v3, I have nothing but love for ReiserFS. The speed and reliability are great right now, I'm really looking forward to using ReiserFS v4. --Dan ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 21:29 ` Avuton Olrich 2005-06-23 21:36 ` Dan Oglesby @ 2005-06-24 1:19 ` Jim Crilly 1 sibling, 0 replies; 620+ messages in thread From: Jim Crilly @ 2005-06-24 1:19 UTC (permalink / raw) To: Avuton Olrich Cc: Adrian Ulrich, Horst von Brand, ninja, reiser, jgarzik, hch, akpm, linux-kernel, reiserfs-list On 06/23/05 02:29:21PM -0700, Avuton Olrich wrote: > On 6/23/05, Adrian Ulrich <reiser4@blinkenlights.ch> wrote: > > From my POV: > > I've been using Reiser4 for almost everything (Rootfs / External > > Harddrives) for about ~8 Months without any data loss.. > > > > Powerloss, unpluging the Disk while writing, full filesystem, > > heavy use : No problems with reiser4.. It *is* stable. > > *From users who use it* I have heard nothing but love for reiser4. > It's amazing how quickly people seem to be dismissive about what > reiser4 has to offer when they more than likely haven't taken it for a > spin at all. All I hear about is 'we can't let something ugly go into > the stable kernel' then in the same day I looked into some of the > config options... > > CONFIG_WDC_ALI15X3: > *snip* > This allows for UltraDMA support for WDC drives that ignore CRC > checking. You are a fool for enabling this option, but there have been > requests. > *snip* That's hardly a good example, that config option, while obviously questionable, only added 2 #ifdef blocks and affects 1 function that's 20 lines long. > > How many have requested that reiser4 make it into the kernel? I'd > imagine many more then requested this IDE feature. And it's an > *option*. Please work something out on this. > > avuton Jim. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 19:24 ` Horst von Brand 2005-06-23 20:12 ` Adrian Ulrich @ 2005-06-23 22:04 ` David Masover 2005-06-23 23:43 ` Alan Cox ` (2 more replies) 2005-06-28 13:54 ` cryptocompress [was Re: reiser4 plugins] Pavel Machek 2 siblings, 3 replies; 620+ messages in thread From: David Masover @ 2005-06-23 22:04 UTC (permalink / raw) To: Horst von Brand Cc: Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Horst von Brand wrote: > David Masover <ninja@slaphack.com> wrote: > >>Hans Reiser wrote: >> >>>Jeff Garzik wrote: > > > [...] > > >>>You missed his point. He is saying ext3 code should migrate towards >>>becoming reiser4 plugins, and reiser4 should be merged now so that the >>>migration can get started. > > >>Sort of. >> >>I think ext3 would be nice as a reiser4 plugin. > > > What for? It works just fine as it stands, AFAICS. So does DOS. Do you use DOS? I don't even use DOS to run DOS programs. "Ain't broke" is the battle cry of stagnation. But, there are some things Reiser does better and faster than ext3, even if you don't count file-as-directory and other toys. There's nothing ext3 does better than Reiser, unless you count the compatibility with random bootloaders and low-level tools. >> Not everyone will want >>to reformat at once, but as the reiser4 code matures and proves itself >>(even more than it already has), > > > I for one have seen mainly people with wild claims that it will make their > machines much faster, and coming back later asking how they can recover > their thrashed partitions... You know how many I've had thrashed on Reiser4? Two. The first one was with a VERY early alpha/beta, and the second one was when I dropped a laptop and the disk failed. And it does make certain things faster. For one thing, "emerge sync" on Gentoo is twice to four times as fast, and /usr/portage is 75% as big, as on ReiserFS (3). >> the ext3 people may find themselves >>wanting some of the more generic optimizations. > > > They'll filch them in due time, don't worry. Duplication of effort. With plugins, we can optimize the upper layers of ALL filesystems, regardless of the lower layers, in such a way that it is optional. I'm sure it's far easier to write a Reiser storage plugin than a brand new FS. Eventually, once competition is only based on storage format, we could end up with just one format. Just one filesystem! (except for fat/ntfs/iso/udf/network...) And in the open source world, sometimes a single product is a good thing. >>But, I don't think that will realistically happen at all. >> >>Instead, what will probably happen is that once Reiser4 is in the >>mainstream kernel, it will become more popular and noticable. Other >>FSes will take note. ext3 people may decide they want >>file-as-directory, > > > That idea is even much older than Linux itself, and no other (Unix) > filesystem has implemented it. Ever. Wonder why... > > >> and vfat people may decide they want cryptocompress, > > > I'm sure they don't, because it is mostly for Windows and assorted devices > (pendrives, digital cameras, ...) compatibility. I, for one, would like to use a pendrive and have certain files be encrypted transparently, only for use on Linux, but others be ready to transfer to a Windows box. >>Eventually, with all the features ported, we end up with a situation >>where there may be no meaningful difference between a filesystem and a >>low-level reiser4 plugin. > > > Could very well take decades, if ever. How would you do it, in a way that doesn't take decades? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrsx6HgHNmZLgCUhAQKXYw//Z9bSh74AWr2o1n+EM0nCHUNV78xgeuey ZHtzUT8rCv2KQ+2fMe5EnRUaRYTKvFnnGccH4vu/OAArqKqt/RgP3NP8UZALsZbB MMoEHZSX/E0BFlKKiBjPgvwfnnY9ZYF0GPkU5L97dj1K0dEQMndOZoYDV07H4TnN /1VkytnsXuhm5nqRhPd89rDbvQtXpzHiDjVNPfT+J6M6JFw8jfYQZ0Fo1T9dVKMg qibVneJj+onHVBmBBqGTZ0Ane5VmG0h0a2ZwsslQPkf03DAgprliykr40NCECdli OdaS73qYPlYRRl1nuw84g2KOLXbTnSGmW1t4qt/tyI6t3TOEn9FqY7YzwvbWIVLP GMkJG1htAQefGNcEXx+15xAHJi6/0DiSoJu+P+ie+yG8pkG943936AxEgs89pTpC z8i/GV9Uq+S6BgA+RJuxpk6U8rQ0YHMhAiK83p+s7vdUsGIKomUmSbn7a6DC7cZt aGkqxYyaoV2MlHUMUebSv5F82JUgx6rPuc8SQW1wvVIVdNA7QhlYYsPa5vFFj34C scxN9vNGyxWGu60yKx7LSYRkB9/prJytKvpezVKRkn8pnKnl4AHodKioSPxt1iHC BvNsLL8Kn8FBe/HG98d1v8vwTqe0Y8KgBMRe/J73u/OzU82lh2V3YbXNaW+DLVGz MOaHYEzfoNg= =yFsv -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 22:04 ` David Masover @ 2005-06-23 23:43 ` Alan Cox 2005-06-24 1:12 ` Hans Reiser 2005-06-24 3:17 ` David Masover 2005-06-24 1:02 ` Hans Reiser 2005-06-24 2:41 ` Horst von Brand 2 siblings, 2 replies; 620+ messages in thread From: Alan Cox @ 2005-06-23 23:43 UTC (permalink / raw) To: David Masover Cc: Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Iau, 2005-06-23 at 23:04, David Masover wrote: > > What for? It works just fine as it stands, AFAICS. > > So does DOS. Do you use DOS? I don't even use DOS to run DOS programs. False argument. So does the pen, so do hinges on doors. Do you still have hinges on your doors - probably. > "Ain't broke" is the battle cry of stagnation. Its also the battle cry of everyone over the age of 20 who also has a real job to do 8) > But, there are some things Reiser does better and faster than ext3, even > if you don't count file-as-directory and other toys. There's nothing > ext3 does better than Reiser, unless you count the compatibility with > random bootloaders and low-level tools. Certainly compared with reiser3 you've missed a few out including resilience to disk errors (nearly nil on reiser3), and SMP scaling. > You know how many I've had thrashed on Reiser4? Two. The first one was > with a VERY early alpha/beta, and the second one was when I dropped a > laptop and the disk failed. Entirely or bad blocks ? The latter should have a minimal cost on a well designed fs. > Duplication of effort. With plugins, we can optimize the upper layers > of ALL filesystems, regardless of the lower layers, in such a way that In which case the features belong in the VFS as all those with experience and kernel contributions have been arguing. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 23:43 ` Alan Cox @ 2005-06-24 1:12 ` Hans Reiser 2005-06-24 1:45 ` Jeff Garzik ` (2 more replies) 2005-06-24 3:17 ` David Masover 1 sibling, 3 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-24 1:12 UTC (permalink / raw) To: Alan Cox Cc: David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Alan Cox wrote: > SMP scaling. Reiser4 should do much better at this, as it was designed for it. I wish we had a nice hunking multiprocessor to verify that and work through the inevitable unintended sources of bottlenecks though. > > >>You know how many I've had thrashed on Reiser4? Two. The first one was >>with a VERY early alpha/beta, and the second one was when I dropped a >>laptop and the disk failed. >> >> > >Entirely or bad blocks ? The latter should have a minimal cost on a well >designed fs. > > > >>Duplication of effort. With plugins, we can optimize the upper layers >>of ALL filesystems, regardless of the lower layers, in such a way that >> >> > >In which case the features belong in the VFS as all those with >experience and kernel contributions have been arguing. > > So you fundamentally reject the prototype it in one fs and then abstract it to others development model? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 1:12 ` Hans Reiser @ 2005-06-24 1:45 ` Jeff Garzik 2005-06-24 2:31 ` Lincoln Dale 2005-06-24 10:41 ` Alan Cox 2 siblings, 0 replies; 620+ messages in thread From: Jeff Garzik @ 2005-06-24 1:45 UTC (permalink / raw) To: Hans Reiser Cc: Alan Cox, David Masover, Horst von Brand, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Hans Reiser wrote: > Alan Cox wrote: >>>Duplication of effort. With plugins, we can optimize the upper layers >>>of ALL filesystems, regardless of the lower layers, in such a way that >>In which case the features belong in the VFS as all those with >>experience and kernel contributions have been arguing. > So you fundamentally reject the prototype it in one fs and then abstract > it to others development model? For similar reasons, we don't let hardware vendors implement software RAID inside the hardware driver. It's not appropriate at that level. Jeff ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 1:12 ` Hans Reiser 2005-06-24 1:45 ` Jeff Garzik @ 2005-06-24 2:31 ` Lincoln Dale 2005-06-24 3:21 ` Jeff Garzik 2005-06-24 6:49 ` Hans Reiser 2005-06-24 10:41 ` Alan Cox 2 siblings, 2 replies; 620+ messages in thread From: Lincoln Dale @ 2005-06-24 2:31 UTC (permalink / raw) To: Hans Reiser Cc: Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Hans Reiser wrote: >So you fundamentally reject the prototype it in one fs and then abstract >it to others development model? > > Hans, after many years here now, one would have thought you would have "got" this part of Linux: kernel development & code that gets into the kernel only does so by getting past the benevolent dictators. instead, it seems that every time there is ReiserFS to be merged (and we can go back in history a number of years here..), it always seems to come as a great shock that your code won't be merged 'as-is' without peer review & comment. don't feel that you're being singled out here. you aren't. there isn't any anti-Hans-and-his-filesystem conspiracy here. there are plenty of examples on where this has happened in Linux previously in other parts of the tree. EVMS is a great example of similar things - a proposal to include kernel code to do various volume-mgmt functions - which was basically accomplishing the same goal as that of LVM/LVM2 and MD drivers (& DM framework). the EVMS team are a great act to follow - see http://lwn.net/Articles/14714/ - they showed high levels of professional conduct and made what was essentially a 'hard' but 'correct' decision in reworking EVMS to use the same DM infrastructure as LVM2. there are countless other examples at various times - various 'competing' IPv6 projects, IPSec, various "hardware" (software) RAID controllers, various IP offload schemes et al. why does Reiserfs have to be any different? you know that VFS is the mechanism in Linux. you know (i hope..) how it works. it isn't so hard to see how many of the Reiser4 "plug-ins" could be tied into VFS calls. OR, if they cannot TODAY, propose how VFS _COULD_ be made to do this. the key here is trust. and trust is a two-way street. the irony of this whole thread is that history is repeating itself. see http://www.ussg.iu.edu/hypermail/linux/kernel/0112.1/0519.html kernel developers pushed back on you 3 years ago - in 2001 - what has really changed? *an observation* cheers, lincoln. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 2:31 ` Lincoln Dale @ 2005-06-24 3:21 ` Jeff Garzik 2005-06-24 6:49 ` Hans Reiser 1 sibling, 0 replies; 620+ messages in thread From: Jeff Garzik @ 2005-06-24 3:21 UTC (permalink / raw) To: Lincoln Dale Cc: Hans Reiser, Alan Cox, David Masover, Horst von Brand, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Lincoln Dale wrote: > the EVMS team are a great act to follow - see > http://lwn.net/Articles/14714/ - they showed high levels of professional > conduct and made what was essentially a 'hard' but 'correct' decision in > reworking EVMS to use the same DM infrastructure as LVM2. I just want to heartily second this. The EVMS team made the hard but ultimately best choice, and everybody benefited. They deserve all the kudos they get. Jeff ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 2:31 ` Lincoln Dale 2005-06-24 3:21 ` Jeff Garzik @ 2005-06-24 6:49 ` Hans Reiser 2005-06-24 7:10 ` Lincoln Dale 2005-06-24 7:11 ` Al Viro 1 sibling, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-24 6:49 UTC (permalink / raw) To: Lincoln Dale Cc: Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Lincoln Dale wrote: > > the irony of this whole thread is that history is repeating itself. > see http://www.ussg.iu.edu/hypermail/linux/kernel/0112.1/0519.html > kernel developers pushed back on you 3 years ago - in 2001 - what has > really changed? It is exactly the same, but rather than dwell on that, I'll just remind that I have sent out two technical emails which talk only about the issue of plugins and pluginids, and whether plugins are classes rather than just instances, and whether the classes really would benefit from being instantiated into VFS at the cost of keeping the same info in two places, and I got no answer on them. Zam pointed out that our plugins do more than just VFS operations, and got no response on that or his other points. Regarding trust, Christophe comes out the gate using the words "useless abstraction layer" that happens to be a core feature of our design, demanding we cut it out, and not really understanding it or recognizing any functionality it provides, and you can't really expect me to respect this, can you? Now, if his target is reduced to whether we can eliminate a function indirection, and whether we can review the code together and see if it is easy to extend plugins and pluginids to other filesystems by finding places to make it more generic and accepting of per filesystem plugins, especially if it is not tied to going into 2.6.13, well, that is the conversation I would have liked to have had. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 6:49 ` Hans Reiser @ 2005-06-24 7:10 ` Lincoln Dale 2005-06-24 8:23 ` Hans Reiser 2005-06-24 9:35 ` Timothy Webster 2005-06-24 7:11 ` Al Viro 1 sibling, 2 replies; 620+ messages in thread From: Lincoln Dale @ 2005-06-24 7:10 UTC (permalink / raw) To: Hans Reiser Cc: Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Hans Reiser wrote: >Lincoln Dale wrote: > > > >>the irony of this whole thread is that history is repeating itself. >>see http://www.ussg.iu.edu/hypermail/linux/kernel/0112.1/0519.html >>kernel developers pushed back on you 3 years ago - in 2001 - what has >>really changed? >> >> > >It is exactly the same, but rather than dwell on that, I'll just remind >that I have sent out two technical emails which talk only about the >issue of plugins and pluginids, and whether plugins are classes rather >than just instances, and whether the classes really would benefit from >being instantiated into VFS at the cost of keeping the same info in two >places, and I got no answer on them. Zam pointed out that our plugins >do more than just VFS operations, and got no response on that or his >other points. > >Regarding trust, Christophe comes out the gate using the words "useless >abstraction layer" that happens to be a core feature of our design, >demanding we cut it out, and not really understanding it or recognizing >any functionality it provides, and you can't really expect me to respect >this, can you? > > heh. one example i didn't mention in the myriad of EVMS, IPv6, IPSec et al was iSCSI. Christoph was (rightly so) very strong in pushing back on things that needed to be fixed/changed/addressed in linux-iscsi, a project near & dear to much of the paid work i do. while i'm sure Christoph could probably be more tactful at times, his views on technical matters in the kernel are respected. the key here is to not take it personally - but instead just understand that this is the framework that you have to work in to get it into the mainline kernel. if you don't want to go down that path, you're free to do so. its open source, you can keep your own linux-kernel fork. but if you want to get your code into mainline, i don't think its so much a case of l-k folks telling you how to make your code work under VFS. the onus is on you to say WHY your code and plugins could never be put into VFS. if anything, you should be thankful that Al hasn't yet commented. :) >Now, if his target is reduced to whether we can eliminate a function >indirection, and whether we can review the code together and see if it >is easy to extend plugins and pluginids to other filesystems by finding >places to make it more generic and accepting of per filesystem plugins, >especially if it is not tied to going into 2.6.13, well, that is the >conversation I would have liked to have had. > > > fantastic - some common ground. any reason WHY there has to be an abstraction of 'pluginid' when in theory VFS operations can already provide the necessary abstraction on a per-object basis? Nikita basically said as much in Message-ID: <17081.30107.751071.983835@gargle.gargle.HOWL> earlier in this thread: "But it is not so. There _are_ plugins-in-the-VFS. VFS operates on opaque objects (inodes, dentries, file system types) through interfaces: {inode,address_space,dentry,sb,etc.}_operations. Every file system back-end if free to implement whatever number of these interfaces. And the do this already: check the sources; even ext2 does this: e.g., ext2_fast_symlink_inode_operations and ext2_symlink_inode_operations. This is exactly what upper level reiser4 plugins are for. I guess that one of Christoph Hellwig's complaints is that reiser4 introduces another layer of abstraction to implement something that already exists." i never saw any reason why there can't be specific VFS operations put together for specific inodes if the policy for that inode dictates any special operation (e.g. compression). cheers, lincoln. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 7:10 ` Lincoln Dale @ 2005-06-24 8:23 ` Hans Reiser 2005-06-24 8:40 ` Lincoln Dale 2005-06-24 15:32 ` Horst von Brand 2005-06-24 9:35 ` Timothy Webster 1 sibling, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-24 8:23 UTC (permalink / raw) To: Lincoln Dale Cc: Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Lincoln Dale wrote: > >> Now, if his target is reduced to whether we can eliminate a function >> indirection, and whether we can review the code together and see if it >> is easy to extend plugins and pluginids to other filesystems by finding >> places to make it more generic and accepting of per filesystem plugins, >> especially if it is not tied to going into 2.6.13, well, that is the >> conversation I would have liked to have had. >> >> >> > fantastic - some common ground. > any reason WHY there has to be an abstraction of 'pluginid' when in > theory VFS operations can already provide the necessary abstraction on > a per-object basis? VFS supplies instances, plugins are classes. If a language can instantiate an object, that does not eliminate the value of being able to create classes. Does it make sense to you now? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 8:23 ` Hans Reiser @ 2005-06-24 8:40 ` Lincoln Dale 2005-06-24 15:32 ` Horst von Brand 1 sibling, 0 replies; 620+ messages in thread From: Lincoln Dale @ 2005-06-24 8:40 UTC (permalink / raw) To: Hans Reiser Cc: Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Hans Reiser wrote: >>>ow, if his target is reduced to whether we can eliminate a function >>>indirection, and whether we can review the code together and see if it >>>is easy to extend plugins and pluginids to other filesystems by finding >>>places to make it more generic and accepting of per filesystem plugins, >>>especially if it is not tied to going into 2.6.13, well, that is the >>>conversation I would have liked to have had. >>> >>fantastic - some common ground. >>any reason WHY there has to be an abstraction of 'pluginid' when in >>theory VFS operations can already provide the necessary abstraction on >>a per-object basis? >> >> >VFS supplies instances, plugins are classes. If a language can >instantiate an object, that does not eliminate the value of being able >to create classes. > >Does it make sense to you now? > > you've lost me . . . regardless, it isn't /me/ that you need to convince. how about a posting to l-k on "why Reiser4 cannot use VFS infrastructure for [crypto,compression,blahblah] plugins" - ideally, for each plugin. cheers, lincoln. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 8:23 ` Hans Reiser 2005-06-24 8:40 ` Lincoln Dale @ 2005-06-24 15:32 ` Horst von Brand 2005-06-27 19:39 ` Tyson Whitehead 1 sibling, 1 reply; 620+ messages in thread From: Horst von Brand @ 2005-06-24 15:32 UTC (permalink / raw) To: Hans Reiser Cc: Lincoln Dale, Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Hans Reiser <reiser@namesys.com> wrote: > Lincoln Dale wrote: > >> Now, if his target is reduced to whether we can eliminate a function > >> indirection, and whether we can review the code together and see if it > >> is easy to extend plugins and pluginids to other filesystems by finding > >> places to make it more generic and accepting of per filesystem plugins, > >> especially if it is not tied to going into 2.6.13, well, that is the > >> conversation I would have liked to have had. > > fantastic - some common ground. > > any reason WHY there has to be an abstraction of 'pluginid' when in > > theory VFS operations can already provide the necessary abstraction on > > a per-object basis? > VFS supplies instances, plugins are classes. If a language can > instantiate an object, that does not eliminate the value of being able > to create classes. In OOP speak, VFS is an abstract class, each individual filesystem derives from this class giving a concrete class, a specific on-disk (or whereever) filesystem is an object of its (concrete) class. The rest of the kernel (as a client) doesn't care for the concrete classes, it speaks only (or mostly) in terms of the abstract class (VFS). And concrete filesystems in turn use the generic block layer. > Does it make sense to you now? No. Sounds jumbled up and backwards. And I don't see how "languages" could even enter the picture here. Again: Is there any (sane) way the /existing/ VFS can be used to express what you want? What are the /minimal/ changes to VFS so each extension can be catered for in an uniform way across filesystems (anything else destroys the core idea of having a VFS in the first place!)? What are the required changes in the clients of VFS (i.e., changes to the core kernel)? What would be the impact on existing filesystems that /don't/ use new functionality (this is a lot harder, because it impacts many independent pieces of code)? What would be required for an existing filesystem to incorporate it? When all this is answered, go over the implementation details. But that is far off still. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 15:32 ` Horst von Brand @ 2005-06-27 19:39 ` Tyson Whitehead 0 siblings, 0 replies; 620+ messages in thread From: Tyson Whitehead @ 2005-06-27 19:39 UTC (permalink / raw) To: linux-kernel Cc: Horst von Brand, Hans Reiser, Lincoln Dale, Alan Cox, David Masover, Jeff Garzik, Christoph Hellwig, Andrew Morton [-- Attachment #1: Type: text/plain, Size: 5253 bytes --] On June 24, 2005 11:32, Horst von Brand wrote: > Hans Reiser <reiser@namesys.com> wrote: > > > > VFS supplies instances, plugins are classes. If a language can > > instantiate an object, that does not eliminate the value of being able > > to create classes. > > In OOP speak, VFS is an abstract class, each individual filesystem derives > from this class giving a concrete class, a specific on-disk (or whereever) > filesystem is an object of its (concrete) class. The rest of the kernel (as > a client) doesn't care for the concrete classes, it speaks only (or mostly) > in terms of the abstract class (VFS). And concrete filesystems in turn use > the generic block layer. > > > Does it make sense to you now? > > No. Sounds jumbled up and backwards. And I don't see how "languages" could > even enter the picture here. There seems to be a bit of confusion by the OO speak (I was initially confused as well -- guess I've spend too much time in broken OO languages recently). *grin* Thinking of interfaces, classes, and objects as a bunch of 1:M relationships make it much clearer. If you start at the bottom, an object (instantiation of a class) has the per-object stuff (usually data). A class (instantiation of a meta-class or interface) has the shared object stuff (methods/functions that work on the data and static/shared data). An interface has the shared class stuff (info on what the methods/functions and static/shared data are). A class is a realization of an interface. An object is a realization of a class. The standard C thing is to have only objects be dynamic at runtime. A slightly less common (but very flexible) thing to do is to make the classes dynamic at runtime. This introduces the following C -> OO talk: -Instances of structures full of data are objects. -Instances of structure full of function pointers are classes. -Instances of structure that are a big mix of both are the crossbred offsprings of programmers committing OO maintenance suicide by violating all the layering (or a savy programmers cleverly saving of one level of indirection by caching an object's class' function pointers in the object -- depends on your point of view *grin*). In the Reiser4 FS I believe it goes like this (apologies Hans if I'm wrong). The objects are files, directories, etc. The classes are the plugins. Associated with each object is a number (plugin id/class pointer) identifying the set of functions (class/plugin) that provides the functions/methods to manipulate that object (ids are required because the objects [files, directories, etc.] live on the device while the plugins live in memory). The good thing is that it is very easy to provide new plugins (instances of the interface to manipulate the objects). To create a new new plugin (a new class) you just fill in a bunch of function pointer fields in a structure (possibly mixing and matching existing functions) and give it a plugin id. To use it, you just link appropriate objects (files for file plugins, directories for directory plugins, etc.) back to it via the plugin id. The VFS also has a bunch of OO stuff. Looking at "include/linux/fs.h", the inode structure is mostly data with a few pointers to structures full of functions pointers. Instances of it are most definitely objects. Instances of the associated structures full of functions pointers (inode_operations, file_operations, etc.) are most definitely classes (they give an implement of their respective interfaces for that inode). Likewise, instances of super_block are objects, while instances of super_operations (and friends) are classes. In short, in Reiser4 FS talk, much of the VFS is done with plugins (a plugin being an instance of one of the *_operations structures). The current merge issue seems to be that apparently (I haven't actually looked myself) lots of the VFS plugins that Reiser4 exports are just thunks. This might imply that there are Reiser4 class/plugin interfaces that correspond very closely to the VFS class/plugin interfaces (the *_operations). In this case it might make good sense to lift those plugins from Reiser4 to VFS plugins. (If the VFS class/plugin interfaces are subsets of some Reiser4 class/plugin interfaces, you could also probably do the lifting. The additional Reiser4 class/plugin interface parts could be added to the split off and added to the VFS objects via the generic pointer field [i.e., use it to attach the additional data and class/plugin pointers to the objects]. As this adds some pointer pain for the additional parts you may not really be gaining anything. Another possibly approach might be to just extend each of the VFS object structures by making them the first entry in more flushed out Reiser4 specific structures [the Reiser4 code better be doing the allocating in this case]. *grin*) Later! -T -- Tyson Whitehead (-twhitehe@uwo.ca -- WSC-) Computer Engineer Dept. of Applied Mathematics, Graduate Student- Applied Mathematics University of Western Ontario, GnuPG Key ID# 0x8A2AB5D8 London, Ontario, Canada [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 7:10 ` Lincoln Dale 2005-06-24 8:23 ` Hans Reiser @ 2005-06-24 9:35 ` Timothy Webster 2005-06-24 15:45 ` Horst von Brand 1 sibling, 1 reply; 620+ messages in thread From: Timothy Webster @ 2005-06-24 9:35 UTC (permalink / raw) To: Lincoln Dale, Hans Reiser Cc: Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List --- Lincoln Dale <ltd@cisco.com> wrote: snip > > Nikita basically said as much in Message-ID: > <17081.30107.751071.983835@gargle.gargle.HOWL> > earlier in this thread: > "But it is not so. There _are_ > plugins-in-the-VFS. VFS operates on > opaque > objects (inodes, dentries, file system types) > through interfaces: > > {inode,address_space,dentry,sb,etc.}_operations. > Every file system > back-end if free to implement whatever number > of these interfaces. And > the do this already: check the sources; even > ext2 does this: e.g., > ext2_fast_symlink_inode_operations and > ext2_symlink_inode_operations. > > This is exactly what upper level reiser4 > plugins are for. > > I guess that one of Christoph Hellwig's > complaints is that reiser4 > introduces another layer of abstraction to > implement something that > already exists." What reiserfs4 brings is file based plugins. Which is extremely useful and powerful. I don't want to see this go away. I think it is the task of the linux community to generalize the vfs layer and not lock out reiserfs4 until that is done. reiserfs4 wants to keep a plugin id for each and every file. An additional filesystem layer is the traditional solution to achieve advanced features, but not an optimal solution in my opinion. Yes gnome, kde and perhaps cifs do it. But if instead they used file plugins a lot more could be shared. Blush, I am not a file system expert -Tim __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 9:35 ` Timothy Webster @ 2005-06-24 15:45 ` Horst von Brand 2005-06-24 17:13 ` Perry Kundert 0 siblings, 1 reply; 620+ messages in thread From: Horst von Brand @ 2005-06-24 15:45 UTC (permalink / raw) To: tdwebste2 Cc: Lincoln Dale, Hans Reiser, Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Timothy Webster <tdwebste2@yahoo.com> wrote: [...] > I think it is the task of the linux community to > generalize the vfs layer and not lock out reiserfs4 > until that is done. No... the task of the Linux community is to make a better kernel. You will have to do the work to convince it that the changes give a better kernel if included. That means the work to find out what they object to, solve the problem (if it can be solved), and try again. > reiserfs4 wants to keep a plugin > id for each and every file. Good for you. Nobody else has shown any interest in that. > An additional filesystem > layer is the traditional solution to achieve advanced > features, but not an optimal solution in my opinion. Right. And what is said here is to /use/ said layer (VFS) or see how it can be changed to cater for your needs. > Yes gnome, kde and perhaps cifs do it. Gnome and KDE are userspace utilities, designed to run on several operating systems that do not have such filesystem plugins at all, plus for the foreseable future the mayority of Linux (file)systems won't have them either; so they probably won't ever use it for portability's sake. CIFS is a second class citizen AFAIU, as it exists for compatibility with legacy systems only. So none of the above is in any way compelling. > But if instead > they used file plugins a lot more could be shared. > Blush, I am not a file system expert /me neither ;-) -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 15:45 ` Horst von Brand @ 2005-06-24 17:13 ` Perry Kundert 2005-06-24 20:01 ` Fwd: " Perry Kundert 2005-06-24 21:38 ` Valdis.Kletnieks 0 siblings, 2 replies; 620+ messages in thread From: Perry Kundert @ 2005-06-24 17:13 UTC (permalink / raw) To: ReiserFS List On 6/24/05, Horst von Brand <vonbrand@inf.utfsm.cl> wrote: > Timothy Webster <tdwebste2@yahoo.com> wrote: > > [...] > > > I think it is the task of the linux community to > > generalize the vfs layer and not lock out reiserfs4 > > until that is done. > > No... the task of the Linux community is to make a better kernel. In general, isn't it better to first include modules providing divergent but possibly interesting functionality (such as Reiser4) as an "optional" or "experimental" component, and then slowly re-factor desirable functionality into higher level facilities like the VFS? One of the strengths of Linux (so far), is that it hasn't tried too hard to "guess" what functionality might be the next "killer" kernel feature; if something is truly interesting, it will eventually find its way more deeply into the kernel. If not, it stays in an "optional" driver. > > You will have to do the work to convince it that the changes give a better > kernel if included. That means the work to find out what they object to, > solve the problem (if it can be solved), and try again. > > > reiserfs4 wants to keep a plugin > > id for each and every file. > > Good for you. Nobody else has shown any interest in that. The jffs2 filesystem may perhaps vary its functionality and "on-flash" format wildly, depending on whether you use it with NOR or NAND flash. Not many people have shown interest in that; however, it is still an important feature to some (ie. ME). This is an implicit analog of a plugin; Why force the VFS to take note of how jffs2 happens to handle its formatting for NAND and NOR flash? Just let it use the correct "plugin"! Virtually every non-trivial filesystem has the concept of varying its behaviour, some implicitly (jffs2 NAND or NOR?), some explicitly (NFS mount options). Should we force them ALL to select ALL format variability via the VFS? So what if reiser4 wants to have a plugin that converts all english files to swahili or pig latin as an on-disk format, and it wants to call it a "plugin". So long as it doesn't interfere with the correctness of other filesystems, why prevent it from merging? I believe that it would be a GREATER error to force VFS changes supporting the generic idea of a "plugin", before proving that the concept has more general applicability. See if reiser4 lives or dies first, as an optional experimental filesystem. No one is forced to use it. In a couple of years, if it becomes an ignored "backwater" filesystem (like, say, coda?), then remove it from the mainline kernel. No one will notice... > > > An additional filesystem > > layer is the traditional solution to achieve advanced > > features, but not an optimal solution in my opinion. > > Right. And what is said here is to /use/ said layer (VFS) or see how it can > be changed to cater for your needs. If reiser4 wants to provide a usable interface (internally), to select behaviour that is traditionally selected by choosing a completely separate filesystem, then why not let it? It harms *no one* -- forcing changes to the VFS before it is proven that this kind of functionality is even generally desirable would be a HUGE error! > > > Yes gnome, kde and perhaps cifs do it. > > Gnome and KDE are userspace utilities, designed to run on several operating > systems that do not have such filesystem plugins at all, plus for the > foreseable future the mayority of Linux (file)systems won't have them > either; so they probably won't ever use it for portability's sake. CIFS is > a second class citizen AFAIU, as it exists for compatibility with legacy > systems only. So none of the above is in any way compelling. > > > But if instead > > they used file plugins a lot more could be shared. The ext2 and ext3 filesystems support the same basic on-disk format, as everyone knows. However, the selection of ext3 provides an on-disk journal, which (if the filesystem is then mounted as ext2), is ignored. You select the "journalling plugin" by mounting the fs as ext3. You turn it off by mounting it as ext2. Reiser4 provides a way to do precisely this kind of selection of variable on-disk formats by specifying a plugin. I ask you -- if everyone in kernel-land is so convinced that you should always select varying on-disk formats via the VFS, then *why* hasn't ext2/ext3 been merged into a single filesystem? Surely the "journalling" plugin of this filesystem is a prime candidate for selection via the VFS? Here's the deal. So Hans can be a butt-head. Get over it. Big projects attract big egos. This is a good thing! Reiser4 provides potentially very interesting functionality, some of which may or may not (over time) be interesting to other FS implementers. At that time, it will be "abstracted" into the VFS. *AFTER* it has been proven safe and useful to do so! Until then, including Reiser4 in the main-line kernel is *completely* consistent with the "do no harm" philosophy that has let other filesystems and drivers have be included. No one running a "stable" server is going to let the "experimental" Reiser4 filesystem code anywhere near their kernel -- just like all other "experimental" code! Others (like me), will use the 'md' drivers to provide the stable, error-free media that ALL filesystems, including reiser4, assume, 'rsync' to provide for backups since it is clearly marked as "experimental", and continue to benefit from the improved atomicity and speed that reiser4 provides -- just as I have been doing for the last 18 months! -- -pjk "If you will not fight when your victory will be sure and not too costly, you may come to the moment when you will have to fight with all the odds against you and only a precarious chance for survival. There may even be a worse case. You may have to fight when there is no hope of victory, because it is better to perish than to live as slaves." -- Winston Churchill ^ permalink raw reply [flat|nested] 620+ messages in thread
* Fwd: reiser4 plugins 2005-06-24 17:13 ` Perry Kundert @ 2005-06-24 20:01 ` Perry Kundert 2005-06-24 21:38 ` Valdis.Kletnieks 1 sibling, 0 replies; 620+ messages in thread From: Perry Kundert @ 2005-06-24 20:01 UTC (permalink / raw) To: Horst von Brand Cc: tdwebste2, Lincoln Dale, Hans Reiser, Alan Cox, David Masover, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On 6/24/05, Horst von Brand <vonbrand@inf.utfsm.cl> wrote: > Timothy Webster <tdwebste2@yahoo.com> wrote: > > [...] > > > I think it is the task of the linux community to > > generalize the vfs layer and not lock out reiserfs4 > > until that is done. > > No... the task of the Linux community is to make a better kernel. In general, isn't it better to first include modules providing divergent but possibly interesting functionality (such as Reiser4) as an "optional" or "experimental" component, and then slowly re-factor desirable functionality into higher level facilities like the VFS? One of the strengths of Linux (so far), is that it hasn't tried too hard to "guess" what functionality might be the next "killer" kernel feature; if something is truly interesting, it will eventually find its way more deeply into the kernel. If not, it stays in an "optional" driver. > > You will have to do the work to convince it that the changes give a better > kernel if included. That means the work to find out what they object to, > solve the problem (if it can be solved), and try again. > > > reiserfs4 wants to keep a plugin > > id for each and every file. > > Good for you. Nobody else has shown any interest in that. The jffs2 filesystem may perhaps vary its functionality and "on-flash" format wildly, depending on whether you use it with NOR or NAND flash. Not many people have shown interest in that; however, it is still an important feature to some (ie. ME). This is an implicit analog of a plugin; Why force the VFS to take note of how jffs2 happens to handle its formatting for NAND and NOR flash? Just let it use the correct "plugin"! Virtually every non-trivial filesystem has the concept of varying its behaviour, some implicitly (jffs2 NAND or NOR?), some explicitly (NFS mount options). Should we force them ALL to select ALL format variability via the VFS? So what if reiser4 wants to have a plugin that converts all english files to swahili or pig latin as an on-disk format, and it wants to call it a "plugin". So long as it doesn't interfere with the correctness of other filesystems, why prevent it from merging? I believe that it would be a GREATER error to force VFS changes supporting the generic idea of a "plugin", before proving that the concept has more general applicability. See if reiser4 lives or dies first, as an optional experimental filesystem. No one is forced to use it. In a couple of years, if it becomes an ignored "backwater" filesystem (like, say, coda?), then remove it from the mainline kernel. No one will notice... > > > An additional filesystem > > layer is the traditional solution to achieve advanced > > features, but not an optimal solution in my opinion. > > Right. And what is said here is to /use/ said layer (VFS) or see how it can > be changed to cater for your needs. If reiser4 wants to provide a usable interface (internally), to select behaviour that is traditionally selected by choosing a completely separate filesystem, then why not let it? It harms *no one* -- forcing changes to the VFS before it is proven that this kind of functionality is even generally desirable would be a HUGE error! > > > Yes gnome, kde and perhaps cifs do it. > > Gnome and KDE are userspace utilities, designed to run on several operating > systems that do not have such filesystem plugins at all, plus for the > foreseable future the mayority of Linux (file)systems won't have them > either; so they probably won't ever use it for portability's sake. CIFS is > a second class citizen AFAIU, as it exists for compatibility with legacy > systems only. So none of the above is in any way compelling. > > > But if instead > > they used file plugins a lot more could be shared. The ext2 and ext3 filesystems support the same basic on-disk format, as everyone knows. However, the selection of ext3 provides an on-disk journal, which (if the filesystem is then mounted as ext2), is ignored. You select the "journalling plugin" by mounting the fs as ext3. You turn it off by mounting it as ext2. Reiser4 provides a way to do precisely this kind of selection of variable on-disk formats by specifying a plugin. I ask you -- if everyone in kernel-land is so convinced that you should always select varying on-disk formats via the VFS, then *why* hasn't ext2/ext3 been merged into a single filesystem? Surely the "journalling" plugin of this filesystem is a prime candidate for selection via the VFS? Here's the deal. So Hans can be a butt-head. Get over it. Big projects attract big egos. This is a good thing! Reiser4 provides potentially very interesting functionality, some of which may or may not (over time) be interesting to other FS implementers. At that time, it will be "abstracted" into the VFS. *AFTER* it has been proven safe and useful to do so! Until then, including Reiser4 in the main-line kernel is *completely* consistent with the "do no harm" philosophy that has let other filesystems and drivers have be included. No one running a "stable" server is going to let the "experimental" Reiser4 filesystem code anywhere near their kernel -- just like all other "experimental" code! Others (like me), will use the 'md' drivers to provide the stable, error-free media that ALL filesystems, including reiser4, assume, 'rsync' to provide for backups since it is clearly marked as "experimental", and continue to benefit from the improved atomicity and speed that reiser4 provides -- just as I have been doing for the last 18 months! -- -pjk "If you will not fight when your victory will be sure and not too costly, you may come to the moment when you will have to fight with all the odds against you and only a precarious chance for survival. There may even be a worse case. You may have to fight when there is no hope of victory, because it is better to perish than to live as slaves." -- Winston Churchill ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 17:13 ` Perry Kundert 2005-06-24 20:01 ` Fwd: " Perry Kundert @ 2005-06-24 21:38 ` Valdis.Kletnieks 2005-06-24 22:20 ` Perry Kundert 1 sibling, 1 reply; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-24 21:38 UTC (permalink / raw) To: perry; +Cc: ReiserFS List [-- Attachment #1: Type: text/plain, Size: 2844 bytes --] On Fri, 24 Jun 2005 11:13:45 MDT, Perry Kundert said: > In general, isn't it better to first include modules providing > divergent but possibly interesting functionality (such as Reiser4) as > an "optional" or "experimental" component, and then slowly re-factor > desirable functionality into higher level facilities like the VFS? The problem arises when the facility is something that is demonstrably borked when done in an optional way in one filesystem, and really needs to be done at the VFS level if it is to be done at all. > I ask you -- if everyone in kernel-land is so convinced that you > should always select varying on-disk formats via the VFS, then *why* > hasn't ext2/ext3 been merged into a single filesystem? Because the formats, although similar enough to be mostly compatible, are still different enough that merging them is difficult. There's some very subtle second-order effects, where the ext3 driver can do things in different orders or with different algorithms because it has a journal, when the ext2 code has to do things in a specific way because it has to *always* have things in a consistent enough state that fsck.ext2 can clean things up. So you end up with code that looks like: if (fs->journalled) { /* 500 lines of code for the ext3 case */ } else { /* 300 lines of different code for ext2 */ } If you don't like that, then you can do this instead: 1) put ext2_do_whatever in ext2_whatever.c 2) put ext3_do_whatever in ext3_whatever.c extern ext2_do_whatever(); extern ext3_do_whatever(); if (fs-> journalled) { ext3_do_whatever(); } else { ext2_do_whatever(); } In fact, I seem to remember Alan Cox answering this with "only about 10% of the code *wouldn't* end up like this" or similar... > Surely the > "journalling" plugin of this filesystem is a prime candidate for > selection via the VFS? To be doing "journalling" at the VFS level implies that a journal is something that makes sense at the VFS level - that it's basically filesystem independent, which is most certainly *not* true - the notations an XFS journal needs to make to indicate which blocks were just removed from the free-block structure are quite different from what ext3 needs to record. Note that journalling is neither an attribute of the actual data, or of the user-visible metadata (inode contents, etc). The only things that care about the journalling format/etc are the filesystem driver, the mount command, and the mkfs/fsck commands. As such, it's a file system issue, not a VFS issue. For a good example of why this is so, go back and read the recent discussion of what happens to flash memory filesystems mounted with 'sync' - this was a case of the VFS doing "journalling by flushing" without consulting the low-level drivers.... [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 21:38 ` Valdis.Kletnieks @ 2005-06-24 22:20 ` Perry Kundert 2005-06-25 0:37 ` Valdis.Kletnieks 0 siblings, 1 reply; 620+ messages in thread From: Perry Kundert @ 2005-06-24 22:20 UTC (permalink / raw) To: Valdis.Kletnieks; +Cc: ReiserFS List, Hans Reiser On 6/24/05, Valdis.Kletnieks@vt.edu <Valdis.Kletnieks@vt.edu> wrote: > On Fri, 24 Jun 2005 11:13:45 MDT, Perry Kundert said: > > > In general, isn't it better to first include modules providing > > divergent but possibly interesting functionality (such as Reiser4) as > > an "optional" or "experimental" component, and then slowly re-factor > > desirable functionality into higher level facilities like the VFS? > > The problem arises when the facility is something that is demonstrably > borked when done in an optional way in one filesystem, and really needs > to be done at the VFS level if it is to be done at all. OK, fair enough. The file-as-directory stuff, which introduces VFS-incompatible issues, was turned off. It requires VFS changes. The remaining plugin architecture, as far as I understand, deals in the on-disk structure of the FS -- just like journals. Encryption, Compression, and the like. So, what you are saying, is -- so long as the "plugins" do stuff that deals in how reiser4 slings bits back and forth to the disk, you're OK, right? > > > I ask you -- if everyone in kernel-land is so convinced that you > > should always select varying on-disk formats via the VFS, then *why* > > hasn't ext2/ext3 been merged into a single filesystem? > > Because the formats, although similar enough to be mostly compatible, are > still different enough that merging them is difficult. So, what you are saying is: if reiser4 wants to provide variability in on-disk format, so long as it implements it using *multiple different filesystems* (eg reiser4-cryptcompress, reiser4-whatever) -- just like ext2 vs. ext3 -- then you are OK with it? But, if they are implemented as "plugins", so that the ONE reiser4 filesystem can modulate its behaviour based on what the on-disk format says, that you are NOT OK with it? And this makes sense, why? > There's some very > subtle second-order effects, where the ext3 driver can do things in different > orders or with different algorithms because it has a journal, when the ext2 > code has to do things in a specific way because it has to *always* have things > in a consistent enough state that fsck.ext2 can clean things up. So you end > up with code that looks like: > ... Don't get me wrong -- I'm not saying that ext2 and ext3 shouldn't be separate file systems. However, if they were designed from the start so that the ONE (say) ext23 filesystem could look at its on-disk format, notice that the data specified the "journal" plugin, and implement the correct behaviour -- that this would be "bad"? Because I can envision an ext23 filesystem that is just like reiser4, that does exactly that -- implements its variable behaviour via a "journal" plugin. So, if it did so, would you be OK with it? As long as it wasn't called reiser4? I really don't mean to sound sarcastic -- it just sounds like there are "other" issues at work here -- like "Hans is a Butt-Head, so I want to reject reiser4's plugin design for modulating its behviour, no matter what". > > Surely the > > "journalling" plugin of this filesystem is a prime candidate for > > selection via the VFS? > > To be doing "journalling" at the VFS level implies that a journal is something > that makes sense at the VFS level - that it's basically filesystem independent, > which is most certainly *not* true - the notations an XFS journal needs to make > to indicate which blocks were just removed from the free-block structure are > quite different from what ext3 needs to record. Sort of like crypt/compress, etc. -- each filesystem may or may not do this, but it just doesn't matter to the VFS -- by the time data hits the VFS it isn't encrypted or compressed. So, why not let the FS say "hey, I happen to see that the on-disk format says it uses cryptcompress -- I'll dispatch the right methods from this aptly-named "plugin"... > > Note that journalling is neither an attribute of the actual data, or of > the user-visible metadata (inode contents, etc). The only things that > care about the journalling format/etc are the filesystem driver, the mount > command, and the mkfs/fsck commands. As such, it's a file system issue, > not a VFS issue. > > For a good example of why this is so, go back and read the recent discussion > of what happens to flash memory filesystems mounted with 'sync' - this was > a case of the VFS doing "journalling by flushing" without consulting the > low-level drivers.... > > > OK, so far it seems like we are actually agreeing -- the stuff that gets done via reiser4 "plugins" actually doesn't have anything to do with the VFS, and it shouldn't be there. So long as reiser4 presents a VFS-sensible, VFS-consistent heirarchy of stuff that "looks like" files and directories to the VFS, then we're OK with it? Whether reiser4 uses "plugins", or ESP, or whatever to decide what behaviour it implements in order to produce this VFS-consistent interface, then that's OK, right? -- -pjk ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 22:20 ` Perry Kundert @ 2005-06-25 0:37 ` Valdis.Kletnieks 2005-06-25 0:53 ` Hans Reiser 2005-06-25 2:49 ` David Masover 0 siblings, 2 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-25 0:37 UTC (permalink / raw) To: perry; +Cc: ReiserFS List, Hans Reiser [-- Attachment #1: Type: text/plain, Size: 3981 bytes --] On Fri, 24 Jun 2005 16:20:45 MDT, Perry Kundert said: > OK, fair enough. The file-as-directory stuff, which introduces > VFS-incompatible issues, was turned off. It requires VFS changes. Mind you, I still think that sounds *interesting*, but it *has* to happen at the VFS level. (And if *that* doesn't force a 2.7 fork to happen, nothing will :) > The remaining plugin architecture, as far as I understand, deals > in the on-disk structure of the FS -- just like journals. Encryption, > Compression, and the like. > > So, what you are saying, is -- so long as the "plugins" do stuff > that deals in how reiser4 slings bits back and forth to the disk, > you're OK, right? Right - once the VFS hands the call off to reiser4, you're on your own as far as I'm concerned.. > So, what you are saying is: if reiser4 wants to provide > variability in on-disk format, so long as it implements it using > *multiple different filesystems* (eg reiser4-cryptcompress, > reiser4-whatever) -- just like ext2 vs. ext3 -- then you are OK with > it? But, if they are implemented as "plugins", so that the ONE > reiser4 filesystem can modulate its behaviour based on what the > on-disk format says, that you are NOT OK with it? > > And this makes sense, why? You misread that - my point was that ext2 and ext3 may look similar, but they're sufficiently divergent that trying to create one driver that handles both results in an ugly driver, thus the split... > Don't get me wrong -- I'm not saying that ext2 and ext3 shouldn't > be separate file systems. However, if they were designed from the > start so that the ONE (say) ext23 filesystem could look at its on-disk > format, notice that the data specified the "journal" plugin, and > implement the correct behaviour -- that this would be "bad"? Well, if they *had* been, it would be a different story. And there's stuff in ext3 (see the "-O feature" section of 'man tune2fs') that *does* do the sort of thing that you're proposing. It's just that the ext2 codebase doesn't fit in well for historical reasons. > Because I can envision an ext23 filesystem that is just like > reiser4, that does exactly that -- implements its variable behaviour > via a "journal" plugin. > So, if it did so, would you be OK with it? As long as it wasn't > called reiser4? No, I'd be perfectly happy with a reiser4 that had a 'tunereiser --enable-plugin=' that had the same sort of format-altering semantics that 'tune2fs -O' has. For bonus points, design a system that stores the plugin *in the file system* (probably need to have a bytecode interpreter for this). Then you eliminate the "can't mount if the kernel can't insmod the plugin" issue ;) > I really don't mean to sound sarcastic -- it just sounds like > there are "other" issues at work here -- like "Hans is a Butt-Head, so > I want to reject reiser4's plugin design for modulating its behviour, > no matter what". I've never actually met Hans, so I don't know if he really *is* a butt-head or not. And I usually at least *try* to phrase it more like "this proposal is a non-starter that only a butt-head could continue to support, because.." :) Of course, I myself have been called a butt-head on numerous occasions, because I'm convinced there's a right and wrong way to design something. ;) > OK, so far it seems like we are actually agreeing -- the stuff > that gets done via reiser4 "plugins" actually doesn't have anything to > do with the VFS, and it shouldn't be there. So long as reiser4 > presents a VFS-sensible, VFS-consistent heirarchy of stuff that "looks > like" files and directories to the VFS, then we're OK with it? > > Whether reiser4 uses "plugins", or ESP, or whatever to decide what > behaviour it implements in order to produce this VFS-consistent > interface, then that's OK, right? Right - not that *my* opinion counts for tons on LKML, and any *other* stylistic/design faults are a separate issue. :) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 0:37 ` Valdis.Kletnieks @ 2005-06-25 0:53 ` Hans Reiser 2005-06-25 2:20 ` Valdis.Kletnieks 2005-06-25 2:49 ` David Masover 1 sibling, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-25 0:53 UTC (permalink / raw) To: Valdis.Kletnieks; +Cc: perry, ReiserFS List Valdis.Kletnieks@vt.edu wrote: > Right - once the VFS hands the call off to reiser4, you're on your own > >as far as I'm concerned.. > > Well, that is all I ask for, and Christophe and company disagree..... Happy to abstract it more into VFS after the merge if others want it though.... ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 0:53 ` Hans Reiser @ 2005-06-25 2:20 ` Valdis.Kletnieks 2005-06-25 2:40 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-25 2:20 UTC (permalink / raw) To: Hans Reiser; +Cc: perry, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1065 bytes --] On Fri, 24 Jun 2005 17:53:15 PDT, Hans Reiser said: > Valdis.Kletnieks@vt.edu wrote: > > > Right - once the VFS hands the call off to reiser4, you're on your own > > > >as far as I'm concerned.. > > > > > Well, that is all I ask for, and Christophe and company disagree..... > Happy to abstract it more into VFS after the merge if others want it > though.... Unfortunately, the only *realistic* path I can see at the moment is to strip it down to those reiser4 features that don't require any VFS help/changes to do, and pursue VFS changes as a separate issue entirely. Also, just as a personal note - talking about a "reiser4 plugin that does ext3 backend storage" doesn't help the cause. What you *should* be trying to sell: 1) a reiser4 storage backend with backend plugins that follows the VFS conventions. 2) a vastly enhanced VFS that has VFS plugins and calls reiser4 and ext3 backends. Approaching it that way will make people think that (a) you really *do* understand the VFS/FS layering and (b) are willing to compromise to get things done.. ;) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 2:20 ` Valdis.Kletnieks @ 2005-06-25 2:40 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-25 2:40 UTC (permalink / raw) To: Valdis.Kletnieks; +Cc: Hans Reiser, perry, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: [...] > Also, just as a personal note - talking about a "reiser4 plugin that does ext3 > backend storage" doesn't help the cause. What you *should* be trying to sell: To be fair, I brought that up. I don't remember if Hans endorsed it, but I don't think so. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrzEB3gHNmZLgCUhAQKltBAAn4+NAPUwiIem30RtTuE+BskPY8rhPuCq ExA/lsbC87MffqEHne5xVMAXGiSNfgb23mab3oQiyvnYy2zOgs0rHLchyAinpw/u q/CoZdi8BcSUpR5jlTYf9BVExJ2IPQ3C/MDDTt/LNqOWosipuiauqlKZk9Mvr35+ nKaA0BRhNbDG9E21ukMGR6/NauJJQwaHR0HE3rxwEyJbtJmpQHX2/YuYv5RhAQFH qq0U/67SmIfGBlhxk2fq6OnMR39lvs6kMt2rt6wqPQi9hi8jeAyBh7OXxfjoqpFF S6oI81dL+HyR1aqwyIK3rR+QBIH8iCc0wU8IaRbWws1v7SD+PUJcE9TmgWkRpxep 5bzOYApFsqKTbcCAiTUbPzVeNW1iglNvg9+tCS6cMJ6pRuEAenmJJeRQNRvho1KQ 7PABQPCzV0DPmX11vsOMJNfxmECKPchpglWtxjq0P6xM9XUxhrAy/S7RxvWlYe/q ckt046jFYzZHq29AZxUNRcruqCBk/Px7ZhI6QVEIHOx0J5SM6hBfKSmkj6WYWLKk o1l/vgaI5nb24zGnSueSSLn2utrZdQ8hRNGaCEbogMek90ltO4Zv0BEamcBDoDof 2s+gk4sBQ/muUPJppJ/a317cyV8BCKLknfT1IPa8G9dePWziKbcUNwA2MzR/EzNl VywXtawu9uA= =P9Qs -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 0:37 ` Valdis.Kletnieks 2005-06-25 0:53 ` Hans Reiser @ 2005-06-25 2:49 ` David Masover 1 sibling, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-25 2:49 UTC (permalink / raw) To: Valdis.Kletnieks; +Cc: perry, ReiserFS List, Hans Reiser -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: [...] >> Because I can envision an ext23 filesystem that is just like >>reiser4, that does exactly that -- implements its variable behaviour >>via a "journal" plugin. > > >> So, if it did so, would you be OK with it? As long as it wasn't >>called reiser4? > > > No, I'd be perfectly happy with a reiser4 that had a 'tunereiser --enable-plugin=' > that had the same sort of format-altering semantics that 'tune2fs -O' has. The question is whether you'd be happy if the kernel did that also. mount -t reiser4 -o enable_plugin= I mean, I doubt the kernel team cares at all how the userland utilities work. After all, some of them seem quite happy to delegate ALL of the cool Reiser4 "plugin" things to userspace... And whether it's true or not, implying an anti-reiser conspiracy is not very diplomatic... > For bonus points, design a system that stores the plugin *in the file > system* (probably need to have a bytecode interpreter for this). Then > you eliminate the "can't mount if the kernel can't insmod the plugin" issue ;) Right now, "plugin" isn't quite accurate, as plugins can't even be separate modules, and to install a new plugin, you have to recompile the kernel, or at least the reiser4 module. But, before anyone gets the wrong idea, they are still somewhat "plugins" because you don't have to _use_ any plugins you don't like. If you don't use them, they will just eat kernel RAM -- and a fixed amount of kernel RAM, not at all relative to how big your FS is. I actually like the bytecode idea, except that then you need a bytecode interpreter (or JIT compiler) in the kernel. I like the idea of that, but I'm sure other people would be screaming bloody kernel bloat. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrzGKXgHNmZLgCUhAQKJag/9GxPnsNpaw3Z7xOoiY9kZVoa6vLWALH7T Xher++7289Ra1mYznInBTBjaBdEkk/AWRLzM6Q5Hl4F3d96HzweHsBTgOg/dtscH R3vTpaOQUxbOJWXBrsX/Jw/Q4NJHLDfbr3fg4vsYEgtzzKWkeIOCJgliW5XAB8+W 2Ry2gMj7QxZoQWeWrOPqAEtDudIHZDFFtg0FzWaZw9d0G/jIrAAi3gcGxqIZ4iK+ vNGnzn9K1Sk01TEclvRwh/+5dRMs/liEAlYxa03FB8Lx/2jj9v0WJBnXiPuxztsv hUtIcd9iA4S3AdYXefxHYbSuV2HO7DKyR+WJriyNqk5cpTFecaQ3BqaBCsiPaKlT IrPsm8R3xmsjZ6yB3qIELpe2wg0knhnbwEWMmFnmWUKrMrzpsrI2fcvxlOdcTZKZ 2kRumVNpZ6E0ZhIs5PxhkfBGoVPYMga4hrWNFCW6wuaFwskEPW+hx8su6amqAr1a AJwndr/QLVOo5f20vOk0eOvhjhLW7iZPLPkLJBw+1BnyS4cZuEg7N3pkZsDvKPCe +qqBPv2wGCzYQWlQDTsYriZA7y6UQLmcED/Y2GBlLYQzm8pq20w715/N3LD0G+GK 4lFwSV+hDFoVn5kSJNemPYhgfDT4hAOQyMHTKKY02XnNynAfT4TsWZgzjjSxWjSg QrueuVgBcRs= =8xv0 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 6:49 ` Hans Reiser 2005-06-24 7:10 ` Lincoln Dale @ 2005-06-24 7:11 ` Al Viro 2005-06-24 9:03 ` Hans Reiser 1 sibling, 1 reply; 620+ messages in thread From: Al Viro @ 2005-06-24 7:11 UTC (permalink / raw) To: Hans Reiser Cc: Lincoln Dale, Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Thu, Jun 23, 2005 at 11:49:51PM -0700, Hans Reiser wrote: > Regarding trust, Christophe comes out the gate using the words "useless > abstraction layer" that happens to be a core feature of our design, > demanding we cut it out, and not really understanding it or recognizing > any functionality it provides, and you can't really expect me to respect > this, can you? > > Now, if his target is reduced to whether we can eliminate a function > indirection, and whether we can review the code together and see if it > is easy to extend plugins and pluginids to other filesystems by finding > places to make it more generic and accepting of per filesystem plugins, > especially if it is not tied to going into 2.6.13, well, that is the > conversation I would have liked to have had. Have I missed the posting with analysis of changes in locking scheme and update of proof of correctness? If so, please give the message ID. _That_ had been the major showstopper for any merges, IIRC. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 7:11 ` Al Viro @ 2005-06-24 9:03 ` Hans Reiser 2005-06-24 14:45 ` Al Viro 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-24 9:03 UTC (permalink / raw) To: Al Viro, Alexander Zarochentcev Cc: Lincoln Dale, Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Al Viro wrote: >Have I missed the posting with analysis of changes in locking scheme >and update of proof of correctness? If so, please give the message ID. > >_That_ had been the major showstopper for any merges, IIRC. > > Ah, the prince of helpfulness has arrived. Yes, as I remember, last time with V3 you announced that there were race conditions that we needed to fix if V3 was to be merged, you would not tell us what they were when asked, Linus merged us anyway, you never did tell us what they were, later vs fixed some race conditions but I have no idea if they were the same ones you found, oh well, getting rid of bugs never was your objective was it? Does V3 still have those race conditions you spoke of? Proof of correctness, is that where we check and see if the filesystems all mount/unmount before checking in code changes to the stable release branch? Oh dear, that was unkind of me. Ok, sure, define what you want in the way of a proof of correctness and an analysis. Is this a new VFS tradition? Is it documented anywhere? Are there tools for it? Probably I should ask Zam if we already did it too..... Zam? Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 9:03 ` Hans Reiser @ 2005-06-24 14:45 ` Al Viro 2005-06-24 14:53 ` Al Viro 2005-06-24 18:16 ` Hans Reiser 0 siblings, 2 replies; 620+ messages in thread From: Al Viro @ 2005-06-24 14:45 UTC (permalink / raw) To: Hans Reiser Cc: Alexander Zarochentcev, Lincoln Dale, Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Fri, Jun 24, 2005 at 02:03:33AM -0700, Hans Reiser wrote: > Al Viro wrote: > > >Have I missed the posting with analysis of changes in locking scheme > >and update of proof of correctness? If so, please give the message ID. > > > >_That_ had been the major showstopper for any merges, IIRC. > > > > > Ah, the prince of helpfulness has arrived. > > Yes, as I remember, Kindly put some efforts into remembering the thread that contains e.g. http://marc.theaimsgroup.com/?l=linux-kernel&m=109347094309283&w=2 If that work (summary: introduction of hybrid objects invalidates the existing locking scheme for directories and that had lead at least to several user-exploitable deadlocks described in details in the same thread; current proof of correctness is in the tree, see Documentation/filesystems/Directory-Locking.txt and at the very least it needs to be updated) had been done - please, give the message ID of posting with such update. If not - please, arrange getting it done. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 14:45 ` Al Viro @ 2005-06-24 14:53 ` Al Viro 2005-06-24 18:16 ` Hans Reiser 1 sibling, 0 replies; 620+ messages in thread From: Al Viro @ 2005-06-24 14:53 UTC (permalink / raw) To: Hans Reiser Cc: Alexander Zarochentcev, Lincoln Dale, Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Fri, Jun 24, 2005 at 03:45:23PM +0100, Al Viro wrote: > On Fri, Jun 24, 2005 at 02:03:33AM -0700, Hans Reiser wrote: > > Al Viro wrote: > > > > >Have I missed the posting with analysis of changes in locking scheme > > >and update of proof of correctness? If so, please give the message ID. > > > > > >_That_ had been the major showstopper for any merges, IIRC. > > > > > > > > Ah, the prince of helpfulness has arrived. > > > > Yes, as I remember, > > Kindly put some efforts into remembering the thread that contains e.g. > http://marc.theaimsgroup.com/?l=linux-kernel&m=109347094309283&w=2 > > If that work (summary: introduction of hybrid objects invalidates the > existing locking scheme for directories and that had lead at least to > several user-exploitable deadlocks described in details in the same > thread; current proof of correctness is in the tree, see > Documentation/filesystems/Directory-Locking.txt and at the very least gaaack... Sorry - should've proofread better - s/Directory-Locking.txt/directory-locking/ in the above. > it needs to be updated) had been done - please, give the message ID > of posting with such update. If not - please, arrange getting it done. > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 14:45 ` Al Viro 2005-06-24 14:53 ` Al Viro @ 2005-06-24 18:16 ` Hans Reiser 1 sibling, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-24 18:16 UTC (permalink / raw) To: Al Viro Cc: Alexander Zarochentcev, Lincoln Dale, Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Al Viro wrote: >On Fri, Jun 24, 2005 at 02:03:33AM -0700, Hans Reiser wrote: > > >>Al Viro wrote: >> >> >> >>>Have I missed the posting with analysis of changes in locking scheme >>>and update of proof of correctness? If so, please give the message ID. >>> >>>_That_ had been the major showstopper for any merges, IIRC. >>> >>> >>> >>> >>Ah, the prince of helpfulness has arrived. >> >>Yes, as I remember, >> >> > >Kindly put some efforts into remembering the thread that contains e.g. >http://marc.theaimsgroup.com/?l=linux-kernel&m=109347094309283&w=2 > >If that work (summary: introduction of hybrid objects invalidates the >existing locking scheme for directories and that had lead at least to >several user-exploitable deadlocks described in details in the same >thread; current proof of correctness is in the tree, see >Documentation/filesystems/Directory-Locking.txt and at the very least >it needs to be updated) had been done - please, give the message ID >of posting with such update. If not - please, arrange getting it done. > > > > We disabled metafiles until we can fix this. With no metafiles, the issue does not exist. Thanks for your assistance with finding that problem. Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 1:12 ` Hans Reiser 2005-06-24 1:45 ` Jeff Garzik 2005-06-24 2:31 ` Lincoln Dale @ 2005-06-24 10:41 ` Alan Cox 2005-06-27 9:18 ` mjt 2 siblings, 1 reply; 620+ messages in thread From: Alan Cox @ 2005-06-24 10:41 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Gwe, 2005-06-24 at 02:12, Hans Reiser wrote: > >In which case the features belong in the VFS as all those with > >experience and kernel contributions have been arguing. > So you fundamentally reject the prototype it in one fs and then abstract > it to others development model? More fundamentally - prototype things *out* of the main kernel. If everyone was doing their prototyping in kernel Andrew Morton would by now be a team of about 25 Alan ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 10:41 ` Alan Cox @ 2005-06-27 9:18 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-27 9:18 UTC (permalink / raw) To: Alan Cox Cc: Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 2587 bytes --] On Fri, Jun 24, 2005 at 11:41:21AM +0100, Alan Cox wrote: > >More fundamentally - prototype things *out* of the main kernel. If >everyone was doing their prototyping in kernel Andrew Morton would by >now be a team of about 25 This is going semi off-topic but how then do you justify regression that's apparently confessed? :) It appears -mm is more of a prototype tree than a development tree; I remember Con Kolivas cursing (with SMP nice support) that the interface is too lively to keep supporting one patch. Unfortunately I can't find the original post I'm thinking about but http://lkml.org/lkml/2005/5/16/68 says essentially the same thing. There's also my all-time favorite, http://lkml.org/lkml/2005/3/14/4 The lack of QA seems appalling here, and I'm sure Reiser has had to do more of that for DARPA than most linux kernel hackers around. What I'm saying here is isn't it a bit hypocritic to ramble on that we need real assurances for this to work, community proof and bugfixes on the list mean nothing, when other core systems seem to be much looser in this respect? Sure, "other people merge design-breakers and bugs" is NOT a justification for merging Reiser4, of course, but Reiser4's track record has vastly cleaned itself up. Most bug reports come from broken hardware or unsupported patches or old code. Just have Namesys swear they won't introduce design changes until the userland utils are available, and won't do it at all after EXPERIMENTAL has been removed. They already did this with changes that require mkfs :> Here's a real suggestion, for LKML et al, dropping the sarcastic mode. 0. Namesys addresses the code beautification reqs mentioned here earlier. 1. Merge Reiser4 sans metas into 2.6.13. 2. Namesys can have a separate metas patch for testing. 3. Gradually merge Reiser4 architecture into VFS and if this really requires a 2.7, as (iirc) Valdis Kletnieks suggested, make it so. Might do the rest of the kernel some good too. The above is a lame compromise; I'd much rather have the meta system in, as is dropping the stupid name, ..metas/ is better, and dropping "meta files can have meta data, ie. endless recursion in ..metas/ Then it can be merged into the VFS as needed, but if there truly are issues left, I realize it can't probably be fixed fast enough. Having metas on by default probably leads to a lot of bug reports, which get fixed, and at the same time makes VFS-merging easier, when metas-related bug issues are fixed in VFS. My two grumpy cents. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-27 9:18 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-27 9:18 UTC (permalink / raw) To: Alan Cox Cc: Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 2587 bytes --] On Fri, Jun 24, 2005 at 11:41:21AM +0100, Alan Cox wrote: > >More fundamentally - prototype things *out* of the main kernel. If >everyone was doing their prototyping in kernel Andrew Morton would by >now be a team of about 25 This is going semi off-topic but how then do you justify regression that's apparently confessed? :) It appears -mm is more of a prototype tree than a development tree; I remember Con Kolivas cursing (with SMP nice support) that the interface is too lively to keep supporting one patch. Unfortunately I can't find the original post I'm thinking about but http://lkml.org/lkml/2005/5/16/68 says essentially the same thing. There's also my all-time favorite, http://lkml.org/lkml/2005/3/14/4 The lack of QA seems appalling here, and I'm sure Reiser has had to do more of that for DARPA than most linux kernel hackers around. What I'm saying here is isn't it a bit hypocritic to ramble on that we need real assurances for this to work, community proof and bugfixes on the list mean nothing, when other core systems seem to be much looser in this respect? Sure, "other people merge design-breakers and bugs" is NOT a justification for merging Reiser4, of course, but Reiser4's track record has vastly cleaned itself up. Most bug reports come from broken hardware or unsupported patches or old code. Just have Namesys swear they won't introduce design changes until the userland utils are available, and won't do it at all after EXPERIMENTAL has been removed. They already did this with changes that require mkfs :> Here's a real suggestion, for LKML et al, dropping the sarcastic mode. 0. Namesys addresses the code beautification reqs mentioned here earlier. 1. Merge Reiser4 sans metas into 2.6.13. 2. Namesys can have a separate metas patch for testing. 3. Gradually merge Reiser4 architecture into VFS and if this really requires a 2.7, as (iirc) Valdis Kletnieks suggested, make it so. Might do the rest of the kernel some good too. The above is a lame compromise; I'd much rather have the meta system in, as is dropping the stupid name, ..metas/ is better, and dropping "meta files can have meta data, ie. endless recursion in ..metas/ Then it can be merged into the VFS as needed, but if there truly are issues left, I realize it can't probably be fixed fast enough. Having metas on by default probably leads to a lot of bug reports, which get fixed, and at the same time makes VFS-merging easier, when metas-related bug issues are fixed in VFS. My two grumpy cents. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 9:18 ` mjt (?) @ 2005-06-27 9:46 ` Nick Piggin 2005-06-27 12:55 ` mjt -1 siblings, 1 reply; 620+ messages in thread From: Nick Piggin @ 2005-06-27 9:46 UTC (permalink / raw) To: Markus Törnqvist Cc: Alan Cox, Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Markus Törnqvist wrote: > I can't find the original post I'm thinking about but > http://lkml.org/lkml/2005/5/16/68 says essentially the same thing. The scheduler is being improved for better behaviour on complex topologies like multi core + NUMA and multi level NUMA systems. If Con's work had gone in first, then conversely these improvements would have had to wait. > There's also my all-time favorite, http://lkml.org/lkml/2005/3/14/4 > What's wrong with that? The slowdown is due to the workload becoming disk bound. The reasons are still not entirely clear, but I don't think it is a recent (ie. 2.6) regression (or even a regression at all IIRC). > The lack of QA seems appalling here, and I'm sure Reiser has had > to do more of that for DARPA than most linux kernel hackers around. > And what QA would you have preferred? I think if you are resorting to bringing up all time favourite blunders when trying to justify Reiser4 being included, then that is a sign right there that something is fundamentally wrong (if not with the code, then with your line of thought0 And note my email has nothing to do with any *real* argument for or against R4. Thanks, Nick Send instant messages to your online friends http://au.messenger.yahoo.com ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 9:46 ` Nick Piggin @ 2005-06-27 12:55 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-27 12:55 UTC (permalink / raw) To: Nick Piggin Cc: Alan Cox, Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 3508 bytes --] On Mon, Jun 27, 2005 at 07:46:15PM +1000, Nick Piggin wrote: > >The scheduler is being improved for better behaviour on complex >topologies like multi core + NUMA and multi level NUMA systems. >If Con's work had gone in first, then conversely these improvements >would have had to wait. Or be merged in later. The problem is, why do the interfaces have to live so much that examples like this come to my ears (or eyes ;) all the time? I hate to say this without digging out any URLs, but one friend of mine says he has a very hard time doing any networking code because it's too labile. Maybe that's being embettered for something else too? Or the other friend who curses that the networking code is just crap and basically has to rewrite the code to get it working. Yes, I've tried to get these guys to submit their code, but they argue back that no one wants to see it. >>There's also my all-time favorite, http://lkml.org/lkml/2005/3/14/4 > >What's wrong with that? The slowdown is due to the workload >becoming disk bound. The reasons are still not entirely clear, >but I don't think it is a recent (ie. 2.6) regression (or even >a regression at all IIRC). It's not that easy. So let's say the scheduler slowdown is a temporary situation to adapt it to multicore+NUMA. I assume that's what you mean, by having the kernels on par with 2.6.10 and the above paragraph. Why on earth did the slowdown ever get merged and we have to wait for it to be on par with some older version? Maybe the multicore+NUMA guys don't think it's a regression, hell, it may be better for them at the cost of the embedded (or desktop or whoever) guys. Still my initial reaction is "if a patch slows things down, revert it. If it didn't do anything, keep reverting until you have the original situation." It's also my reaction after the initial one :) Sure, 2.4 and others did also have issues, so you couldn't automatically assume each and every new kernel would be better than the last, but it seems nowadays it's also hard to predict even the trend. But to be fair, Linux is not a disaster yet and you guys are doing good work, despite things looking very shaky at times and my temper being more and more easily flared nowadays at signs of trouble :) >I think if you are resorting to bringing up all time favourite >blunders when trying to justify Reiser4 being included, then >that is a sign right there that something is fundamentally wrong >(if not with the code, then with your line of thought0 Myeah, I know, this is not helping the Reiser4 cause _as_such_. What I'm saying is, Reiser4 could be merged as it looks like a lot of other stuff is merged too, which may not be very tested. Reiser4 at least is tested. We can skip the parts that appear to suck. >And note my email has nothing to do with any *real* argument for >or against R4. I think this could be further split into an argument about the development model :P Having a separate dev tree would keep the majority happy, as things are "good enough for most people" and after all has been evened out in the dev tree, a new stable tree can be launched. This particular new file system should not be too drastic a thing to merge into a stable kernel, without the instability- inducing factors, and I think in the old model, all this would have been fleshed out in the dev tree automagically. Please do correct me if I'm wrong :) Thanks! -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-27 12:55 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-27 12:55 UTC (permalink / raw) To: Nick Piggin Cc: Alan Cox, Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 3508 bytes --] On Mon, Jun 27, 2005 at 07:46:15PM +1000, Nick Piggin wrote: > >The scheduler is being improved for better behaviour on complex >topologies like multi core + NUMA and multi level NUMA systems. >If Con's work had gone in first, then conversely these improvements >would have had to wait. Or be merged in later. The problem is, why do the interfaces have to live so much that examples like this come to my ears (or eyes ;) all the time? I hate to say this without digging out any URLs, but one friend of mine says he has a very hard time doing any networking code because it's too labile. Maybe that's being embettered for something else too? Or the other friend who curses that the networking code is just crap and basically has to rewrite the code to get it working. Yes, I've tried to get these guys to submit their code, but they argue back that no one wants to see it. >>There's also my all-time favorite, http://lkml.org/lkml/2005/3/14/4 > >What's wrong with that? The slowdown is due to the workload >becoming disk bound. The reasons are still not entirely clear, >but I don't think it is a recent (ie. 2.6) regression (or even >a regression at all IIRC). It's not that easy. So let's say the scheduler slowdown is a temporary situation to adapt it to multicore+NUMA. I assume that's what you mean, by having the kernels on par with 2.6.10 and the above paragraph. Why on earth did the slowdown ever get merged and we have to wait for it to be on par with some older version? Maybe the multicore+NUMA guys don't think it's a regression, hell, it may be better for them at the cost of the embedded (or desktop or whoever) guys. Still my initial reaction is "if a patch slows things down, revert it. If it didn't do anything, keep reverting until you have the original situation." It's also my reaction after the initial one :) Sure, 2.4 and others did also have issues, so you couldn't automatically assume each and every new kernel would be better than the last, but it seems nowadays it's also hard to predict even the trend. But to be fair, Linux is not a disaster yet and you guys are doing good work, despite things looking very shaky at times and my temper being more and more easily flared nowadays at signs of trouble :) >I think if you are resorting to bringing up all time favourite >blunders when trying to justify Reiser4 being included, then >that is a sign right there that something is fundamentally wrong >(if not with the code, then with your line of thought0 Myeah, I know, this is not helping the Reiser4 cause _as_such_. What I'm saying is, Reiser4 could be merged as it looks like a lot of other stuff is merged too, which may not be very tested. Reiser4 at least is tested. We can skip the parts that appear to suck. >And note my email has nothing to do with any *real* argument for >or against R4. I think this could be further split into an argument about the development model :P Having a separate dev tree would keep the majority happy, as things are "good enough for most people" and after all has been evened out in the dev tree, a new stable tree can be launched. This particular new file system should not be too drastic a thing to merge into a stable kernel, without the instability- inducing factors, and I think in the old model, all this would have been fleshed out in the dev tree automagically. Please do correct me if I'm wrong :) Thanks! -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 12:55 ` mjt (?) @ 2005-06-28 0:23 ` Nick Piggin 2005-06-28 20:39 ` mjt -1 siblings, 1 reply; 620+ messages in thread From: Nick Piggin @ 2005-06-28 0:23 UTC (permalink / raw) To: Markus Törnqvist Cc: Alan Cox, Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Markus Törnqvist wrote: > On Mon, Jun 27, 2005 at 07:46:15PM +1000, Nick Piggin wrote: > >>The scheduler is being improved for better behaviour on complex >>topologies like multi core + NUMA and multi level NUMA systems. >>If Con's work had gone in first, then conversely these improvements >>would have had to wait. > > > Or be merged in later. > It _was_ merged later. Con was talking about Andrew's -mm tree. > The problem is, why do the interfaces have to live so much that > examples like this come to my ears (or eyes ;) all the time? > The scheduler interfaces are probably among the most stable in the kernel. Con was talking about internal implementation. > I hate to say this without digging out any URLs, but one friend > of mine says he has a very hard time doing any networking code > because it's too labile. Maybe that's being embettered for something > else too? > It seems there are plenty of people who can do networking code just fine. > Or the other friend who curses that the networking code is just > crap and basically has to rewrite the code to get it working. > Yes, I've tried to get these guys to submit their code, but they > argue back that no one wants to see it. > I get the feeling your friend is making up some tall tales if he says he rewrote the networking code to be much better than it is now. But I can guarantee him that if he has it will be of great interest to the network developers. > >>>There's also my all-time favorite, http://lkml.org/lkml/2005/3/14/4 >> >>What's wrong with that? The slowdown is due to the workload >>becoming disk bound. The reasons are still not entirely clear, >>but I don't think it is a recent (ie. 2.6) regression (or even >>a regression at all IIRC). > > > It's not that easy. > > So let's say the scheduler slowdown is a temporary situation to > adapt it to multicore+NUMA. > I assume that's what you mean, by having the kernels on par > with 2.6.10 and the above paragraph. I'm pretty sure it isn't a scheduler slowdown, but rather the system is getting disk bound. I think the reporter has performance back to 2.6.10 levels - it is actually something I still have to follow up on, but it is difficult because the reporter isn't able to share his code, although he's otherwise very helpful. > > Why on earth did the slowdown ever get merged and we have to wait > for it to be on par with some older version? > If slowdowns do get merged, it is generally because they either haven't been reported until being merged or they make an accepted tradeoff. Not that they haven't been tested. Let's just get this straight, no amount of QA other than releasing a kernel and asking people to use it is going to find all the bugs that will be encountered. > > Please do correct me if I'm wrong :) > I mostly agree with you apart from your specific examples I think we could do things better, and most of us have made some big blunders, However there is just no way that bringing any of that up will help Reiser4 being merged. Send instant messages to your online friends http://au.messenger.yahoo.com ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 0:23 ` Nick Piggin @ 2005-06-28 20:39 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-28 20:39 UTC (permalink / raw) To: Nick Piggin Cc: Alan Cox, Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 5986 bytes --] On Tue, Jun 28, 2005 at 10:23:19AM +1000, Nick Piggin wrote: >The scheduler interfaces are probably among the most stable >in the kernel. Con was talking about internal implementation. Hmm, I'm by no means a kernel developer, but looking at the patches, it looked like the functions were just renamed and maybe something else was changed. I may be totally wrong here, and I apologize in advance if I am, but it seems to me like there would be no real need in changing some syntax? >I get the feeling your friend is making up some tall tales if he >says he rewrote the networking code to be much better than it is >now. But I can guarantee him that if he has it will be of great >interest to the network developers. Yeah, I just talked this over with him. [ IF THIS IS BULLSHIT, LET ME KNOW AND I'LL CORRECT HIM ;) POINTERS TO RELEVANT DOCUMENTATION ALSO WELCOME ;) ] His grudge was that conntrack was used everywhere, not like in 2.4, with a slow lookup in proc, that made it unusable for large nat systems. They wrote some reimplementation of the lookup part, which they didn't release. Then they figured out how to capture the packets before reaching conntrack and doing some mojo on them and reinjecting them somewhere later or bypassing conntrack altogether. However, his opinions were even less motivated than mine, it seems, as he didn't have anything except some ipfilter archives he read, where they cursed the Linux way of doing things. I haven't looked all too deeply into the network part, though, as I'm starting to get lazy and like to have a separate, dedicated piece of hardware do the network stuff for me... So maybe I should just retract my comments and look into every possible subsystem's development and see for myself, which I don't have time for :) >I'm pretty sure it isn't a scheduler slowdown, but rather the >system is getting disk bound. I think the reporter has >performance back to 2.6.10 levels - it is actually something I >still have to follow up on, but it is difficult because the >reporter isn't able to share his code, although he's otherwise >very helpful. It's just that for cases like these testing would be great. I'm not really sure how to do it, though, as it would require a lot of hardware and time and someone who can write a decent test suite. If anyone has links to good test suites, I'm all ears :) So, any chances of something being messed up in the disk part of the code? ...Too bad I just heard a big slowdown-in-disk-i/o curse was caused by dma falling off, for whatever reason (bug?) so that can't be used as forensics either :P >If slowdowns do get merged, it is generally because they either >haven't been reported until being merged or they make an accepted >tradeoff. Not that they haven't been tested. > >Let's just get this straight, no amount of QA other than releasing >a kernel and asking people to use it is going to find all the bugs >that will be encountered. Would you think a separate dev tree would help here? I've also talked long with a guy from one of our major customers who says he's been losing faith in Linux over the past year, as his boxes are going oops or panicking every once in a while. The problem is that their system is too redundantly set up for a single oops or panic to matter much, so I haven't convinced them to report bugs. They'd rather boot the boxes and curse. He just figured it odd that this started happening after the new dev model, but the timelines are too granular for a real correlation. Just a strong gut feeling, and that's the kind of FUD that makes people look to *BSD, and a gut feeling that may have some merit to it, although it could never be proven. So don't take that as a flame or anything, just an observation :P Sure it must be difficult developing the kernel, if it were easier, I'd probably be at it too, but people don't like not being well informed of accepted tradeoffs, as they might not accept said tradeoff, and tracking what caused it may be too much. A lot of people out there don't like vendor kernels either, with 65536 backported patches and the 65537th voids your warranty. It's also probably very difficult to do analytical QA before submitting code, as bugs ALWAYS run through. But my gut tells me this -mm thing may not be the optimal way, as people are more scared of -mm, where some things may or may not work, than a separate dev tree, of which kerneltrap or whoever can post "this is severely broken, test if you like" and "this seems to be going mostly strong now, test it on your everyday desktop" A lot of complaints seem to come from people who use patchsets with half-assed backports and experimental code, having a separate tree may drive these people to use the experimental stuff knowing what it is, instead of believing their patch set is as good as vanilla. >I mostly agree with you apart from your specific examples I think >we could do things better, and most of us have made some big blunders, >However there is just no way that bringing any of that up will help >Reiser4 being merged. Myeah, if I try to keep it a bit more on topic, I'm just saying that Reiser4 (sans metas) has been heavily tested, and now that the Namesys guys are apparently beautifying the code, we should be good with everything. Still, this is an instinct, but it seems like the bigger VFS overhauls would be easier to do in an official dev tree with a big note saying "THIS WILL BE BROKEN FOR A MONTH." If this was the case, so bad that people couldn't develop anything with the dev tree, it would probably be easier to maintain a rollback-vfs-changes patch than constantly keeping in sync with -mm. That method could be used for every system undergoing extensive work that shows up in other systems. The difference may not be big, but it may be enough. Thanks! -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-28 20:39 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-28 20:39 UTC (permalink / raw) To: Nick Piggin Cc: Alan Cox, Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 5986 bytes --] On Tue, Jun 28, 2005 at 10:23:19AM +1000, Nick Piggin wrote: >The scheduler interfaces are probably among the most stable >in the kernel. Con was talking about internal implementation. Hmm, I'm by no means a kernel developer, but looking at the patches, it looked like the functions were just renamed and maybe something else was changed. I may be totally wrong here, and I apologize in advance if I am, but it seems to me like there would be no real need in changing some syntax? >I get the feeling your friend is making up some tall tales if he >says he rewrote the networking code to be much better than it is >now. But I can guarantee him that if he has it will be of great >interest to the network developers. Yeah, I just talked this over with him. [ IF THIS IS BULLSHIT, LET ME KNOW AND I'LL CORRECT HIM ;) POINTERS TO RELEVANT DOCUMENTATION ALSO WELCOME ;) ] His grudge was that conntrack was used everywhere, not like in 2.4, with a slow lookup in proc, that made it unusable for large nat systems. They wrote some reimplementation of the lookup part, which they didn't release. Then they figured out how to capture the packets before reaching conntrack and doing some mojo on them and reinjecting them somewhere later or bypassing conntrack altogether. However, his opinions were even less motivated than mine, it seems, as he didn't have anything except some ipfilter archives he read, where they cursed the Linux way of doing things. I haven't looked all too deeply into the network part, though, as I'm starting to get lazy and like to have a separate, dedicated piece of hardware do the network stuff for me... So maybe I should just retract my comments and look into every possible subsystem's development and see for myself, which I don't have time for :) >I'm pretty sure it isn't a scheduler slowdown, but rather the >system is getting disk bound. I think the reporter has >performance back to 2.6.10 levels - it is actually something I >still have to follow up on, but it is difficult because the >reporter isn't able to share his code, although he's otherwise >very helpful. It's just that for cases like these testing would be great. I'm not really sure how to do it, though, as it would require a lot of hardware and time and someone who can write a decent test suite. If anyone has links to good test suites, I'm all ears :) So, any chances of something being messed up in the disk part of the code? ...Too bad I just heard a big slowdown-in-disk-i/o curse was caused by dma falling off, for whatever reason (bug?) so that can't be used as forensics either :P >If slowdowns do get merged, it is generally because they either >haven't been reported until being merged or they make an accepted >tradeoff. Not that they haven't been tested. > >Let's just get this straight, no amount of QA other than releasing >a kernel and asking people to use it is going to find all the bugs >that will be encountered. Would you think a separate dev tree would help here? I've also talked long with a guy from one of our major customers who says he's been losing faith in Linux over the past year, as his boxes are going oops or panicking every once in a while. The problem is that their system is too redundantly set up for a single oops or panic to matter much, so I haven't convinced them to report bugs. They'd rather boot the boxes and curse. He just figured it odd that this started happening after the new dev model, but the timelines are too granular for a real correlation. Just a strong gut feeling, and that's the kind of FUD that makes people look to *BSD, and a gut feeling that may have some merit to it, although it could never be proven. So don't take that as a flame or anything, just an observation :P Sure it must be difficult developing the kernel, if it were easier, I'd probably be at it too, but people don't like not being well informed of accepted tradeoffs, as they might not accept said tradeoff, and tracking what caused it may be too much. A lot of people out there don't like vendor kernels either, with 65536 backported patches and the 65537th voids your warranty. It's also probably very difficult to do analytical QA before submitting code, as bugs ALWAYS run through. But my gut tells me this -mm thing may not be the optimal way, as people are more scared of -mm, where some things may or may not work, than a separate dev tree, of which kerneltrap or whoever can post "this is severely broken, test if you like" and "this seems to be going mostly strong now, test it on your everyday desktop" A lot of complaints seem to come from people who use patchsets with half-assed backports and experimental code, having a separate tree may drive these people to use the experimental stuff knowing what it is, instead of believing their patch set is as good as vanilla. >I mostly agree with you apart from your specific examples I think >we could do things better, and most of us have made some big blunders, >However there is just no way that bringing any of that up will help >Reiser4 being merged. Myeah, if I try to keep it a bit more on topic, I'm just saying that Reiser4 (sans metas) has been heavily tested, and now that the Namesys guys are apparently beautifying the code, we should be good with everything. Still, this is an instinct, but it seems like the bigger VFS overhauls would be easier to do in an official dev tree with a big note saying "THIS WILL BE BROKEN FOR A MONTH." If this was the case, so bad that people couldn't develop anything with the dev tree, it would probably be easier to maintain a rollback-vfs-changes patch than constantly keeping in sync with -mm. That method could be used for every system undergoing extensive work that shows up in other systems. The difference may not be big, but it may be enough. Thanks! -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 12:55 ` mjt (?) (?) @ 2005-06-30 23:20 ` Jesper Juhl -1 siblings, 0 replies; 620+ messages in thread From: Jesper Juhl @ 2005-06-30 23:20 UTC (permalink / raw) To: Markus Törnqvist Cc: Nick Piggin, Alan Cox, Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On 6/27/05, Markus Törnqvist <mjt@nysv.org> wrote: [...] > > I hate to say this without digging out any URLs, but one friend > of mine says he has a very hard time doing any networking code > because it's too labile. Maybe that's being embettered for something > else too? > > Or the other friend who curses that the networking code is just > crap and basically has to rewrite the code to get it working. > Yes, I've tried to get these guys to submit their code, but they > argue back that no one wants to see it. > [...] I'm pretty damn sure the relevant maintainers (and a bunch of people on LKML and the netdev lists in general) would like to see patches that improves their code. If these friends of yours are sitting on patches that make massive improvements to the code and they are not submitting patches then they are not exactely helping (and I'd suspect them to just be full of BS) - they should get their code merged instead of having to maintain it themselves out-of-tree for ever - let the rest of us bennefit as well. It's my experience that if you can explain the problems your patch fix and explain well why the fix is sane, then getting your patches merged doesn't have to be hard. -- Jesper Juhl <jesper.juhl@gmail.com> Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html Plain text mails only, please http://www.expita.com/nomime.html ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 9:18 ` mjt @ 2005-06-27 14:01 ` Alan Cox -1 siblings, 0 replies; 620+ messages in thread From: Alan Cox @ 2005-06-27 14:01 UTC (permalink / raw) To: Markus Törnqvist Cc: Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Llu, 2005-06-27 at 10:18, Markus Törnqvist wrote: > Sure, "other people merge design-breakers and bugs" is NOT a justification > for merging Reiser4, of course, but Reiser4's track record has vastly > cleaned itself up. The discussion is about merging from -mm, not into -mm. The merge into -mm isn't at all under discussion so I don't the relevance of the comparison. > 0. Namesys addresses the code beautification reqs mentioned here earlier. > > 1. Merge Reiser4 sans metas into 2.6.13. > 2. Namesys can have a separate metas patch for testing. > 3. Gradually merge Reiser4 architecture into VFS and if this really > requires a 2.7, as (iirc) Valdis Kletnieks suggested, make it so. > Might do the rest of the kernel some good too. 0 and 1 look like the first right steps to take to me as well. That will allow people to use reiser4 and give it a good hammering see what comes out and how it benches in real life. Alan ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-27 14:01 ` Alan Cox 0 siblings, 0 replies; 620+ messages in thread From: Alan Cox @ 2005-06-27 14:01 UTC (permalink / raw) To: Markus Törnqvist Cc: Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Llu, 2005-06-27 at 10:18, Markus Törnqvist wrote: > Sure, "other people merge design-breakers and bugs" is NOT a justification > for merging Reiser4, of course, but Reiser4's track record has vastly > cleaned itself up. The discussion is about merging from -mm, not into -mm. The merge into -mm isn't at all under discussion so I don't the relevance of the comparison. > 0. Namesys addresses the code beautification reqs mentioned here earlier. > > 1. Merge Reiser4 sans metas into 2.6.13. > 2. Namesys can have a separate metas patch for testing. > 3. Gradually merge Reiser4 architecture into VFS and if this really > requires a 2.7, as (iirc) Valdis Kletnieks suggested, make it so. > Might do the rest of the kernel some good too. 0 and 1 look like the first right steps to take to me as well. That will allow people to use reiser4 and give it a good hammering see what comes out and how it benches in real life. Alan ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 23:43 ` Alan Cox 2005-06-24 1:12 ` Hans Reiser @ 2005-06-24 3:17 ` David Masover 2005-06-24 3:34 ` Horst von Brand ` (3 more replies) 1 sibling, 4 replies; 620+ messages in thread From: David Masover @ 2005-06-24 3:17 UTC (permalink / raw) To: Alan Cox Cc: Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Alan Cox wrote: > On Iau, 2005-06-23 at 23:04, David Masover wrote: > >>>What for? It works just fine as it stands, AFAICS. >> >>So does DOS. Do you use DOS? I don't even use DOS to run DOS programs. > > > False argument. So does the pen, so do hinges on doors. Do you still > have hinges on your doors - probably. Indeed. Because there's nothing better -- not because I "like it the way it is". >>"Ain't broke" is the battle cry of stagnation. > > > Its also the battle cry of everyone over the age of 20 who also has a > real job to do 8) You caught me. I'm not over 20. But I have a real job, with a company that understands the difference between "ain't broke" and "works well". >>But, there are some things Reiser does better and faster than ext3, even >>if you don't count file-as-directory and other toys. There's nothing >>ext3 does better than Reiser, unless you count the compatibility with >>random bootloaders and low-level tools. > > > Certainly compared with reiser3 you've missed a few out including > resilience to disk errors (nearly nil on reiser3), and SMP scaling. Actually, I was talking about reiser4. And Hans corrected me on that... Although resilience to disk errors isn't a design decision. That's what SMART and new hard drives are for. And if you're stubborn enough to keep the same FS around, there's dm-bbr. I think Hans (or someone) decided that when hardware stops working, it's not the job of the FS to compensate, it's the job of lower layers, or better, the job of the admin to replace the disk and restore from backups. >>You know how many I've had thrashed on Reiser4? Two. The first one was >>with a VERY early alpha/beta, and the second one was when I dropped a >>laptop and the disk failed. > > > Entirely or bad blocks ? The latter should have a minimal cost on a well > designed fs. I was able to recover from bad blocks, though of course no Reiser that I know of has had bad block relocation built in... But I got all my files off of it, fortunately. But the disk did fail, completely, later. Lots of loud clicking. >>Duplication of effort. With plugins, we can optimize the upper layers >>of ALL filesystems, regardless of the lower layers, in such a way that > > > In which case the features belong in the VFS as all those with > experience and kernel contributions have been arguing. No one's arguing that. What we're arguing is that there does seem to be a bit of prejudice when fuck-me-with-a-chainsaw and YOU ARE A FOOL FOR ENABLING THIS all got in, and with at least one of those, there doesn't seem to be any intention of changing it -- all of that, and even with an expressed intention to fix the aesthetical problems with Reiser4 later, we can't get the working version in now. More infuriatingly, I, at least, have a distinct feeling that once all the issues are fixed, an entirely different crowd of benevolent dictators will come around and say that we can't get in because we change the VFS. At least some people on this list have said things to that effect. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrt7MngHNmZLgCUhAQKdXA/+PDpOzZYTVXgF2n4qiFyrmjFeQ6h0n8i6 c/hXx+QUU0Hw5mjq31+jf2vNpDCKQxcE/HTLdJlRfw8az+xklVOfxzEHf9yV41tv mVKMRJYBhzk2mEvKEDNtnw47SQPBAKo9BtJvl7gOEofiPK/f2K/cy8yMUrow1E9D 02PNT0XX8ysoe86Dqip35+sphczkQN8gilXyUQujNe8edEdkW7PBhbJn92zBQag2 JxA194bquxRyhW78T3tKAEN6/tTPgZYJNy202KC619zzLlK3TslwjjfOQILdRb2i NNkaSQBdYDK70BiFs5Ri7ZbfJHenY6mgGv7yG0vjGF6zjoXtVNsKGrYt9KBL6E1D 392ayxOlWCvBoG9n9sAUzHzcQxmU1lP6OHcO9xWrL6ySD7Fzv4rCtM0uo7gXOzNB aDl5vK7q+ysEjOXZJWT0ikj5ndATCv0Ry8wnt1uL/uktOuaE0egwsouU0jgCDgx1 8Ib8KX/B6bhCy13WkYLPTb6Yg0k+ph0BUyONEcN5cxJIcqfcEt/4Un4MYM+CjGck KLYrrZDclZr8p/paWdNqx1dI9NIBn+F3u78OcWY3NIGfZPh1jmSdG3LRt9DZo5bZ ua2UlAiAWnEDHNLQyMH2zcji31DMqIUwmQ5bX9isBcxt8LORX+IKxoLdUICoY8JS Ii5kNWJAjf4= =Xsdv -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 3:17 ` David Masover @ 2005-06-24 3:34 ` Horst von Brand 2005-06-25 3:38 ` David Masover 2005-06-27 9:21 ` mjt 2005-06-24 11:34 ` Alan Cox ` (2 subsequent siblings) 3 siblings, 2 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-24 3:34 UTC (permalink / raw) To: David Masover Cc: Alan Cox, Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List David Masover <ninja@slaphack.com> wrote: > Alan Cox wrote: > > On Iau, 2005-06-23 at 23:04, David Masover wrote: [...] > >>>What for? It works just fine as it stands, AFAICS. > >>So does DOS. Do you use DOS? I don't even use DOS to run DOS programs. > > False argument. So does the pen, so do hinges on doors. Do you still > > have hinges on your doors - probably. > Indeed. Because there's nothing better -- not because I "like it the > way it is". There being nothing better means nobody has ever been able to come up with a better way. Most of the time at least. > >>"Ain't broke" is the battle cry of stagnation. > > Its also the battle cry of everyone over the age of 20 who also has a > > real job to do 8) > You caught me. I'm not over 20. But I have a real job, with a company > that understands the difference between "ain't broke" and "works well". "Doesn't work well" /is/ the definition of "broken". Modulo how irritating the "not working well" is... > >>But, there are some things Reiser does better and faster than ext3, even > >>if you don't count file-as-directory and other toys. There's nothing > >>ext3 does better than Reiser, unless you count the compatibility with > >>random bootloaders and low-level tools. > > Certainly compared with reiser3 you've missed a few out including > > resilience to disk errors (nearly nil on reiser3), and SMP scaling. > Actually, I was talking about reiser4. And Hans corrected me on that... > > Although resilience to disk errors isn't a design decision. That's what > SMART and new hard drives are for. And if you're stubborn enough to > keep the same FS around, there's dm-bbr. > I think Hans (or someone) decided that when hardware stops working, it's > not the job of the FS to compensate, it's the job of lower layers, or > better, the job of the admin to replace the disk and restore from > backups. Handling other people's data this way is just reckless irresponsibility. Sure, you can get high performance if you just forego some of your basic responsibilities. > >>You know how many I've had thrashed on Reiser4? Two. The first one was > >>with a VERY early alpha/beta, and the second one was when I dropped a > >>laptop and the disk failed. > > Entirely or bad blocks ? The latter should have a minimal cost on a well > > designed fs. > I was able to recover from bad blocks, though of course no Reiser that I > know of has had bad block relocation built in... But I got all my files > off of it, fortunately. And if the bad blocks had been on files? [...] > > In which case the features belong in the VFS as all those with > > experience and kernel contributions have been arguing. > No one's arguing that. What we're arguing is that there does seem to be > a bit of prejudice when fuck-me-with-a-chainsaw and YOU ARE A FOOL FOR > ENABLING THIS all got in, and with at least one of those, there doesn't > seem to be any intention of changing it -- all of that, and even with an > expressed intention to fix the aesthetical problems with Reiser4 later, > we can't get the working version in now. > > More infuriatingly, I, at least, have a distinct feeling that once all > the issues are fixed, an entirely different crowd of benevolent > dictators will come around and say that we can't get in because we > change the VFS. At least some people on this list have said things to > that effect. The dictators here have changed over time, sure. Their general attitude has been the same all the way through, since the list started. And you might not like it that way, but I am sure Linux is of the current high quality exactly because any core code that gets into the kernel (and a new filesystem that wants to be central is clearly of this kind) is checked by multiple people with (sometimes widely) diverging backgrounds, interests, and views. Linus himself has stated more than once that his principal job isn't to integrate code into the kernel, but leaving stuff out. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 3:34 ` Horst von Brand @ 2005-06-25 3:38 ` David Masover 2005-06-27 9:21 ` mjt 1 sibling, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-25 3:38 UTC (permalink / raw) To: Horst von Brand Cc: Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Horst von Brand wrote: > David Masover <ninja@slaphack.com> wrote: > >>Alan Cox wrote: >> >>>On Iau, 2005-06-23 at 23:04, David Masover wrote: > > > [...] [...] >>>>"Ain't broke" is the battle cry of stagnation. > > >>>Its also the battle cry of everyone over the age of 20 who also has a >>>real job to do 8) > > >>You caught me. I'm not over 20. But I have a real job, with a company >>that understands the difference between "ain't broke" and "works well". > > > "Doesn't work well" /is/ the definition of "broken". Modulo how irritating > the "not working well" is... Well, I agree with you there, but to most people, "broken" = "Doesn't work at all." Stagnation happens when people cling to the old way of doing something until they are actually forced to change. One example of this is energy. Why do we still burn coal and oil? Even lithium ion batteries still give a 200-300 mile range on one electric car I was looking at. There is enough wind in Iowa alone to power half the country, and if we were to put up a bunch of solar panels in Nevada, we'd be done. With the money we would save on not buying oil, we could finish the fuel-cell technologies being worked on, and eventually, the US would get its energy entirely from clean sources. For that matter, we'd easily save enough money, not to mention the public health benefits (no more smog), to pay for the transition (building hydrogen pumping stations and such, buying new cars...) But people don't like to change. So we burn oil and make smog and choke on our own stupidity. That debate really belongs off-list, by the way. All flames on this topic, hit "reply", not "reply-to-all". >>>>But, there are some things Reiser does better and faster than ext3, even >>>>if you don't count file-as-directory and other toys. There's nothing >>>>ext3 does better than Reiser, unless you count the compatibility with >>>>random bootloaders and low-level tools. > > >>>Certainly compared with reiser3 you've missed a few out including >>>resilience to disk errors (nearly nil on reiser3), and SMP scaling. > > >>Actually, I was talking about reiser4. And Hans corrected me on that... >> >>Although resilience to disk errors isn't a design decision. That's what >>SMART and new hard drives are for. And if you're stubborn enough to >>keep the same FS around, there's dm-bbr. > > >>I think Hans (or someone) decided that when hardware stops working, it's >>not the job of the FS to compensate, it's the job of lower layers, or >>better, the job of the admin to replace the disk and restore from >>backups. > > > Handling other people's data this way is just reckless irresponsibility. > Sure, you can get high performance if you just forego some of your basic > responsibilities. Who's responsibility? It's the responsibility of the user to back up their files. Sure, this time it was just a little dust, but maybe next time it'll be coffee spilled on the box. Or maybe a power surge. Maybe someone just gets mad enough and kicks the computer very hard. Reiser4 has been absolutely immune to power loss for me, and quite stable in the face of some corruption, but as soon as I see bad blocks, I throw out the disk, I don't try to make the FS deal with it. And that's all that's missing right now -- a way to use a disk which has some bad blocks -- and that's implemented by dm-bbr if you're that stubborn. Ultimately, disks are cheap. Cheaper than data loss. >>>>You know how many I've had thrashed on Reiser4? Two. The first one was >>>>with a VERY early alpha/beta, and the second one was when I dropped a >>>>laptop and the disk failed. > > >>>Entirely or bad blocks ? The latter should have a minimal cost on a well >>>designed fs. > > >>I was able to recover from bad blocks, though of course no Reiser that I >>know of has had bad block relocation built in... But I got all my files >>off of it, fortunately. > > > And if the bad blocks had been on files? Then no FS would save me. > [...] > > >>>In which case the features belong in the VFS as all those with >>>experience and kernel contributions have been arguing. [...] >>More infuriatingly, I, at least, have a distinct feeling that once all >>the issues are fixed, an entirely different crowd of benevolent >>dictators will come around and say that we can't get in because we >>change the VFS. At least some people on this list have said things to >>that effect. > > > The dictators here have changed over time, sure. Their general attitude has > been the same all the way through, since the list started. And you might > not like it that way, but I am sure Linux is of the current high quality > exactly because any core code that gets into the kernel (and a new > filesystem that wants to be central is clearly of this kind) is checked by > multiple people with (sometimes widely) diverging backgrounds, interests, > and views. Linus himself has stated more than once that his principal job > isn't to integrate code into the kernel, but leaving stuff out. I mean, half of the benevolent dictators at any given time don't seem to like reiser4 for some reason or other. I get the feeling that if we satisfy that half, the way in which we do it will irritate the other half. But I don't know, and I'm going to leave the more specific debates soon. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrzRmHgHNmZLgCUhAQIOyw//bL4sUYQyGSwjKnKZxTznzNnchn0rpNsx PnxrQEvETtkqFlidToJPKDEP3H32tE5W3SeONhIprbQVOK9BagI5Ojm+AhM8pXXa 8K0NJmedlnjMJp5yAxbB+dIwPQ8IN8cG7cdeZCi4ij76yKiDFEqZHXrpm/bvXcbB jzzOF86XPVfexGOUQkwDpR0gmFeY+zIp3+7TGKwG17xHYOej7KpyQctYB7e1me9t MKFll/2USUAglvtdHQ6on2T9slmBTRMny4OFJXKnBTo6rILBTvo6sxC3hG2lv1lN RYXHOd/AV/JQHzhZ1f1vRAn9VxdsUkMgvcOq3k3+ofUP86b1VqxsqGP4Mv3S8uka hghTqBh7ivOr5llCwUHudDIXjRzNw1UBwi6UBC+YB0QQzR3vzc3Q0rvkD1v6hoV+ Li+sX0GhgjaIDzTE+e1rJOAlTAIxeQ9dRMDVBOR5b9HVtkxZ20SSdagDSeHz9JPH Lv0JmqkvYgVf703a3wt56FpTIMd7XvY6dUSR9u3k1xUGQg2jeJA0gZ3tySSogKbT xick+z+m1THTIyA5tXA1JRS1+GNAc4yCL4mHPcUKVWPsKfxaIDB4/+6eCll7Ctxz XzP6s8nv4HkPAsWe1iLvJmuUXdiradC87MJiDGCKM6mbNZ0hFGnKa8zG8nI4C/Ua qHz+cXVkgi8= =QFCz -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 3:34 ` Horst von Brand @ 2005-06-27 9:21 ` mjt 2005-06-27 9:21 ` mjt 1 sibling, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-27 9:21 UTC (permalink / raw) To: Horst von Brand Cc: David Masover, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 717 bytes --] On Thu, Jun 23, 2005 at 11:34:50PM -0400, Horst von Brand wrote: >David Masover <ninja@slaphack.com> wrote: >> I think Hans (or someone) decided that when hardware stops working, it's >> not the job of the FS to compensate, it's the job of lower layers, or >> better, the job of the admin to replace the disk and restore from >> backups. >Handling other people's data this way is just reckless irresponsibility. >Sure, you can get high performance if you just forego some of your basic >responsibilities. Your honest-to-bog opinion is that the FS vendor is responsible for the admin not taking backups or the hardware vendor shipping crap? *still trying to understand how that can be* -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-27 9:21 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-27 9:21 UTC (permalink / raw) To: Horst von Brand Cc: David Masover, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 717 bytes --] On Thu, Jun 23, 2005 at 11:34:50PM -0400, Horst von Brand wrote: >David Masover <ninja@slaphack.com> wrote: >> I think Hans (or someone) decided that when hardware stops working, it's >> not the job of the FS to compensate, it's the job of lower layers, or >> better, the job of the admin to replace the disk and restore from >> backups. >Handling other people's data this way is just reckless irresponsibility. >Sure, you can get high performance if you just forego some of your basic >responsibilities. Your honest-to-bog opinion is that the FS vendor is responsible for the admin not taking backups or the hardware vendor shipping crap? *still trying to understand how that can be* -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 9:21 ` mjt (?) @ 2005-06-27 12:42 ` Theodore Ts'o 2005-06-27 15:19 ` David Masover 2005-06-27 19:46 ` Hans Reiser -1 siblings, 2 replies; 620+ messages in thread From: Theodore Ts'o @ 2005-06-27 12:42 UTC (permalink / raw) To: Markus T?rnqvist Cc: Horst von Brand, David Masover, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Mon, Jun 27, 2005 at 12:21:38PM +0300, Markus T?rnqvist wrote: > On Thu, Jun 23, 2005 at 11:34:50PM -0400, Horst von Brand wrote: > >David Masover <ninja@slaphack.com> wrote: > > >> I think Hans (or someone) decided that when hardware stops working, it's > >> not the job of the FS to compensate, it's the job of lower layers, or > >> better, the job of the admin to replace the disk and restore from > >> backups. > >Handling other people's data this way is just reckless irresponsibility. > >Sure, you can get high performance if you just forego some of your basic > >responsibilities. > > Your honest-to-bog opinion is that the FS vendor is responsible for > the admin not taking backups or the hardware vendor shipping crap? > > *still trying to understand how that can be* Most Linux users are using PC-class hardware. And Ted's First Law of PC-Class Hardware is: "Most of it is crap". And then there's Ted's Second Law, "Too many system administrators don't do backups". This is because most system admins are users who've never been trained to be a sysadmin, or who haven't (yet) had weeks or months of works disappear after a hardware failure. So it's a matter of matching the filesystem to the needs of the user. If you have a filesystem which is blazingly fast, but which at the slightest sign of trouble, trashes your data, versus one which is fast but perhaps not-so-fast as the other filesystem, but which is much more reliable, which would you choose? XFS has similar issues where it assumes that hardware has powerfail interrupts, and that the OS can use said powerfail interrupt to stop DMA's in its tracks on an power failure, so that you don't have garbage written to key filesystem data structures when the memory starts suffering from the dropping voltage on the power bus faster than the DMA engine or the disk drives. So XFS is a great filesystem --- but you'd better be running it on a UPS, or on a system which has power fail interrupts and an OS that knows what to do. Ext3, because it does physical block journalling, does not suffer from this problem. (Yes, Resierfs uses logical journalling as well, so it suffers from the same problem.) So perhaps it's not the job of the FS vendor to be responsible for crap hardware or lazy sysadmins that don't do backups. But a system administrator who knows that he doesn't do backups frequently enough, or is running on cheap, crap hardware, would be wise to consider carefully which filesystem he/she wants to use given the systems configuration and his backup habits. Me, I'll go for the robust filesystem, just on general principles. As a friend from the large-scale enterprise storage world once put it, "Performance is Job 2. Robustness is Job #1." (Of course, if you want to put your fragile filesystem on a multi-million dollar enterprise storage system such as an IBM Shark or an EMC Symmetrix box, I'm sure IBM or EMC will be happy to sell you one. :-) - Ted ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 12:42 ` Theodore Ts'o @ 2005-06-27 15:19 ` David Masover 2005-06-27 16:28 ` Theodore Ts'o 2005-06-27 19:46 ` Hans Reiser 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-27 15:19 UTC (permalink / raw) To: Theodore Ts'o Cc: Markus T?rnqvist, Horst von Brand, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Theodore Ts'o wrote: > On Mon, Jun 27, 2005 at 12:21:38PM +0300, Markus T?rnqvist wrote: > >>On Thu, Jun 23, 2005 at 11:34:50PM -0400, Horst von Brand wrote: >> >>>David Masover <ninja@slaphack.com> wrote: >> >>>>I think Hans (or someone) decided that when hardware stops working, it's >>>>not the job of the FS to compensate, it's the job of lower layers, or >>>>better, the job of the admin to replace the disk and restore from >>>>backups. >>> >>>Handling other people's data this way is just reckless irresponsibility. >>>Sure, you can get high performance if you just forego some of your basic >>>responsibilities. >> >>Your honest-to-bog opinion is that the FS vendor is responsible for >>the admin not taking backups or the hardware vendor shipping crap? >> >>*still trying to understand how that can be* > > > Most Linux users are using PC-class hardware. And Ted's First Law of > PC-Class Hardware is: "Most of it is crap". And then there's Ted's > Second Law, "Too many system administrators don't do backups". Not our problem anymore. This is like saying that we should run all filesystems in synchronous mode because some users will grab stuff out of the computer without unmounting it -- even more a problem now that we have SATA, which supports hot-swapping. Too many system administrators don't do backups? Some day the building will burn down and they will wish they had. > So it's a matter of matching the filesystem to the needs of the user. My needs are a filesystem which doesn't assume I'm an idiot who can't make backups. > If you have a filesystem which is blazingly fast, but which at the > slightest sign of trouble, trashes your data, versus one which is fast > but perhaps not-so-fast as the other filesystem, but which is much > more reliable, which would you choose? Hypothetically? I choose the faster one. Failures happen only rarely, and when they happen, there's no telling how small or large they will be, therefore I keep regular backups, and as soon as I see my hardware starts to fail, I grab what I can off of it, merge that with the latest backup, and buy new hardware. > XFS has similar issues where it assumes that hardware has powerfail > interrupts, and that the OS can use said powerfail interrupt to stop > DMA's in its tracks on an power failure, so that you don't have > garbage written to key filesystem data structures when the memory > starts suffering from the dropping voltage on the power bus faster > than the DMA engine or the disk drives. So XFS is a great filesystem > --- but you'd better be running it on a UPS, or on a system which has > power fail interrupts and an OS that knows what to do. XFS, Reiser3, and Reiser4 all pass the pull-the-plug test. Maybe I just haven't pushed them hard enough? > Ext3, because > it does physical block journalling, does not suffer from this problem. > (Yes, Resierfs uses logical journalling as well, so it suffers from > the same problem.) I don't actually understand the difference here. Are we talking about metadata journalling vs. data journalling? > So perhaps it's not the job of the FS vendor to be responsible for > crap hardware or lazy sysadmins that don't do backups. Good! Glad we agree! > But a system > administrator who knows that he doesn't do backups frequently enough, > or is running on cheap, crap hardware, would be wise to consider > carefully which filesystem he/she wants to use given the systems > configuration and his backup habits. Given a choice between changing filesystems or getting a Streamload account, I choose Streamload. (streamload.com) Given a choice between reformatting or editing cron, I'd edit cron. > Me, I'll go for the robust filesystem, just on general principles. As > a friend from the large-scale enterprise storage world once put it, > "Performance is Job 2. Robustness is Job #1." (Of course, if you > want to put your fragile filesystem on a multi-million dollar > enterprise storage system such as an IBM Shark or an EMC Symmetrix > box, I'm sure IBM or EMC will be happy to sell you one. :-) Job 1 is done by either that system or by good backups. Don't overkill Job 1 at the expense of Job 2. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsAY5XgHNmZLgCUhAQJq6g/9G2GEK0R1kujoqn3tjsTIbgC23iu8byYV d6NHSrmmDV8035oo5iCgjSXbaTlIEyOQQqSB78fBC27eHbGctEZJIpSQfHrWlJu6 UJvvaEyl+sR/Mzq8sButKeSUv/T8RMCBKyhdfSgm28lDM/kt9OGZcrf1P1ChUSeK ZnaToh4SAReRhGzq247o+qa2rW5IuIlpKeYGMZhiOB/tXGC3IDMtUOE9IBkn8KVd TtDN4f74PuDJ9VX8K/tZ9DDtKebOJ1wKvIQ/BwyNXt5g36vwj2UwK6GkT2ZJ3bd6 2XLL24LpIBcVU16bk5WPkADRKe0W7S6AseUj7ggjAK0OL8z889vQ3NWJ/rmT/Wx7 OQi58QBwL89OZGSM2qZFJ09CLEH42GXD5jjTrKt4URudYzoIa45VZxD9YYhzeu5e Gxxj4r2BM8bf/AHczCU/tEQwb6hfqxrq+SImWd1uIsozjeiwInHBC56x2qUDjFB/ kiyO1LVmVzrq+6frjPAg6n/i0zuPfF24SeeHX02UOx0jlNLkq2D4Qu1wDB5XlVTK 4xrN83Uy+dHSuDBvnf2tWTqmBju2YG6zmOl1xWKzi3/AVV6mfg1ur0gX0Q6GiZVa 5Nqp618Cs00ZtSk759eAPQRu7Ico0vTHhA0+rTUhApLYMboGaRN5W8OtQu2utHz8 ALX7pD3DrfE= =wdYo -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 15:19 ` David Masover @ 2005-06-27 16:28 ` Theodore Ts'o 2005-06-27 21:12 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Theodore Ts'o @ 2005-06-27 16:28 UTC (permalink / raw) To: David Masover Cc: Markus T?rnqvist, Horst von Brand, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Mon, Jun 27, 2005 at 10:19:01AM -0500, David Masover wrote: > > XFS has similar issues where it assumes that hardware has powerfail > > interrupts, and that the OS can use said powerfail interrupt to stop > > DMA's in its tracks on an power failure, so that you don't have > > garbage written to key filesystem data structures when the memory > > starts suffering from the dropping voltage on the power bus faster > > than the DMA engine or the disk drives. So XFS is a great filesystem > > --- but you'd better be running it on a UPS, or on a system which has > > power fail interrupts and an OS that knows what to do. > > XFS, Reiser3, and Reiser4 all pass the pull-the-plug test. Maybe I just > haven't pushed them hard enough? Try doing huge amounts of metadata updates while doing the pull-the-plug test. The problem comes when the disk drive is writing into the inode table or other filesystem metadata at the precise moment when power goes down. If the disk is quiescent, you won't see the issue. XFS is known to have this problem --- the problem was first told to me by an SGI engineer, and they solved this problem in hardware, by adding a power fail interrupt and larger capacitors to the power supply, and then modifying Irix to frantically run around shutting down all outstanding DMA transfer during the grace period provided by the enlarged capacitors. Unfortunately, PC-class hardware don't have power fail interrupts, so.... I have seen this problem reported on ext2 filesystems, so I know it will happen on at least some, and probably most, PC-class hardware. Fortunately switching to ext3 solves the problem, since metadata updates are first written to the journal first, and the recovery replays the full block, which papers over the problem. Unfortunately, a filesystem which uses logical journalling doesn't have a complete copy of the metadata block in the log (only the logical changes are recorded in the journal). This is more compact, and will result in better numbers in artificial benchmarks that emphasize metadata updates (which is not the case in most real-world workloads in my experience). But it the drawback is that filesystems that journal logical updates instead of physical blocks are vulnerable to this particular lossage mode. > Given a choice between changing filesystems or getting a Streamload > account, I choose Streamload. (streamload.com) *If* you can afford the upload bandwidth to backup over the network, and *if* you don't mind these gems in their legal T's and C's: Streamload cannot warrant and does not guarantee, and You should not expect, that all of Your private communications and other personal information will never be disclosed in ways not otherwise described in this Privacy Policy. As-Is and As-Available. Neither Streamload nor any User, or their respective agents, warrants that the service, Private Content or Public Content, or Your access thereto, will be uninterrupted or error free, and Streamload's services and both Public Content and Private Content are provided on an "as is, as available" basis. Streamload has the right to make changes to its services without notice to You. Neither Streamload nor any User warrants that Public Content or Private Content will be free of viruses or similar contamination or destructive features. You expressly agree to assume any and all risk as to the use, quality, performance, accuracy or completeness of any Private Content, Public Content, or Streamload's services. And of course, while Streamload is pretty cheap for a minimal account, if you have say, a several hundred gigabytes of data which is lost when your filesystem implodes, they have you over a barrel and charge $$$ in order for you to recover it. But if you think it's this works for you, please, go right ahead. Just don't come whining to us afterwards if you get screwed. - Ted ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 16:28 ` Theodore Ts'o @ 2005-06-27 21:12 ` David Masover 2005-06-30 21:49 ` Theodore Ts'o 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-27 21:12 UTC (permalink / raw) To: Theodore Ts'o Cc: Markus T?rnqvist, Horst von Brand, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Theodore Ts'o wrote: > On Mon, Jun 27, 2005 at 10:19:01AM -0500, David Masover wrote: [...] >>Given a choice between changing filesystems or getting a Streamload >>account, I choose Streamload. (streamload.com) > > > *If* you can afford the upload bandwidth to backup over the network, > and *if* you don't mind these gems in their legal T's and C's: > > Streamload cannot warrant and does not guarantee, and You > should not expect, that all of Your private communications and > other personal information will never be disclosed in ways not > otherwise described in this Privacy Policy. gpg. Was in my upload script to begin with. I keep my key written many times on a single hidden CD. So long as the isofs can be read, at least one of the copies should be usable. > As-Is and As-Available. Neither Streamload nor any User, or > their respective agents, warrants that the service, Private > Content or Public Content, or Your access thereto, will be > uninterrupted or error free, and Streamload's services and > both Public Content and Private Content are provided on an "as > is, as available" basis. Streamload has the right to make > changes to its services without notice to You. Neither They don't have any billing information on me. If they charge me for something, I'll cancel my account. > Streamload nor any User warrants that Public Content or > Private Content will be free of viruses or similar > contamination or destructive features. You expressly agree to > assume any and all risk as to the use, quality, performance, > accuracy or completeness of any Private Content, Public > Content, or Streamload's services. They are giving you unlimited storage for free. Even with all of that, it's a damn good deal. > And of course, while Streamload is pretty cheap for a minimal account, > if you have say, a several hundred gigabytes of data which is lost > when your filesystem implodes, they have you over a barrel and charge > $$$ in order for you to recover it. But if you think it's this works > for you, please, go right ahead. Just don't come whining to us > afterwards if you get screwed. I don't have several hundred gigabytes of dynamic data, and neither does the average desktop user. The several hundred gigabytes of static data I have would be backed up on DVDs anyway. Now, if we're talking about a sysadmin, then of course you need a better backup solution -- no matter what FS you use. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsBrxngHNmZLgCUhAQLihw//Qz+m971rPW90Fw9We2pqK49pstfsdk4Z n7kimE7kptdcQX0ndnBa1UJ/1bTdIsaGrL+OFTkqLVKtNfitt8DuhdrbXlgm9gvJ k8mwiopXIcfPA7BZ7vxzudttX/35tRfV2ubosagqWZjnpRgBLE3mEdDRCAJb1Z7+ YkynyekwzLPNo4qxWjhazDLqkBvGz4gntQrAwMoOYFlgpQZHlcDAABdRgd1rv+EA paf/ZD90sGm5ZSodwURA8rUKIhRGqv+a5icJQ/PFWl8Cyjtg61w6toWjeK52tE+t 1AezDvjYy7JBItV3BlScgcCze6lKlAEHXDgjUZzKBllh4+h8b46IR6qjvpEk1YeX jOeJpUwIv/YPtRgfGbUBgVyd5wBWqD4+W7a2qn+GIr6sQT176X+fNFzs6RKy+QlZ Ty6yXGejFoydOhbg8zuhZvbEfJKuZCAv+1JkUijFZI6uVZj4Yychc04+qRybg+SQ yY+52KbHyeeH3yQSQrGNa3ngEh84LJjoUpvYXGhQg+ex7kDuQAYlqdUAd7NIQIFj LFvGzkVT/8SLAOi9D7lXB0g52+W4Atl7a3nYpFhIkm2AthNjUkSFhZxyu6wmhFrj 5SPReftDmp4MVQeC4FVZfB72muY4zcO/WYSVmpGn2P89lHEgD4uw8gqQ0XAi5xe6 Gwdb2XcczCo= =+R+m -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 21:12 ` David Masover @ 2005-06-30 21:49 ` Theodore Ts'o 2005-06-30 22:34 ` Dmitry Torokhov 2005-07-01 8:06 ` Hans Reiser 0 siblings, 2 replies; 620+ messages in thread From: Theodore Ts'o @ 2005-06-30 21:49 UTC (permalink / raw) To: David Masover Cc: Markus T?rnqvist, Horst von Brand, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Mon, Jun 27, 2005 at 04:12:38PM -0500, David Masover wrote: > > Streamload cannot warrant and does not guarantee, and You > > should not expect, that all of Your private communications and > > other personal information will never be disclosed in ways not > > otherwise described in this Privacy Policy. > > gpg. Was in my upload script to begin with. I keep my key written many > times on a single hidden CD. So long as the isofs can be read, at least > one of the copies should be usable. > > They don't have any billing information on me. If they charge me for > something, I'll cancel my account. If you are using the free service, and are encrypting the data, you are explicitly violating their terms of service, and they can delete your data at any time, once they notice. - Ted ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 21:49 ` Theodore Ts'o @ 2005-06-30 22:34 ` Dmitry Torokhov 2005-07-01 7:03 ` backup (was Re: reiser4 plugins) David Masover 2005-07-01 8:08 ` reiser4 plugins Hans Reiser 2005-07-01 8:06 ` Hans Reiser 1 sibling, 2 replies; 620+ messages in thread From: Dmitry Torokhov @ 2005-06-30 22:34 UTC (permalink / raw) To: Theodore Ts'o, David Masover, Markus T?rnqvist, Horst von Brand, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On 6/30/05, Theodore Ts'o <tytso@mit.edu> wrote: > On Mon, Jun 27, 2005 at 04:12:38PM -0500, David Masover wrote: > > > Streamload cannot warrant and does not guarantee, and You > > > should not expect, that all of Your private communications and > > > other personal information will never be disclosed in ways not > > > otherwise described in this Privacy Policy. > > > > gpg. Was in my upload script to begin with. I keep my key written many > > times on a single hidden CD. So long as the isofs can be read, at least > > one of the copies should be usable. > > > > They don't have any billing information on me. If they charge me for > > something, I'll cancel my account. > > If you are using the free service, and are encrypting the data, you > are explicitly violating their terms of service, and they can delete > your data at any time, once they notice. > Does not look like it: 3c. No encryption and/or steganography for the purpose of circumventing Streamload's rules. ... For example, if you'd like to encrypt something for an extra sense of security and privacy, please feel free to do so. However, when these tools are used solely for the purpose of circumventing Streamload's rules, as determined by the sole discretion of Streamload, the files will be deleted. -- Dmitry ^ permalink raw reply [flat|nested] 620+ messages in thread
* backup (was Re: reiser4 plugins) 2005-06-30 22:34 ` Dmitry Torokhov @ 2005-07-01 7:03 ` David Masover 2005-07-01 14:19 ` Theodore Ts'o 2005-07-01 8:08 ` reiser4 plugins Hans Reiser 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-01 7:03 UTC (permalink / raw) To: dtor_core Cc: Theodore Ts'o, Markus T?rnqvist, Horst von Brand, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Dmitry Torokhov wrote: > On 6/30/05, Theodore Ts'o <tytso@mit.edu> wrote: > >>On Mon, Jun 27, 2005 at 04:12:38PM -0500, David Masover wrote: >> >>>> Streamload cannot warrant and does not guarantee, and You >>>> should not expect, that all of Your private communications and >>>> other personal information will never be disclosed in ways not >>>> otherwise described in this Privacy Policy. >>> >>>gpg. Was in my upload script to begin with. I keep my key written many >>>times on a single hidden CD. So long as the isofs can be read, at least >>>one of the copies should be usable. >>> >>>They don't have any billing information on me. If they charge me for >>>something, I'll cancel my account. >> >>If you are using the free service, and are encrypting the data, you >>are explicitly violating their terms of service, and they can delete >>your data at any time, once they notice. >> > > > Does not look like it: > > 3c. No encryption and/or steganography for the purpose of > circumventing Streamload's rules. > ... For example, if you'd like to encrypt something for an extra sense > of security and privacy, please feel free to do so. However, when > these tools are used solely for the purpose of circumventing > Streamload's rules, as determined by the sole discretion of > Streamload, the files will be deleted. Trasnlation: They really, honestly, have no way at all of knowing if my GPG-encrypted 450 meg tarball violates their terms. They have legally given themselves the right to delete it just because they think it might violate their terms, but it wouldn't make sense for them to delete a file called "backup.tar.bz2.gpg" when there are no doubt a lot of other encrypted files on their servers. It wouldn't make sense because they would either have to delete such files completely at random, or they'd have to delete all of them at once. Further translation: This was probably put here to keep the lawyers happy. They don't want to be held liable for illegal stuff on their servers, for example, so they have at least spelled out, in legalese, that illegal stuff (and probably a few other categories) is not allowed on their servers, and that encyrpting it doesn't magically make it legal. This kind of stuff makes lawyers happy, even if it *practically* does nothing at all. Regardless, I feel reasonably safe with a backup there. If I ever start to feel unsafe, I can always back up the most critical stuff to gmail, and create a few permanent copies on DVDs. But I feel considerably safer with Streamload than with Gmail, because using Gmail for my own custom backup app directly violates their terms of service. I think we should officially kill this particular thread. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: backup (was Re: reiser4 plugins) 2005-07-01 7:03 ` backup (was Re: reiser4 plugins) David Masover @ 2005-07-01 14:19 ` Theodore Ts'o 2005-07-01 19:49 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Theodore Ts'o @ 2005-07-01 14:19 UTC (permalink / raw) To: David Masover Cc: dtor_core, Markus T?rnqvist, Horst von Brand, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Rule #3 from http://www.streamload.com/About/Legal_eng.asp?page=id73# is pretty clear about what applies if you have a trial account (which seems to be what you have since you say you'll cancel your account if they charge you anything): 3. Do not circumvent Freeloader download restrictions. Any attempt to circumvent Freeloader trial account restrictions will result in a permanent banishment from the Streamload community. This includes (3a) files with an invalid or disguised file format; (3b) encryption; (3c) steganography and/or (3d) creating multiple freeloader accounts. You can interpret that whatever way you like, but if you're that cavalier with your data, hey, I'm not sure _I'd_ trust your judgement about which filesystem to trust. :-) - Ted ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: backup (was Re: reiser4 plugins) 2005-07-01 14:19 ` Theodore Ts'o @ 2005-07-01 19:49 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-07-01 19:49 UTC (permalink / raw) To: Theodore Ts'o Cc: dtor_core, Markus T?rnqvist, Horst von Brand, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Theodore Ts'o wrote: > Rule #3 from http://www.streamload.com/About/Legal_eng.asp?page=id73# > is pretty clear about what applies if you have a trial account (which > seems to be what you have since you say you'll cancel your account if > they charge you anything): > > 3. Do not circumvent Freeloader download restrictions. > Any attempt to circumvent Freeloader trial account restrictions will > result in a permanent banishment from the Streamload community. This > includes (3a) files with an invalid or disguised file format; (3b) > encryption; (3c) steganography and/or (3d) creating multiple > freeloader accounts. > > You can interpret that whatever way you like, but if you're that > cavalier with your data, hey, I'm not sure _I'd_ trust your judgement > about which filesystem to trust. :-) Streamload is one possibility. There are others -- gmail and such. This is just what can be had *for*free*. For that matter, the uploading process doesn't require a user to agree to those terms at all. There's a module in CPAN that allows an upload, and never asks me to agree to any terms, nor does it ask me for an account password. I don't have multiple accounts, I don't *use* the encryption to attempt to circumvent the download restrictions, especially considering that said download restrictions don't even come into play until I need a restore. By the time I do need a restore, I'm willing to pay money, as 100 megs a month is certainly not going to be enough. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 22:34 ` Dmitry Torokhov 2005-07-01 7:03 ` backup (was Re: reiser4 plugins) David Masover @ 2005-07-01 8:08 ` Hans Reiser 1 sibling, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-07-01 8:08 UTC (permalink / raw) To: dtor_core Cc: Theodore Ts'o, David Masover, Markus T?rnqvist, Horst von Brand, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Dmitry Torokhov wrote: > >>If you are using the free service, and are encrypting the data, you >>are explicitly violating their terms of service, and they can delete >>your data at any time, once they notice. >> >> >> > >Does not look like it: > >3c. No encryption and/or steganography for the purpose of >circumventing Streamload's rules. >... For example, if you'd like to encrypt something for an extra sense >of security and privacy, please feel free to do so. However, when >these tools are used solely for the purpose of circumventing >Streamload's rules, as determined by the sole discretion of >Streamload, the files will be deleted. > > > Oops, sorry, got that wrong, ok, it is more that they provide this service and the terms are at their convenience not anyone else's. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 21:49 ` Theodore Ts'o 2005-06-30 22:34 ` Dmitry Torokhov @ 2005-07-01 8:06 ` Hans Reiser 1 sibling, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-07-01 8:06 UTC (permalink / raw) To: Theodore Ts'o Cc: David Masover, Markus T?rnqvist, Horst von Brand, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Wow. Good that you noticed it. It sounds like they have the business model of an intelligence agency..... Hans Theodore Ts'o wrote: >If you are using the free service, and are encrypting the data, you >are explicitly violating their terms of service, and they can delete >your data at any time, once they notice. > > - Ted > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 12:42 ` Theodore Ts'o 2005-06-27 15:19 ` David Masover @ 2005-06-27 19:46 ` Hans Reiser 2005-06-27 20:18 ` Steve Lord 2005-06-27 21:26 ` Theodore Ts'o 1 sibling, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-27 19:46 UTC (permalink / raw) To: Theodore Ts'o Cc: Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List, Steve Lord Steve, there is a remark about XFS below which you are going to be more expert on. Theodore Ts'o wrote: >Most Linux users are using PC-class hardware. And Ted's First Law of >PC-Class Hardware is: "Most of it is crap". And then there's Ted's >Second Law, "Too many system administrators don't do backups". This >is because most system admins are users who've never been trained to >be a sysadmin, or who haven't (yet) had weeks or months of works >disappear after a hardware failure. > > The above I agree with. Your words below though are irresponsible. I get users who tell me that ext* is crap and fsck.ext2 corrupted their filesystem and thats why they use ReiserFS. A difference between us is that I tell them that with all the major linux filesystems (I include XFS and JFS in this) it is by this time far more likely to be hardware that caused corruption than the filesystem software, whereas I guess you tell them something else. Describing us as trashing data at the slightest sign of trouble is irresponsible. We do block journaling not logical journaling, but there is absolutely nothing inherently wrong with logical journaling, and no reason why it could not be just as robust if that is what XFS uses, so I suspect the rest of your remarks about XFS are unlikely to be sound. I would say much more, but I am trying to avoid flames this week. Ted, please try to consider that maybe your competitors also do a fairly good job at the mundane but important aspects of filesystem work, and lets all avoid FUDing each others work. Both ext* and XFS are excellent filesystems, and Linux is lucky to have 3 of the 4 best filesystems available for it. It will be interesting to see though whether Dominic Giampaolo beats all three of us in the next five years. He probably won't have the performance, but he sure might have the semantic features, and he is very bright. If MS ever makes good use of the talent they have been hiring, we'll have a hard time there also. We may get lucky on that though.....;-) >So it's a matter of matching the filesystem to the needs of the user. >If you have a filesystem which is blazingly fast, but which at the >slightest sign of trouble, trashes your data, versus one which is fast >but perhaps not-so-fast as the other filesystem, but which is much >more reliable, which would you choose? > >XFS has similar issues where it assumes that hardware has powerfail >interrupts, and that the OS can use said powerfail interrupt to stop >DMA's in its tracks on an power failure, so that you don't have >garbage written to key filesystem data structures when the memory >starts suffering from the dropping voltage on the power bus faster >than the DMA engine or the disk drives. So XFS is a great filesystem >--- but you'd better be running it on a UPS, or on a system which has >power fail interrupts and an OS that knows what to do. Ext3, because >it does physical block journalling, does not suffer from this problem. >(Yes, Resierfs uses logical journalling as well, so it suffers from >the same problem.) > >So perhaps it's not the job of the FS vendor to be responsible for >crap hardware or lazy sysadmins that don't do backups. But a system >administrator who knows that he doesn't do backups frequently enough, >or is running on cheap, crap hardware, would be wise to consider >carefully which filesystem he/she wants to use given the systems >configuration and his backup habits. > >Me, I'll go for the robust filesystem, just on general principles. As >a friend from the large-scale enterprise storage world once put it, >"Performance is Job 2. Robustness is Job #1." (Of course, if you >want to put your fragile filesystem on a multi-million dollar >enterprise storage system such as an IBM Shark or an EMC Symmetrix >box, I'm sure IBM or EMC will be happy to sell you one. :-) > > - Ted > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 19:46 ` Hans Reiser @ 2005-06-27 20:18 ` Steve Lord 2005-06-27 20:28 ` Theodore Ts'o 2005-06-28 6:37 ` Al Boldi 2005-06-27 21:26 ` Theodore Ts'o 1 sibling, 2 replies; 620+ messages in thread From: Steve Lord @ 2005-06-27 20:18 UTC (permalink / raw) To: Hans Reiser Cc: Theodore Ts'o, Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Hans Reiser wrote: > Steve, there is a remark about XFS below which you are going to be more > expert on. > > Theodore Ts'o wrote: > >> >>XFS has similar issues where it assumes that hardware has powerfail >>interrupts, and that the OS can use said powerfail interrupt to stop >>DMA's in its tracks on an power failure, so that you don't have >>garbage written to key filesystem data structures when the memory >>starts suffering from the dropping voltage on the power bus faster >>than the DMA engine or the disk drives. So XFS is a great filesystem >>--- but you'd better be running it on a UPS, or on a system which has >>power fail interrupts and an OS that knows what to do. Ext3, because >>it does physical block journalling, does not suffer from this problem. >>(Yes, Resierfs uses logical journalling as well, so it suffers from >>the same problem.) >> I presume Ted is referring to problems guaranteeing the integrity of the journal at recovery time. I am coming into this without all the available context, so I may be barking up the wrong tree.... In particular, I am not sure how journaling whole blocks protects you from this. The xfs journal protects itself against partial writes, to a certain degree. The header of a journal write (inside a 512 byte sector) contains an array of words which are swapped out from the start of each following 512 byte sector of the journal write. The following sectors then each have the log sequence number (LSN) of the write inserted in place of that data. During recovery, we find the most recent LSN via a binary chop search, this gives us an associated tail LSN. A scan backwards from the head LSN is then done - this covers the total possible amount of in flight data (maximum log buffers x maximum log buffer size). If any of the sectors has the wrong LSN in the first word, then it an all following data is discarded from replay. Of course, we will also not replay any journal entry for which we do not find the transaction commit record. Now, this protects against some failure cases, it assumes that sector writes are atomic, they either happen or they do not happen. If sector writes are not atomic and one end can be good with the other is bad, then a partial sector is possibly going to get replayed. There have been discussions about doing this with the head and tail of each sector, or using a checksum instead. XFS on linux has had power cycle crash testing, but there is no way you can cover all possible hardware configurations, and I seem to recall some hardware never recovered from this testing, by that I mean the PC did not survive the continual power cycling and went up in smoke. Steve ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 20:18 ` Steve Lord @ 2005-06-27 20:28 ` Theodore Ts'o 2005-06-27 20:47 ` Hans Reiser 2005-06-27 20:58 ` Steve Lord 2005-06-28 6:37 ` Al Boldi 1 sibling, 2 replies; 620+ messages in thread From: Theodore Ts'o @ 2005-06-27 20:28 UTC (permalink / raw) To: Steve Lord Cc: Hans Reiser, Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Mon, Jun 27, 2005 at 03:18:30PM -0500, Steve Lord wrote: > I presume Ted is referring to problems guaranteeing the integrity of > the journal at recovery time. I am coming into this without all the > available context, so I may be barking up the wrong tree.... In > particular, I am not sure how journaling whole blocks protects > you from this. Actually, I was talking about the problem what happens when power fails while DMA'ing to the disk, and memory, which is more sensitive to voltage drops than the rest of the system, starts sending garbage to the bus, which the disk then faithfully writes to the inode table. As I recall, you were the one who told me about this problem, and how it was fixed in Irix by using a powerfail interrupt to abort DMA transfers, as well as giving me a program which tests for this condition (basically it writes known test pattern to the disk, and then you do an unclean shutdown, and you look to see if garbage is written to the disk instead of one of the known test patterns). If it wasn't you, it must have been Jim Mostek --- but I could have sworn it was you..... - Ted ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 20:28 ` Theodore Ts'o @ 2005-06-27 20:47 ` Hans Reiser 2005-06-27 20:58 ` Steve Lord 1 sibling, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-27 20:47 UTC (permalink / raw) To: Theodore Ts'o Cc: Steve Lord, Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List, vitaly, Chris Mason Theodore Ts'o wrote: >On Mon, Jun 27, 2005 at 03:18:30PM -0500, Steve Lord wrote: > > >>I presume Ted is referring to problems guaranteeing the integrity of >>the journal at recovery time. I am coming into this without all the >>available context, so I may be barking up the wrong tree.... In >>particular, I am not sure how journaling whole blocks protects >>you from this. >> >> > >Actually, I was talking about the problem what happens when power >fails while DMA'ing to the disk, and memory, which is more sensitive >to voltage drops than the rest of the system, starts sending garbage >to the bus, which the disk then faithfully writes to the inode table. > >As I recall, you were the one who told me about this problem, and how >it was fixed in Irix by using a powerfail interrupt to abort DMA >transfers, as well as giving me a program which tests for this >condition (basically it writes known test pattern to the disk, and >then you do an unclean shutdown, and you look to see if garbage is >written to the disk instead of one of the known test patterns). If it >wasn't you, it must have been Jim Mostek --- but I could have sworn it >was you..... > > - Ted > > > > Maybe a more positive approach would be to say, hey, I got this program from the XFS guys that tests for DMA problems on power failure, maybe all the Linux filesystem developers should give it a try? Shall we try that style of interaction? Ted, if this corruption hits an ext3 directory structure, and that corrupted directory structure commits, what happens? With ReiserFS (Chris please comment here) it would have to corrupt the directory in the journal, return successful IO, then after that point not corrupt the commit block and have enough power to write the commit block. How is that different for XFS or ext3? Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 20:28 ` Theodore Ts'o 2005-06-27 20:47 ` Hans Reiser @ 2005-06-27 20:58 ` Steve Lord 2005-06-27 23:06 ` Prakash Punnoor 1 sibling, 1 reply; 620+ messages in thread From: Steve Lord @ 2005-06-27 20:58 UTC (permalink / raw) To: Theodore Ts'o Cc: Hans Reiser, Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Theodore Ts'o wrote: > On Mon, Jun 27, 2005 at 03:18:30PM -0500, Steve Lord wrote: > >>I presume Ted is referring to problems guaranteeing the integrity of >>the journal at recovery time. I am coming into this without all the >>available context, so I may be barking up the wrong tree.... In >>particular, I am not sure how journaling whole blocks protects >>you from this. > > > Actually, I was talking about the problem what happens when power > fails while DMA'ing to the disk, and memory, which is more sensitive > to voltage drops than the rest of the system, starts sending garbage > to the bus, which the disk then faithfully writes to the inode table. > > As I recall, you were the one who told me about this problem, and how > it was fixed in Irix by using a powerfail interrupt to abort DMA > transfers, as well as giving me a program which tests for this > condition (basically it writes known test pattern to the disk, and > then you do an unclean shutdown, and you look to see if garbage is > written to the disk instead of one of the known test patterns). If it > wasn't you, it must have been Jim Mostek --- but I could have sworn it > was you..... > > - Ted > Your memory is better than mine, not sure about the test program, but there was at one point a scenario like that on Irix, and I quite probably did mention it to you. That was certainly not PC hardware, but more likely a large Irix system with multiple power busses spread around a large amount of hardware. I presume you are saying that the fact that ext3 journals complete inode blocks rather that subsections of inodes means that, should a similar trashing of metadata occur during power down, a journal replay would fix it? I see no way short of hardware fixes of avoiding the general problem of a system failing in an ugly manner like this. Unless you write everything to disk twice (i.e. journal all data), you can still end up with a legitimate set of metadata, and the master copy of your employee database full of nasty little bits of corruption. Steve ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 20:58 ` Steve Lord @ 2005-06-27 23:06 ` Prakash Punnoor 2005-06-28 0:40 ` Hans Reiser 2005-06-28 1:07 ` Jim Crilly 0 siblings, 2 replies; 620+ messages in thread From: Prakash Punnoor @ 2005-06-27 23:06 UTC (permalink / raw) To: Steve Lord Cc: Theodore Ts'o, Hans Reiser, Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 3268 bytes --] Steve Lord schrieb: > I see no way short of hardware fixes of avoiding the general problem of > a system failing in an ugly manner like this. Unless you write everything > to disk twice (i.e. journal all data), you can still end up with a > legitimate set of metadata, and the master copy of your employee > database full of nasty little bits of corruption. If I as a user may add my own little experience with different fs on my computer, where I tend to use kernels or features that cause my machine to (hard) lock up: 2.4.2x kernel and reiserfs (V3): This was the only one which really managed to smoke my partition... 2.6.x-test-y till 2.6.x stable kernel and reiserfs (V3): I did a short intermezzo with xfs but at that time it felt too slow to bear, so I gave reiserfs another chance. And here it was very robust. No lock-up did serious damage to the data. Only currently accessed files were partly overwritten with @^@^@^ and so on... After a year or so of heavy usage the formerly fast reiserfs partition became more and more slow till I got sick of it. So I tried JFS and forced a hard lock. Data was OK, but (probably error in gentoo scripts) fsck wasn't automatically called (or failed, I don't remember) and I had to do it by hand, which was a major annoyance - I thought then. So I gave ext3 a try. Very robust, but at the same time slooow. I couldn't bear it after some months. So I gave xfs another try. Yes, now it felt much better. Still not that fast as reiserfs, IIRC, but better than the first time I tried. I am still having xfs on / and it works pretty well, and is rather robust against hard locks with about the same amount of data losing as reiserfs. But what annoys me very much, is that I have to run xfs_repair by hand and by booting from another partition. Even after a hard lock, the partition mounts w/o problems and everything seems OK, but it only seems like that. In fact after some hours/days of use, you'll notice oddities, like files or directories which cannot be removed and things like that. After running xfs_repair everything is back in order. In between I gave an alpha (or rather several alphas) of reiser4 a try - but not on /, just on /usr. Well, I wouldn't say it was extraordinary fast. In fact it felt slower than reiserfs V3, but much more space efficient. And to my surprise it was very robust as well. Hard-locks were no problem. Only annoying then was, that unmounting regularly produced oops but later versions corrected that. But nevertheless it didn't survive, as like V3, with time V4 became slower and slower. In this case no year was needed, but just one month or alike. So end of test...but in fact I'll give V4 another go in the near future. All in all I can say that every fs I tested was able to not smoke all of my data, even using an instable machine - but only ext3, reiser v3 and v4 were non-annoying. But xfs is majorly annoying in that respect. I hope that new versions will be able to keep consistency w/o having to run xfs_repair every time after a lock-up... But what I don't understand is, that sometimes even files, which were only opened for reading, got overwritten with @^@^@ after a lock-up. I don't understand the logics here, how this could happen. Thx for your time, Prakash [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 23:06 ` Prakash Punnoor @ 2005-06-28 0:40 ` Hans Reiser 2005-06-28 1:00 ` Zan Lynx 2005-06-28 1:10 ` Gregory Maxwell 2005-06-28 1:07 ` Jim Crilly 1 sibling, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-28 0:40 UTC (permalink / raw) To: Prakash Punnoor Cc: Steve Lord, Theodore Ts'o, Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Prakash Punnoor wrote: > >. But nevertheless it didn't survive, as like V3, with time V4 became >slower and slower. In this case no year was needed, but just one month or >alike. So end of test...but in fact I'll give V4 another go in the near future. > > Interesting that it got slower with time. It sounds like our online repacker is much needed. It will be a priority for after the kernel merge. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 0:40 ` Hans Reiser @ 2005-06-28 1:00 ` Zan Lynx 2005-06-28 1:10 ` Gregory Maxwell 1 sibling, 0 replies; 620+ messages in thread From: Zan Lynx @ 2005-06-28 1:00 UTC (permalink / raw) To: Hans Reiser Cc: Prakash Punnoor, Steve Lord, Theodore Ts'o, Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 593 bytes --] On Mon, 2005-06-27 at 17:40 -0700, Hans Reiser wrote: > Prakash Punnoor wrote: > > > > >. But nevertheless it didn't survive, as like V3, with time V4 became > >slower and slower. In this case no year was needed, but just one month or > >alike. So end of test...but in fact I'll give V4 another go in the near future. > > > > > Interesting that it got slower with time. It sounds like our online > repacker is much needed. It will be a priority for after the kernel merge. I told y'all that last year on the reiserfs-list, with benchmarks :-) -- Zan Lynx <zlynx@acm.org> [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 0:40 ` Hans Reiser 2005-06-28 1:00 ` Zan Lynx @ 2005-06-28 1:10 ` Gregory Maxwell 2005-06-28 20:15 ` David Masover 1 sibling, 1 reply; 620+ messages in thread From: Gregory Maxwell @ 2005-06-28 1:10 UTC (permalink / raw) To: Hans Reiser, reiserfs-list On 6/27/05, Hans Reiser <reiser@namesys.com> wrote: > >. But nevertheless it didn't survive, as like V3, with time V4 became > >slower and slower. In this case no year was needed, but just one month or > >alike. So end of test...but in fact I'll give V4 another go in the near future. > Interesting that it got slower with time. It sounds like our online > repacker is much needed. It will be a priority for after the kernel merge. Whered it go, I recall it being activated with: echo 1 >/sys/fs/reiser4/*/repacker/start ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 1:10 ` Gregory Maxwell @ 2005-06-28 20:15 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-28 20:15 UTC (permalink / raw) To: Gregory Maxwell; +Cc: Hans Reiser, reiserfs-list -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Gregory Maxwell wrote: > On 6/27/05, Hans Reiser <reiser@namesys.com> wrote: > >>>. But nevertheless it didn't survive, as like V3, with time V4 became >>>slower and slower. In this case no year was needed, but just one month or >>>alike. So end of test...but in fact I'll give V4 another go in the near future. >> >>Interesting that it got slower with time. It sounds like our online >>repacker is much needed. It will be a priority for after the kernel merge. > > > Whered it go, I recall it being activated with: > echo 1 >/sys/fs/reiser4/*/repacker/start That proved to be unstable (on my box, the repacker never finished) and there was also some talk of making it only available commercially -- that is, pay some amount based on how much storage you're using it on, get the repacker. I like open source better, but whatever amount was proposed is more than fair for me, especially considering how fast stuff is even without the repacker. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsGv0XgHNmZLgCUhAQKluQ/+Pjg6h5qUpO/6pG7v8ku2OhyUoR6aneTb FlqjaqNuxqKaT/C3iqZYLZ+YUk+MYdXKUYNthdlY9ZtaPNwnp34/AHUSLtU4Imgw Z4kIn3csKCkpeLRQMJ1o5nRSBwa+38Ta17ktYhn7L+ln53/ElULbPrmxjoSmA+SK ocjkVcUI8azJJosD1Y0tb5BSVa9PC+buPOgbkinvFVeme2ouwG+tAeAMECShface 0FbT1+Ai//S/t5rz/oz6BzlV+K7Jl+8OopFs9MnwD8Gqh21zeGiawLQGsDwvZuEt wJq73Y6b573RVJUy4dHAUqBZen0gar1GPWS2RpB3RwIYgb7/Ui3Rf5t//vTrZMNo Ej9R3AXcU82M8YHHmwr1QJfhrt7kddXPw5TSpA66RWNJdeFxoaXxja6sFFjQbSTs G5BHy2P/oHO8+uhbS+NI8T1N8KsrRrhJWr0pcjhHaoDogGdUj7TZtB+fANgLHbdV GstRVz8QvKGOZ8FrX/mMPzsfUbRx3pq+C/FJzSgqVI/4oTc/XIOS7cb92EC+1WrO KhYI3vy1zeKximboJXrPlr7QXkakjoOSFNH/MVzmF3sP1pe92hey7SXrpfgrOCOa 8gVo0u1tE98fuatl3MA8t3OGXGM5JXKpHVk/FiQt4358wAMmyDizvOeZ3GL1CA/M ixjK/caPrP0= =LFCF -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 23:06 ` Prakash Punnoor 2005-06-28 0:40 ` Hans Reiser @ 2005-06-28 1:07 ` Jim Crilly 2005-06-28 4:03 ` Prakash Punnoor 1 sibling, 1 reply; 620+ messages in thread From: Jim Crilly @ 2005-06-28 1:07 UTC (permalink / raw) To: Prakash Punnoor Cc: Steve Lord, Theodore Ts'o, Hans Reiser, Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On 06/28/05 01:06:54AM +0200, Prakash Punnoor wrote: > > So I gave ext3 a try. Very robust, but at the same time slooow. I couldn't > bear it after some months. So I gave xfs another try. Yes, now it felt much > better. Still not that fast as reiserfs, IIRC, but better than the first time > I tried. I am still having xfs on / and it works pretty well, and is rather > robust against hard locks with about the same amount of data losing as > reiserfs. But what annoys me very much, is that I have to run xfs_repair by > hand and by booting from another partition. Even after a hard lock, the > partition mounts w/o problems and everything seems OK, but it only seems like > that. In fact after some hours/days of use, you'll notice oddities, like files > or directories which cannot be removed and things like that. After running > xfs_repair everything is back in order. I don't know what was going on with your systems, but I've been using XFS since the original 1.0 Linux release from SGI and I'd guess that I've had to run xfs_repair less than 10 times and most of them were on Alpha and Sparc64 before issues with those arches got ironed out. Right now I have XFS on both Alpha and Sparc64 and I haven't had any issues on either box. Infact the Sparc64 box's XFS filesystem was converted from reiser3 because the some part of the filesystem got mysteriously corrupted in such a way that reiserfsck and the reiser3 driver both thought it was fine but accessing a certain file would cause a lockup. I really hope the reiser4 userland tools are a lot better than the reiser3 tools, that's an area that reiserfs has lagged behind hugely IMO. > > In between I gave an alpha (or rather several alphas) of reiser4 a try - but > not on /, just on /usr. Well, I wouldn't say it was extraordinary fast. In > fact it felt slower than reiserfs V3, but much more space efficient. And to my > surprise it was very robust as well. Hard-locks were no problem. Only annoying > then was, that unmounting regularly produced oops but later versions corrected > that. But nevertheless it didn't survive, as like V3, with time V4 became > slower and slower. In this case no year was needed, but just one month or > alike. So end of test...but in fact I'll give V4 another go in the near future. > > > All in all I can say that every fs I tested was able to not smoke all of my > data, even using an instable machine - but only ext3, reiser v3 and v4 were > non-annoying. But xfs is majorly annoying in that respect. I hope that new > versions will be able to keep consistency w/o having to run xfs_repair every > time after a lock-up... > > But what I don't understand is, that sometimes even files, which were only > opened for reading, got overwritten with @^@^@ after a lock-up. I don't > understand the logics here, how this could happen. > > Thx for your time, > > Prakash Jim. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 1:07 ` Jim Crilly @ 2005-06-28 4:03 ` Prakash Punnoor 2005-06-28 4:19 ` Jim Crilly 0 siblings, 1 reply; 620+ messages in thread From: Prakash Punnoor @ 2005-06-28 4:03 UTC (permalink / raw) To: Jim Crilly Cc: Steve Lord, Theodore Ts'o, Hans Reiser, Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1542 bytes --] Jim Crilly schrieb: > On 06/28/05 01:06:54AM +0200, Prakash Punnoor wrote: > >>So I gave ext3 a try. Very robust, but at the same time slooow. I couldn't >>bear it after some months. So I gave xfs another try. Yes, now it felt much >>better. Still not that fast as reiserfs, IIRC, but better than the first time >>I tried. I am still having xfs on / and it works pretty well, and is rather >>robust against hard locks with about the same amount of data losing as >>reiserfs. But what annoys me very much, is that I have to run xfs_repair by >>hand and by booting from another partition. Even after a hard lock, the >>partition mounts w/o problems and everything seems OK, but it only seems like >>that. In fact after some hours/days of use, you'll notice oddities, like files >>or directories which cannot be removed and things like that. After running >>xfs_repair everything is back in order. > > > I don't know what was going on with your systems, but I've been using XFS > since the original 1.0 Linux release from SGI and I'd guess that I've had to run > xfs_repair less than 10 times and most of them were on Alpha and Sparc64 > before issues with those arches got ironed out. Perhaps it is due to the fact that I use xfs on software RAID-0 and both HDs have 8MB cache write-back enabled? So, all in all 16MB needs to be commited on/before lock-up, maybe too much for xfs? (This situation was no prob for ext3, though. Thinking again, I never used reiser V3 or V4 on the RAID-0, so my comparison might not have been fair.) Prakash [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 4:03 ` Prakash Punnoor @ 2005-06-28 4:19 ` Jim Crilly 0 siblings, 0 replies; 620+ messages in thread From: Jim Crilly @ 2005-06-28 4:19 UTC (permalink / raw) To: Prakash Punnoor Cc: Steve Lord, Theodore Ts'o, Hans Reiser, Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On 06/28/05 06:03:25AM +0200, Prakash Punnoor wrote: > Jim Crilly schrieb: > > On 06/28/05 01:06:54AM +0200, Prakash Punnoor wrote: > > > >>So I gave ext3 a try. Very robust, but at the same time slooow. I couldn't > >>bear it after some months. So I gave xfs another try. Yes, now it felt much > >>better. Still not that fast as reiserfs, IIRC, but better than the first time > >>I tried. I am still having xfs on / and it works pretty well, and is rather > >>robust against hard locks with about the same amount of data losing as > >>reiserfs. But what annoys me very much, is that I have to run xfs_repair by > >>hand and by booting from another partition. Even after a hard lock, the > >>partition mounts w/o problems and everything seems OK, but it only seems like > >>that. In fact after some hours/days of use, you'll notice oddities, like files > >>or directories which cannot be removed and things like that. After running > >>xfs_repair everything is back in order. > > > > > > I don't know what was going on with your systems, but I've been using XFS > > since the original 1.0 Linux release from SGI and I'd guess that I've had to run > > xfs_repair less than 10 times and most of them were on Alpha and Sparc64 > > before issues with those arches got ironed out. > > Perhaps it is due to the fact that I use xfs on software RAID-0 and both HDs > have 8MB cache write-back enabled? So, all in all 16MB needs to be commited > on/before lock-up, maybe too much for xfs? (This situation was no prob for > ext3, though. Thinking again, I never used reiser V3 or V4 on the RAID-0, so > my comparison might not have been fair.) Maybe, I've never used XFS on software RAID-0. The Sparc64 I mentioned has it's XFS filesystem on software RAID-1 and that's introduced no problems, but I've seen way too many drives die to risk running RAID-0. > > Prakash Jim. ^ permalink raw reply [flat|nested] 620+ messages in thread
* RE: reiser4 plugins 2005-06-27 20:18 ` Steve Lord 2005-06-27 20:28 ` Theodore Ts'o @ 2005-06-28 6:37 ` Al Boldi 1 sibling, 0 replies; 620+ messages in thread From: Al Boldi @ 2005-06-28 6:37 UTC (permalink / raw) To: linux-kernel; +Cc: linux-fsdevel Hans Reiser wrote: > Steve, there is a remark about XFS below which you are going to be > more expert on. > > Theodore Ts'o wrote: > >> >>XFS has similar issues where it assumes that hardware has powerfail >>interrupts, and that the OS can use said powerfail interrupt to stop >>DMA's in its tracks on an power failure, so that you don't have >>garbage written to key filesystem data structures when the memory >>starts suffering from the dropping voltage on the power bus faster >>than the DMA engine or the disk drives. So XFS is a great filesystem >>--- but you'd better be running it on a UPS, or on a system which has >>power fail interrupts and an OS that knows what to do. Ext3, because >>it does physical block journalling, does not suffer from this problem. >>(Yes, Resierfs uses logical journalling as well, so it suffers from >>the same problem.) >> True now, not so around 2.4.20 when XFS was rock-solid. I think they tried to improve on performance and broke something. I wish they would fix that because it forced me back to ext3, as in consistency over performance any time. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 19:46 ` Hans Reiser 2005-06-27 20:18 ` Steve Lord @ 2005-06-27 21:26 ` Theodore Ts'o 2005-06-27 23:00 ` reiser4 merging action list Hans Reiser 2005-06-28 0:06 ` reiser4 plugins Hans Reiser 1 sibling, 2 replies; 620+ messages in thread From: Theodore Ts'o @ 2005-06-27 21:26 UTC (permalink / raw) To: Hans Reiser Cc: Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List, Steve Lord On Mon, Jun 27, 2005 at 12:46:23PM -0700, Hans Reiser wrote: > A difference between us is > that I tell them that with all the major linux filesystems (I include > XFS and JFS in this) it is by this time far more likely to be hardware > that caused corruption than the filesystem software, whereas I guess you > tell them something else. Oh, I agree with this, and I do tell people that. The question though is how the filesystem recovers from said hardware-caused corruption once it does happen. You've admitted that reiserfs3 has less than optimal recovery characteristics from hardware-induced corruptions if said filesystem contains reiserfs filesystem images; that would be an example of a filesystem not being as robust as it could be. (It'll be interesting to see if SuSE will support reiserfsv3 in combination with the Xen hypervisor or other virtualization systems, which makes use of filesystem images.) Another example would be DMA'ing garbage into the hard drive after a power failure --- how does a filesystem respond to this eventuality? You probably hear more stories people who got unlucky with hardware-induced corruptions with ext2/3, and I probably hear more from users who have sworn off of reiserfs just because are sample sets are somewhat biased. Such are the dangers of relying on anecdotal evidence. However, logically speaking, if a filesystem is designed such that in certain cases, the fsck program has to brute-force search every single disk block looking for data structures that _look_ like they might be part of the filesystem data, well, that's always going to be more error prone than one where the filesystem metadata is in easily-predicted locations. It sounds like you've added some more checks in reiser4, and that's definitely a good thing. Time will tell whether they are sufficient or not. Regards, - Ted ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 merging action list 2005-06-27 21:26 ` Theodore Ts'o @ 2005-06-27 23:00 ` Hans Reiser 2005-06-27 23:23 ` Andrew Morton ` (2 more replies) 2005-06-28 0:06 ` reiser4 plugins Hans Reiser 1 sibling, 3 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-27 23:00 UTC (permalink / raw) To: Theodore Ts'o Cc: Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List, Steve Lord Andrew asked me to put together a list of things that need to be done before merging: * VFS will dispatch directly to the method of the plugin for the *_operations methods. This requires duplicating to all plugin methods the common code currently used by all reiser4 plugins for a given method. It has the desirable side effect of making the methods more fully self-contained, which is somethng I had wanted two years ago and was a little sad to not get, and the cost of duplicating some code. Since not all plugin methods are *_operations, it means we have two structures with duplicated data, and duplicate data that must be in sync at all times is classical badness in programming technique (see Codd and normalization). vs owns this task * review all sparse complaints, and revise as appropriate. * panic and code beauty: everyone agrees that having function, file, and line added to reiser4_panic output hurts nothing (I hope). Everyone agrees that restarting the machine without an error message seems like a useless option to allow. Much else was argued, not sure if anything was a consensus view. Various detail improvements were suggested by Pecca, and I agreed with half of them. * metafiles should be disabled until we can present code that works right. Half the list thinks we cannot solve the cycles problem ever. Disable metafiles and postpone problem until working code, or the failure to produce it, makes it possible to do more than rant at each other. This is currently already done in the -mm patches, but is mentioned lest someone think it forgotten. * update the locking documentation Probably I forget something. Best, Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 merging action list 2005-06-27 23:00 ` reiser4 merging action list Hans Reiser @ 2005-06-27 23:23 ` Andrew Morton 2005-06-29 5:41 ` Hans Reiser 2005-06-28 9:41 ` Christoph Hellwig 2005-06-28 9:48 ` Adrian Bunk 2 siblings, 1 reply; 620+ messages in thread From: Andrew Morton @ 2005-06-27 23:23 UTC (permalink / raw) To: Hans Reiser Cc: tytso, mjt, vonbrand, ninja, alan, jgarzik, hch, linux-kernel, reiserfs-list, lord Hans Reiser <reiser@namesys.com> wrote: > > > Andrew asked me to put together a list of things that need to be done > before merging: Thanks. As I said to Hans, if we can get a list of bullet-point actions nailed down and agreed to then we have an uncontroversial path to happiness and a merge. Let's get down and concentrate on technical specifics. Hans, please maintain this list and republish it as we work through things. > * VFS will dispatch directly to the method of the plugin for the > *_operations methods. This requires duplicating to all plugin methods > the common code currently used by all reiser4 plugins for a given > method. It has the desirable side effect of making the methods more > fully self-contained, which is somethng I had wanted two years ago and > was a little sad to not get, and the cost of duplicating some code. > Since not all plugin methods are *_operations, it means we have two > structures with duplicated data, and duplicate data that must be in sync > at all times is classical badness in programming technique (see Codd and > normalization). vs owns this task > > * review all sparse complaints, and revise as appropriate. > > * panic and code beauty: everyone agrees that having function, file, > and line added to reiser4_panic output hurts nothing (I hope). Everyone > agrees that restarting the machine without an error message seems like a > useless option to allow. Much else was argued, not sure if anything > was a consensus view. Various detail improvements were suggested by > Pecca, and I agreed with half of them. > > > * metafiles should be disabled until we can present code that works > right. Half the list thinks we cannot solve the cycles problem ever. > Disable metafiles and postpone problem until working code, or the > failure to produce it, makes it possible to do more than rant at each > other. This is currently already done in the -mm patches, but is > mentioned lest someone think it forgotten. > > * update the locking documentation > There's also the custom list, hash and debug code. We should either a) remove them or b) generify them and submit as standalone works or c) justify them as custom-to-reiser4 and leave them as-is. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 merging action list 2005-06-27 23:23 ` Andrew Morton @ 2005-06-29 5:41 ` Hans Reiser 2005-06-29 6:18 ` Pekka Enberg 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-29 5:41 UTC (permalink / raw) To: Andrew Morton Cc: tytso, mjt, vonbrand, ninja, alan, jgarzik, hch, linux-kernel, reiserfs-list, lord, vs Andrew Morton wrote: >Hans Reiser <reiser@namesys.com> wrote: > > >>Andrew asked me to put together a list of things that need to be done >>before merging: >> >> > >Thanks. > >As I said to Hans, if we can get a list of bullet-point actions nailed down >and agreed to then we have an uncontroversial path to happiness and a >merge. Let's get down and concentrate on technical specifics. > >Hans, please maintain this list and republish it as we work through things. > > > >> * VFS will dispatch directly to the method of the plugin for the >>*_operations methods. This requires duplicating to all plugin methods >>the common code currently used by all reiser4 plugins for a given >>method. It has the desirable side effect of making the methods more >>fully self-contained, which is somethng I had wanted two years ago and >>was a little sad to not get, and the cost of duplicating some code. >>Since not all plugin methods are *_operations, it means we have two >>structures with duplicated data, and duplicate data that must be in sync >>at all times is classical badness in programming technique (see Codd and >>normalization). vs owns this task >> >> * review all sparse complaints, and revise as appropriate. >> >> * panic and code beauty: everyone agrees that having function, file, >>and line added to reiser4_panic output hurts nothing (I hope). Everyone >>agrees that restarting the machine without an error message seems like a >>useless option to allow. Much else was argued, not sure if anything >>was a consensus view. Various detail improvements were suggested by >>Pecca, and I agreed with half of them. >> >> >> * metafiles should be disabled until we can present code that works >>right. Half the list thinks we cannot solve the cycles problem ever. >>Disable metafiles and postpone problem until working code, or the >>failure to produce it, makes it possible to do more than rant at each >>other. This is currently already done in the -mm patches, but is >>mentioned lest someone think it forgotten. >> >> * update the locking documentation >> >> >> > >There's also the custom list, hash and debug code. We should either > >a) remove them or > >b) generify them and submit as standalone works or > >c) justify them as custom-to-reiser4 and leave them as-is. > > > > > > either b) or c) is ok with me for the list code. The debug code should be c) I think. Probably vs can offer a more detailed and accurate opinion, Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 merging action list 2005-06-29 5:41 ` Hans Reiser @ 2005-06-29 6:18 ` Pekka Enberg 2005-06-29 22:59 ` Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: Pekka Enberg @ 2005-06-29 6:18 UTC (permalink / raw) To: Hans Reiser Cc: Andrew Morton, tytso, mjt, vonbrand, ninja, alan, jgarzik, hch, linux-kernel, reiserfs-list, lord, vs, Pekka Enberg Andrew Morton wrote: > > There's also the custom list, hash and debug code. We should either > > > > a) remove them or > > > > b) generify them and submit as standalone works or > > > > c) justify them as custom-to-reiser4 and leave them as-is. On 6/29/05, Hans Reiser <reiser@namesys.com> wrote: > either b) or c) is ok with me for the list code. The debug code should > be c) I think. > > Probably vs can offer a more detailed and accurate opinion, I completely agree that the current state of the generic hashing facilities is somewhat poor but I fail to see why you can't use <linux/list.h>. As for the debugging code, I would love to see that turned into something generic (every subsystem has their own now) but it is definitely not something that should stop you from merging. Pekka ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 merging action list 2005-06-29 6:18 ` Pekka Enberg @ 2005-06-29 22:59 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-29 22:59 UTC (permalink / raw) To: Pekka Enberg Cc: Andrew Morton, tytso, mjt, vonbrand, ninja, alan, jgarzik, hch, linux-kernel, reiserfs-list, lord, vs, Pekka Enberg, Nikita Danilov Pekka Enberg wrote: >Andrew Morton wrote: > > >>>There's also the custom list, hash and debug code. We should either >>> >>>a) remove them or >>> >>>b) generify them and submit as standalone works or >>> >>>c) justify them as custom-to-reiser4 and leave them as-is. >>> >>> > >On 6/29/05, Hans Reiser <reiser@namesys.com> wrote: > > >>either b) or c) is ok with me for the list code. The debug code should >>be c) I think. >> >>Probably vs can offer a more detailed and accurate opinion, >> >> > >I completely agree that the current state of the generic hashing >facilities is somewhat poor but I fail to see why you can't use ><linux/list.h>. > > I'll let vs and maybe nikita comment. >As for the debugging code, I would love to see that turned into >something generic (every subsystem has their own now) but it is >definitely not something that should stop you from merging. > > Pekka > > > > If I encourage you to make a patch, is that ok of me? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 merging action list 2005-06-27 23:00 ` reiser4 merging action list Hans Reiser 2005-06-27 23:23 ` Andrew Morton @ 2005-06-28 9:41 ` Christoph Hellwig 2005-06-28 9:48 ` Adrian Bunk 2 siblings, 0 replies; 620+ messages in thread From: Christoph Hellwig @ 2005-06-28 9:41 UTC (permalink / raw) To: Hans Reiser Cc: Theodore Ts'o, Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Andrew Morton, Linux Kernel Mailing List, ReiserFS List, Steve Lord On Mon, Jun 27, 2005 at 04:00:01PM -0700, Hans Reiser wrote: > > Andrew asked me to put together a list of things that need to be done > before merging: ... > Probably I forget something. I've started to do a very basic look over the tree and there's a few more things that spring to mind: - cpp abuse. There's quite a lot of really odd macros - the typesafe_list, typesafe_hash stuff is mentioned already, but there's really horrible stuff like _INIT_ and _DONE_ bits in init_super.c, and the wrappers for plugin method invocations. - endianess handling. The d* types are a lovely attempt to make sure you're not missing endianess conversions. We have a more general way to ensure that now using sparse, that's the __le* / __be* types. Try running sparse -Wbitwise to find places you'll need that annotations (after the normal sparse warnings are fixed, else you'll have a hard time seeing them I guess) - after that the single element struct thing can go away. Also you're defining a CPU_IN_DISK_ORDER macro that's never used.. - lease use kthread_* instead of kernel_thread() + lots of opencoding ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 merging action list 2005-06-27 23:00 ` reiser4 merging action list Hans Reiser 2005-06-27 23:23 ` Andrew Morton 2005-06-28 9:41 ` Christoph Hellwig @ 2005-06-28 9:48 ` Adrian Bunk 2 siblings, 0 replies; 620+ messages in thread From: Adrian Bunk @ 2005-06-28 9:48 UTC (permalink / raw) To: Hans Reiser Cc: Theodore Ts'o, Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List, Steve Lord On Mon, Jun 27, 2005 at 04:00:01PM -0700, Hans Reiser wrote: > > Andrew asked me to put together a list of things that need to be done > before merging: >... > Probably I forget something. * remove the dependency on !4KSTACKS > Best, > > Hans cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 21:26 ` Theodore Ts'o 2005-06-27 23:00 ` reiser4 merging action list Hans Reiser @ 2005-06-28 0:06 ` Hans Reiser 1 sibling, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-28 0:06 UTC (permalink / raw) To: Theodore Ts'o Cc: Markus T?rnqvist, Horst von Brand, David Masover, Alan Cox, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List, Steve Lord, vitaly What happens when the ext3 inode tables get hit by sector damage? Like us, ext3 has data munging if you hit the wrong thing, its just that our sources of data munging are different in the details. Details matter though, and so we are improving with each release, and V4 does have some real improvements. I hope the next major release will have atomic transactions that don't have that niggling 5% left undone that prevents them from being fully there, and then reiser4 can seriously advance the field of reliability in filesystems rather than just playing catchup. Good luck with ext3, Hans Theodore Ts'o wrote: >However, logically speaking, if a filesystem is designed such that in >certain cases, the fsck program has to brute-force search every single >disk block looking for data structures that _look_ like they might be >part of the filesystem data, well, that's always going to be more >error prone than one where the filesystem metadata is in >easily-predicted locations. It sounds like you've added some more >checks in reiser4, and that's definitely a good thing. Time will tell >whether they are sufficient or not. > >Regards, > > - Ted > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 9:21 ` mjt @ 2005-06-28 13:44 ` Horst von Brand -1 siblings, 0 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-28 13:44 UTC (permalink / raw) To: Markus Törnqvist Cc: Horst von Brand, David Masover, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain, Size: 1550 bytes --] Markus Törnqvist <mjt@nysv.org> wrote: > On Thu, Jun 23, 2005 at 11:34:50PM -0400, Horst von Brand wrote: > >David Masover <ninja@slaphack.com> wrote: > >> I think Hans (or someone) decided that when hardware stops working, it's > >> not the job of the FS to compensate, it's the job of lower layers, or > >> better, the job of the admin to replace the disk and restore from > >> backups. > >Handling other people's data this way is just reckless irresponsibility. > >Sure, you can get high performance if you just forego some of your basic > >responsibilities. > Your honest-to-bog opinion is that the FS vendor is responsible for > the admin not taking backups or the hardware vendor shipping crap? No. But just relying on perfect hardware and concientious sysadmins is reckless. Hardware /is/ flaky, sysadmins /are/ (sometimes) lazy (and besides, today they are increasingly just plain Joe Sixpack users). Also, backing up a few hundred GiB is /not/ fun, and then keeping track of all the backups is messy. Also, I'm not claiming that they are /solely/ responsible, but not having the filesystem fall apart utterly every time some bug breaths on it /is/ a requirement. > *still trying to understand how that can be* You haven't been around too much yet, do you? -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-28 13:44 ` Horst von Brand 0 siblings, 0 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-28 13:44 UTC (permalink / raw) To: Markus Törnqvist Cc: Horst von Brand, David Masover, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Markus Törnqvist <mjt@nysv.org> wrote: > On Thu, Jun 23, 2005 at 11:34:50PM -0400, Horst von Brand wrote: > >David Masover <ninja@slaphack.com> wrote: > >> I think Hans (or someone) decided that when hardware stops working, it's > >> not the job of the FS to compensate, it's the job of lower layers, or > >> better, the job of the admin to replace the disk and restore from > >> backups. > >Handling other people's data this way is just reckless irresponsibility. > >Sure, you can get high performance if you just forego some of your basic > >responsibilities. > Your honest-to-bog opinion is that the FS vendor is responsible for > the admin not taking backups or the hardware vendor shipping crap? No. But just relying on perfect hardware and concientious sysadmins is reckless. Hardware /is/ flaky, sysadmins /are/ (sometimes) lazy (and besides, today they are increasingly just plain Joe Sixpack users). Also, backing up a few hundred GiB is /not/ fun, and then keeping track of all the backups is messy. Also, I'm not claiming that they are /solely/ responsible, but not having the filesystem fall apart utterly every time some bug breaths on it /is/ a requirement. > *still trying to understand how that can be* You haven't been around too much yet, do you? -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 13:44 ` Horst von Brand @ 2005-06-28 20:47 ` mjt -1 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-28 20:47 UTC (permalink / raw) To: Horst von Brand Cc: David Masover, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1661 bytes --] On Tue, Jun 28, 2005 at 09:44:59AM -0400, Horst von Brand wrote: > >No. But just relying on perfect hardware and concientious sysadmins is >reckless. Hardware /is/ flaky, sysadmins /are/ (sometimes) lazy (and >besides, today they are increasingly just plain Joe Sixpack users). Also, >backing up a few hundred GiB is /not/ fun, and then keeping track of all >the backups is messy. Even home users have started to set up raid mirrors at home now that disk space is cheap. That's a step in the right direction, I suppose, with hardware never being good. Taking backups in an environment where you need a few hundred GiB backed up is not that difficult. Get a separate, redundant box with a big tape changer and drop periodical backups off at your bank's vault. Get a separate, very reduntant box, with a truckload of proven drives in a separate raid box and run your stuff there. Get both of the above. If Joe Sixpack loses his mp3 collection, I don't really care, nor should anyone else. Anything important enough to care about is easy enough to back up. Always. Arrogance? Maybe. >Also, I'm not claiming that they are /solely/ responsible, but not having >the filesystem fall apart utterly every time some bug breaths on it /is/ a >requirement. Reiserfs does not fall apart utterly every time some bug breaths on it. >> *still trying to understand how that can be* >You haven't been around too much yet, do you? Rather I take backups, buy better hardware and understand there's a risk involved. Computers as a complete set can't be trusted, you can only make the best accomodations you can. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-28 20:47 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-28 20:47 UTC (permalink / raw) To: Horst von Brand Cc: David Masover, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1661 bytes --] On Tue, Jun 28, 2005 at 09:44:59AM -0400, Horst von Brand wrote: > >No. But just relying on perfect hardware and concientious sysadmins is >reckless. Hardware /is/ flaky, sysadmins /are/ (sometimes) lazy (and >besides, today they are increasingly just plain Joe Sixpack users). Also, >backing up a few hundred GiB is /not/ fun, and then keeping track of all >the backups is messy. Even home users have started to set up raid mirrors at home now that disk space is cheap. That's a step in the right direction, I suppose, with hardware never being good. Taking backups in an environment where you need a few hundred GiB backed up is not that difficult. Get a separate, redundant box with a big tape changer and drop periodical backups off at your bank's vault. Get a separate, very reduntant box, with a truckload of proven drives in a separate raid box and run your stuff there. Get both of the above. If Joe Sixpack loses his mp3 collection, I don't really care, nor should anyone else. Anything important enough to care about is easy enough to back up. Always. Arrogance? Maybe. >Also, I'm not claiming that they are /solely/ responsible, but not having >the filesystem fall apart utterly every time some bug breaths on it /is/ a >requirement. Reiserfs does not fall apart utterly every time some bug breaths on it. >> *still trying to understand how that can be* >You haven't been around too much yet, do you? Rather I take backups, buy better hardware and understand there's a risk involved. Computers as a complete set can't be trusted, you can only make the best accomodations you can. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 20:47 ` mjt (?) @ 2005-06-28 21:48 ` Jake Maciejewski -1 siblings, 0 replies; 620+ messages in thread From: Jake Maciejewski @ 2005-06-28 21:48 UTC (permalink / raw) To: Markus Törnqvist Cc: Horst von Brand, David Masover, Alan Cox, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Tue, 2005-06-28 at 23:47 +0300, Markus Törnqvist wrote: > On Tue, Jun 28, 2005 at 09:44:59AM -0400, Horst von Brand wrote: > > > >No. But just relying on perfect hardware and concientious sysadmins is > >reckless. Hardware /is/ flaky, sysadmins /are/ (sometimes) lazy (and > >besides, today they are increasingly just plain Joe Sixpack users). Also, > >backing up a few hundred GiB is /not/ fun, and then keeping track of all > >the backups is messy. > > Even home users have started to set up raid mirrors at home now that > disk space is cheap. That's a step in the right direction, I > suppose, with hardware never being good. I've lost more data to my own recklessness and stupidity than filesystem corruption and hardware failure combined. RAID isn't really a good solution. My policy for cheap storage (currently the only variety I own) is when I buy a new hard drive, use the old drive for backups. The old drive is always large enough to hold everything I'd miss and then some. Home users should be capable of doing the same, assuming Windows has something similar to rsync. > > Taking backups in an environment where you need a few hundred GiB > backed up is not that difficult. > > Get a separate, redundant box with a big tape changer and drop > periodical backups off at your bank's vault. > > Get a separate, very reduntant box, with a truckload of proven > drives in a separate raid box and run your stuff there. > > Get both of the above. > > If Joe Sixpack loses his mp3 collection, I don't really care, > nor should anyone else. Anything important enough to care > about is easy enough to back up. Always. > > Arrogance? Maybe. > > >Also, I'm not claiming that they are /solely/ responsible, but not having > >the filesystem fall apart utterly every time some bug breaths on it /is/ a > >requirement. > > Reiserfs does not fall apart utterly every time some bug breaths on it. > > >> *still trying to understand how that can be* > >You haven't been around too much yet, do you? > > Rather I take backups, buy better hardware and understand there's a > risk involved. > > Computers as a complete set can't be trusted, you can only make > the best accomodations you can. > -- Jake Maciejewski <maciejej@msoe.edu> ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 3:17 ` David Masover 2005-06-24 3:34 ` Horst von Brand @ 2005-06-24 11:34 ` Alan Cox 2005-06-24 19:21 ` Hans Reiser 2005-06-25 3:14 ` reiser4 plugins David Masover 2005-06-24 12:49 ` Olivier Galibert 2005-06-24 19:46 ` Hans Reiser 3 siblings, 2 replies; 620+ messages in thread From: Alan Cox @ 2005-06-24 11:34 UTC (permalink / raw) To: David Masover Cc: Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Gwe, 2005-06-24 at 04:17, David Masover wrote: > > False argument. So does the pen, so do hinges on doors. Do you still > > have hinges on your doors - probably. > > Indeed. Because there's nothing better -- not because I "like it the > way it is". I chose hinges carefully - there are better technologies than the generic hinge and there have been for many many years (plus cool stuff like magnetic doors). You of course aren't a door geek (nor am I I'd point out) but a computing one... > Although resilience to disk errors isn't a design decision. That's what > SMART and new hard drives are for. And if you're stubborn enough to > keep the same FS around, there's dm-bbr. Its very much a file system design consideration. If you get a small amount of corruption or lost blocks which is the usual drive failure case you want a high probability of getting the data back. BSD style bitmap/inode table file systems based on FFS concepts (UFS, ext2, ext3 etc) are very good in that respect and your loss is usually minimal. > I think Hans (or someone) decided that when hardware stops working, it's > not the job of the FS to compensate, it's the job of lower layers, or > better, the job of the admin to replace the disk and restore from backups. Most desktop users today don't have backups because there is no credible backup technology for 500Gb of data. They may have partial backups. Some things the fs can't deal with - if the disk goes boom then thats a lower level concern. Also certain bits like writing to spare blocks on a problem write are indeed handled drive level nowdays. > > Entirely or bad blocks ? The latter should have a minimal cost on a well > > designed fs. > > I was able to recover from bad blocks, though of course no Reiser that I > know of has had bad block relocation built in... But I got all my files > off of it, fortunately. Thats a good sign. reiser3 was very fragile when blocks of disk went for a walk. If you got most of your data back thats a positive sign, not statistically a valid sample but a positive sign > the issues are fixed, an entirely different crowd of benevolent > dictators will come around and say that we can't get in because we > change the VFS. At least some people on this list have said things to > that effect. There are four important issues I see here 1. It must work 2. It must be clean code that follows the kernel style 3. It must not break other stuff 4. It needs a maintainer who won't get bored 12 months later and only support reiser5 #3 is the VFS stuff and getting the VFS locking wrong or unclean is a *very* big deal because you'll cause corruption to non reiser4 users. Alan ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 11:34 ` Alan Cox @ 2005-06-24 19:21 ` Hans Reiser 2005-06-24 22:04 ` Olivier Galibert ` (2 more replies) 2005-06-25 3:14 ` reiser4 plugins David Masover 1 sibling, 3 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-24 19:21 UTC (permalink / raw) To: Alan Cox Cc: David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Alan Cox wrote: >Thats a good sign. reiser3 was very fragile when blocks of disk went for >a walk. If you got most of your data back thats a positive sign, not >statistically a valid sample but a positive sign > > Alan, this is FUD. Our V3 fsck was written after everything else was, for lack of staffing reasons (why write an fsck before you have an FS worth using). As a result, there was a long period where the fsck code was unstable. It is reliable now. People often think that our tree makes fsck less robust. Actually fsck can throw the entire internal tree away and rebuild from leaf nodes, and frankly that makes things pretty robust. There is an area where we suffered from writing fsck last. When there are two leaf nodes with the same key range AND the bitmap cannot be trusted to tell us which is the valid one, we don't know which is the most recent, and pick arbitrarily. Also, if you store a backup of V3, and you don't compress it, and you wipe out the bitmap blocks and need to use fsck, we don't know what blocks are backup image and what blocks are the fs. We advise users to never store a V3 backup on V3 without compressing it. Additionally, if you create a filesystem, use it a lot, and then mkfs on top of it, and use the new filesystem, and then wipe out the bitmap blocks so that we cannot tell what was in the original fs and what is in the current fs, things get merged together. Also, anytime you wipe out the bitmap blocks, you can get files that were deleted. This turns out to be a feature for many users who delete a file, regret it, email us, we usually get it back just fine by ignoring the bitmap blocks and recovering everything we can find. In V4 we fixed all this. In V4 we started by asking Vitaly our fsck guy what node format changes would make his code easier and better. We put timestamps in every formatted node, and then because we don't trust clocks we also put an mkfsid. Thus, we can distinguish the current and the previously made filesystem even if the clock is moved backwards. We can distinguish backups of other filesystems that were not compressed from the real filesystem. Timestamps allows fixing everything that I described above. What is more important, to fix V3 as described above would require a disk format change that makes it prohibitive to roll out in a minor release, and in V4, to change the disk format just requires adding a plugin which could be done in a minor release. Not only did we fix the problem in V4, we fixed the meta problem so that any future disk format problems are now no biggie to fix. >4. It needs a maintainer who won't get bored 12 months later and only >support reiser5 > > Alan, Chris and Jeff support V3. Whenever they hit a bug that I think someone else should fix, or fail to get around to something fast enough (rare), I have vs do it. What more do you want? What Chris and Jeff do that really irks me is add FEATURES not bugfixes to V3. As a result, almost all bugs for the last 2 years in V3 have been theirs, and mostly due to features being added. It also annoys me that they don't send every patch to a second person to test before sending it in and getting it accepted, like everyone else at Namesys does. What can I say, despite these gripes I am lucky to have them helping. Features (with some exceptions) belong in major releases, not in bugfix releases. Namesys follows an industry standard process of putting features in major releases and only bugfixes in the stable incremental releases. When we, or anyone else in the industry, do this, there are always users who complain that their feature that they need right now has been put into the major release and not released to them right now in the incremental releases. They usually use the phrase "you aren't supporting the (insert name of stable branch), you have abandoned us!" Now Alan, as I understand it, it is usually YOU who is wishing that Linux would follow something closer to the release methodology that I just described. Maybe we have more commonality than you realize? Hans PS Of course, with V4 we will have an interesting issue in that the plugins make it so easy to add features that there will be a temptation to let things dribble out one plugin at a time, with releases being more releases of plugins than releases of an entire filesystem. I haven't really thought through how to handle that best. I think I want a user to be able to define that he wants the mission critical server version of reiser4, and have that be an utterly stable body of code with the latest plugins excluded from it. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 19:21 ` Hans Reiser @ 2005-06-24 22:04 ` Olivier Galibert 2005-06-24 23:06 ` Theodore Ts'o 2005-06-26 17:20 ` Alan Cox 2 siblings, 0 replies; 620+ messages in thread From: Olivier Galibert @ 2005-06-24 22:04 UTC (permalink / raw) To: Linux Kernel Mailing List On Fri, Jun 24, 2005 at 12:21:18PM -0700, Hans Reiser wrote: > Alan, this is FUD. Our V3 fsck was written after everything else was, > for lack of staffing reasons (why write an fsck before you have an FS > worth using). As a result, there was a long period where the fsck code > was unstable. It is reliable now. Was that more or less that about a year ago? OG. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 19:21 ` Hans Reiser 2005-06-24 22:04 ` Olivier Galibert @ 2005-06-24 23:06 ` Theodore Ts'o 2005-06-25 0:35 ` Hans Reiser 2005-06-26 17:20 ` Alan Cox 2 siblings, 1 reply; 620+ messages in thread From: Theodore Ts'o @ 2005-06-24 23:06 UTC (permalink / raw) To: Hans Reiser Cc: Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Fri, Jun 24, 2005 at 12:21:18PM -0700, Hans Reiser wrote: > There is an area where we suffered from writing fsck last. When there > are two leaf nodes with the same key range AND the bitmap cannot be > trusted to tell us which is the valid one, we don't know which is the > most recent, and pick arbitrarily. Also, if you store a backup of V3, > and you don't compress it, and you wipe out the bitmap blocks and need > to use fsck, we don't know what blocks are backup image and what blocks > are the fs. We advise users to never store a V3 backup on V3 without > compressing it. Unfortunately, there are plenty of reasons why you might want to store a filesystem image on disk besides for backup purposes: * Regression tests --- I have some 70+ small filesystem images used for e2fsck's regression test suite. (I am always amazed how many filesystem fsck programs don't have regression test suites.) * Initial ram-disk images * Image files for qemu or user-mode-linux ... and probably many more. None of these are safe to store on a reiserfs3 filesystem if you're worried about fsck being robust after a disk failure. Funny thing. When I tell system administrators who have been around the block more than a few times about this particular "feature" of reiserfs3, they usually very quickly decide that it's time to switch to another filesystem..... - Ted ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 23:06 ` Theodore Ts'o @ 2005-06-25 0:35 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-25 0:35 UTC (permalink / raw) To: Theodore Ts'o Cc: Alan Cox, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List fsck is better in V4 than it is in V3. Users should move from V3 to V4, as V3 is obsolete. I agree on that Ted. I also agree that Ted did a great job with fsck.ext*. V3 was where we learned. There are performance problems in V3 that I could only fix by writing V4. The balancing code for V3 was extremely difficult to modify because it understood all the internals of the item structures, and that is why we based V4 on plugins. V3 had a time where it was really useful, and the time when it was the only metadata journaling filesystem for Linux was its high point (thanks Chris), but its usefulness is leaving us very soon now with V4. In 12 months, after V4 has been pounded on by a few million users, few will want to make new installs onto V3 instead of V4. It would be nice if we could concentrate on speeding that transition instead of flaming each other. I would like to thank the ext* team, especially Ted, for a great filesystem that I used for many years while developing V3, that was crucial to the early success of Linux, and that is still useful to a great many. Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 19:21 ` Hans Reiser 2005-06-24 22:04 ` Olivier Galibert 2005-06-24 23:06 ` Theodore Ts'o @ 2005-06-26 17:20 ` Alan Cox 2005-06-26 17:38 ` Grzegorz Kulewski 2005-06-29 16:44 ` torturing filesystems [was Re: reiser4 plugins] Pavel Machek 2 siblings, 2 replies; 620+ messages in thread From: Alan Cox @ 2005-06-26 17:20 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Gwe, 2005-06-24 at 20:21, Hans Reiser wrote: > Alan, this is FUD. Our V3 fsck was written after everything else was, > for lack of staffing reasons (why write an fsck before you have an FS > worth using). As a result, there was a long period where the fsck code > was unstable. It is reliable now. > > People often think that our tree makes fsck less robust. Actually fsck > can throw the entire internal tree away and rebuild from leaf nodes, and > frankly that makes things pretty robust. I did a series of tests well after resier3 had fsck that consisted of modelling the behaviour of systems under error state. I modelled random bit errors, bit errors at a fixed offset (class ram failure), sector 4 byte slip (known IDE fail case) and sectors going away. Reiserfs didn't handle it anything like as gracefully as ext2. Its a pretty easy experiment to write the code for and the results are interesting. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 17:20 ` Alan Cox @ 2005-06-26 17:38 ` Grzegorz Kulewski 2005-06-29 16:44 ` torturing filesystems [was Re: reiser4 plugins] Pavel Machek 1 sibling, 0 replies; 620+ messages in thread From: Grzegorz Kulewski @ 2005-06-26 17:38 UTC (permalink / raw) To: Alan Cox Cc: Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Sun, 26 Jun 2005, Alan Cox wrote: > On Gwe, 2005-06-24 at 20:21, Hans Reiser wrote: >> Alan, this is FUD. Our V3 fsck was written after everything else was, >> for lack of staffing reasons (why write an fsck before you have an FS >> worth using). As a result, there was a long period where the fsck code >> was unstable. It is reliable now. >> >> People often think that our tree makes fsck less robust. Actually fsck >> can throw the entire internal tree away and rebuild from leaf nodes, and >> frankly that makes things pretty robust. > > I did a series of tests well after resier3 had fsck that consisted of > modelling the behaviour of systems under error state. I modelled random > bit errors, bit errors at a fixed offset (class ram failure), sector 4 > byte slip (known IDE fail case) and sectors going away. > > Reiserfs didn't handle it anything like as gracefully as ext2. Its a > pretty easy experiment to write the code for and the results are > interesting. Maybe but I once checked some other error scenario. I generated (by mistake of course) dm table that lineary connected 3 times the same partition (instead of 3 different partitions). Both Reiser4 and Reiserfs3 gave a lot of errors while trying to use such device. Ext3 did not give single error and was hapily droping my data, I agree that this is not very useful test case for disk problems but it shows that, at least, checks for trouble in Reiser4 are miles before those in Ext2/3. If only Reiser4 could print a note what I done wrong... ;-) Grzegorz Kulewski ^ permalink raw reply [flat|nested] 620+ messages in thread
* torturing filesystems [was Re: reiser4 plugins] 2005-06-26 17:20 ` Alan Cox 2005-06-26 17:38 ` Grzegorz Kulewski @ 2005-06-29 16:44 ` Pavel Machek 1 sibling, 0 replies; 620+ messages in thread From: Pavel Machek @ 2005-06-29 16:44 UTC (permalink / raw) To: Alan Cox Cc: Hans Reiser, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Hi! > > Alan, this is FUD. Our V3 fsck was written after everything else was, > > for lack of staffing reasons (why write an fsck before you have an FS > > worth using). As a result, there was a long period where the fsck code > > was unstable. It is reliable now. > > > > People often think that our tree makes fsck less robust. Actually fsck > > can throw the entire internal tree away and rebuild from leaf nodes, and > > frankly that makes things pretty robust. > > I did a series of tests well after resier3 had fsck that consisted of > modelling the behaviour of systems under error state. I modelled random > bit errors, bit errors at a fixed offset (class ram failure), sector 4 > byte slip (known IDE fail case) and sectors going away. Do you have that code somewhere? Best I could do was overwriting random parts of partition with random data, you seem to have "more interesting" failures... Pavel -- teflon -- maybe it is a trademark, but it should not be. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 11:34 ` Alan Cox 2005-06-24 19:21 ` Hans Reiser @ 2005-06-25 3:14 ` David Masover 2005-06-25 3:18 ` Jeff Garzik ` (2 more replies) 1 sibling, 3 replies; 620+ messages in thread From: David Masover @ 2005-06-25 3:14 UTC (permalink / raw) To: Alan Cox Cc: Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Alan Cox wrote: > On Gwe, 2005-06-24 at 04:17, David Masover wrote: > >>>False argument. So does the pen, so do hinges on doors. Do you still >>>have hinges on your doors - probably. >> >>Indeed. Because there's nothing better -- not because I "like it the >>way it is". > > > I chose hinges carefully - there are better technologies than the > generic hinge and there have been for many many years (plus cool stuff > like magnetic doors). You of course aren't a door geek (nor am I I'd > point out) but a computing one... Right, but even if I was a door geek, changing hinges costs more than changing code. Now, if I were building a new house and I happened to know about other alternatives, I might not still be using hinges. Also, I leave enough doors open enough of the time that it's not a serious impact on my life. Now, OS design is a place which does affect my productivity, so I do care about the design, and while it's hard to change that design, it doesn't actually involve buying stuff. >>I think Hans (or someone) decided that when hardware stops working, it's >>not the job of the FS to compensate, it's the job of lower layers, or >>better, the job of the admin to replace the disk and restore from backups. > > > Most desktop users today don't have backups because there is no credible > backup technology for 500Gb of data. They may have partial backups. Some Bandwidth is getting faster. And I just found a nice site for backups called streamload.com. They don't seem to support rsync, and allow only 100 meg downloads, but unlimited uploads. Few desktop users today really need to backup more than 50 megs of data. > things the fs can't deal with - if the disk goes boom then thats a lower > level concern. Also certain bits like writing to spare blocks on a > problem write are indeed handled drive level nowdays. Right. And putting them in the FS is unneccesary bloat if you've got another mechanism for dealing with it. Anyone with 500 gigs of data can afford to do a little RAID, or at least some burned DVDs. DVDs are cheap nowdays: http://www.newegg.com/Product/Product.asp?Item=N82E16817502002 Ok, you might need two of those. Still, it's not much, unless that's 500 gigs of data which is actually changing pretty rapidly. Most users I know seem to burn CDs or DVDs of static things (like music) and only keep games and office-type files, plus copies of the stuff they've burned, on their hard drives. So, backup for most people I know = not throwing away game install disks + cron job to send < 50 megs of data to Streamload. I agree it's nice to have a more corruption-proof filesystem. Convenient. But not absolutely necessary. >>the issues are fixed, an entirely different crowd of benevolent >>dictators will come around and say that we can't get in because we >>change the VFS. At least some people on this list have said things to >>that effect. > > > There are four important issues I see here > > 1. It must work > 2. It must be clean code that follows the kernel style > 3. It must not break other stuff > 4. It needs a maintainer who won't get bored 12 months later and only > support reiser5 > > #3 is the VFS stuff and getting the VFS locking wrong or unclean is a > *very* big deal because you'll cause corruption to non reiser4 users. It's #2 and #3 that I'm worried about. #4 is a little unfair, and I can verify that #1 is satisfied. And, by "worried about", I mean I'm worried about the attitude of other people. As far as I'm concerned, Reiser4 is mergeable now. But I'm not exactly an authority. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrzMMXgHNmZLgCUhAQLqMQ/+LeGfZlmMt8pog6StIbWj3ZtfSXXnOAtu Os6jMMITPL7QlGfD8ren6nExtS/zxAfua6eQmoc9Lk4qPSFGJsSj8s3APE6p+igr YLHSD6yKMW+sEwz+9d/jLI22R+Ct6x2AUGeYKbmJ4GnYYSeDMsJieeG/OJJscQFG ETjSDWYQM8Ba7YgiJBWfJFYfD9LuM2wE1yUY5zmqVlT1hKSEB6rKEpx+J4MpAj6H u19DCDoluVqTI/4XIFDjpHkwsNfYElCDe6dbdtgeHlDIjcgX8PRu2ZEVAhDjwG3H wlLc1V6lE/qznf4kUgWg3XAc6P2MbJAJd6S7xg5ifSaEzYE7sYwPrXAaPOh+BpNV P01T5eqsSoIbDQb1q1w686EVdQKXSXms2W21IFctHi60FBwfBIfLNwJ8I590MIhG SkhBj/LAnZrFDztf8Z16oBy0zNrVKLQBGkd3FlD8YDv8J7II266yv+aOVEvNp0Sw 5g8RnlxnUqnl1JCo7TRHLe3pknPXms9JISd/wHvjIPuxgOBSVw0nPDde+T/mdlOP 28ef3MDEDwc2cBe3xPLj1nXYjrLvw6zGrSR22IRYDrdZgqPtKjrUEj+0T6pmmHW/ NAH8Ck9zROvadnaB9/st4JJT74axRtwI1HFb4SMfo23vQ6za7RNIHKjDMVsAjtqV fmgsvO2QUo8= =MlUY -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 3:14 ` reiser4 plugins David Masover @ 2005-06-25 3:18 ` Jeff Garzik 2005-06-25 4:31 ` Hans Reiser 2005-06-25 5:26 ` Jesper Krogh 2005-06-26 17:23 ` Alan Cox 2 siblings, 1 reply; 620+ messages in thread From: Jeff Garzik @ 2005-06-25 3:18 UTC (permalink / raw) To: David Masover Cc: Alan Cox, Horst von Brand, Hans Reiser, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List David Masover wrote: > And, by "worried about", I mean I'm worried about the attitude of other > people. As far as I'm concerned, Reiser4 is mergeable now. But I'm not > exactly an authority. Then why not listen to authorities, all of whom are saying "it's not ready yet" Jeff ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 3:18 ` Jeff Garzik @ 2005-06-25 4:31 ` Hans Reiser 2005-06-25 4:43 ` Jeff Garzik ` (2 more replies) 0 siblings, 3 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-25 4:31 UTC (permalink / raw) To: Jeff Garzik Cc: David Masover, Alan Cox, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Jeff Garzik wrote: > Then why not listen to authorities, all of whom are saying "it's not > ready yet" What exactly is not ready Jeff? As I see it, this thread is 95% posturing, and almost no technical reasons for not merging. These "authorities"seem perfectly content with echoing the words of someone who skimmed the code and shot his mouth off without understanding it, and now these "authorities" just echo him because they like him and assume he must be right. Actually, David's understanding of reiser4 is far deeper than those of anyone claiming to be an "authority". He understands what a plugin is. Wow. He does not have to be told that VFS does not currently support a notion of pluginids, so it is not duplicative. Wow. He does not have to be told that plugins are analogous to classes and what VFS currently has is analogous to instances. Wow. He gets it that reiser4 plugins disturb absolutely nothing in the rest of the kernel because we use the standard VFS interface. Wow. Does any of the rest of that echo chamber of authority understand those things? Seems to me that these "authorities" are just as capable of shooting their mouths off before doing their homework as anyone else is..... David has used reiser4. Have you? Maybe you guys should try it. Maybe you should all have less faith in your ability to skim code and understand it instantly. Maybe you guys should actually read the technical parts of this thread rather than the flame parts only? Maybe our community's social traditions are a bit lacking? Maybe saying that everyone else goes through this just means that changing the way we do things is needed even more? Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 4:31 ` Hans Reiser @ 2005-06-25 4:43 ` Jeff Garzik 2005-06-25 6:01 ` Hans Reiser 2005-06-25 4:49 ` David Masover 2005-06-25 14:45 ` Diego Calleja 2 siblings, 1 reply; 620+ messages in thread From: Jeff Garzik @ 2005-06-25 4:43 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Alan Cox, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Hans Reiser wrote: > Jeff Garzik wrote: > > >>Then why not listen to authorities, all of whom are saying "it's not >>ready yet" > > > What exactly is not ready Jeff? As I see it, this thread is 95% > posturing, and almost no technical reasons for not merging. These > "authorities"seem perfectly content with echoing the words of someone > who skimmed the code and shot his mouth off without understanding it, > and now these "authorities" just echo him because they like him and > assume he must be right. I've already listed two rather large technical reasons. When a vendor submits code, they know their [hardware | filesystem] best, but Linux kernel developers know the kernel code and kernel direction best. There needs to be a meeting of the minds. Vendors that stick to "I know best" line of reasoning don't get very far in Linux. Jeff ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 4:43 ` Jeff Garzik @ 2005-06-25 6:01 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-25 6:01 UTC (permalink / raw) To: Jeff Garzik Cc: David Masover, Alan Cox, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Jeff Garzik wrote: > > I've already listed two rather large technical reasons. They are? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 4:31 ` Hans Reiser 2005-06-25 4:43 ` Jeff Garzik @ 2005-06-25 4:49 ` David Masover 2005-06-25 6:15 ` Hans Reiser 2005-06-25 14:45 ` Diego Calleja 2 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-25 4:49 UTC (permalink / raw) To: Hans Reiser Cc: Jeff Garzik, Alan Cox, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hans Reiser wrote: [...] > David has used reiser4. Have you? Maybe you guys should try it. Maybe > you should all have less faith in your ability to skim code and > understand it instantly. Maybe you guys should actually read the > technical parts of this thread rather than the flame parts only? To be fair, I use it, but I haven't read much of it. I mostly read the whitepaper, and then read it again, several times, to make sure I groked it. Then I read the Future Vision paper, once, and completely failed to understand it. Now that I understand Spotlight, I might go back to it. This is why I stay out of the deeply technical stuff, and frequently insist that I don't know what I'm doing and don't work here. > Maybe our community's social traditions are a bit lacking? Maybe saying > that everyone else goes through this just means that changing the way we > do things is needed even more? The social traditions aren't uniform. In fact, if we're referring to all of open source, go hang out on irc.freenode.net#gentoo for a great community, whether or not you like the distro. If you want developers, there's not too many RTFM's and "I was coding bytecode engines when you were in dipers" attitudes in #perl, either. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrziPXgHNmZLgCUhAQKJiA//aIvOMGjwvorsU8YhIhR9jpqITFzR16/y YeXef0o7vX5UC6ws/eP3EQbl46/NbT8mth0l3x0OgXIkTha3edCPMtNUBuJ/ISad nYY17mNPCr5NWzyh5aL7Cw1DYBIVrwQ8dcVg7VbmnbutPyTxKnFxugABExk9MASQ +1n1FVmKRRl4cjte3qERnFN9crPxkLnGWWT5UneGo9tnxKCqd3VBcJQ23PYHsskF dyYmh9/YJaPMskFRK9oQzw9TAM740rwZvOOrQGO84fQkYnrgjrVxis3O6JYbVYzP tiQNC7kSE2HQoMwFratL+tTj2mpx2+Akmhb/XT738tDSNvpSIBhy2eGqXU2p61MF sZiz0kNiSpW2e2kUGYiDKtey9o7bTlQJJ1KmzLtVKqOhPgMpRkyR8PG7X9Sz0P// zLaETf02VHt+1stN9xKum9zYI3Ba0aOt3fhA+1RqBSszV9JdTYVA8/zekGSTYezx neEGJGMkZCRISY2hWQTmHYNVG55ie3cxGgKQdYRxM7wO6AMQID1074UMIqE2aPn5 qjGdtkLXtiLQ33gg7v5uAQeysyQxBZSEBI+uQSH3guXIjOX9XItMaqqygKNXB7ak zsZw8qguDVNHIAyPlHVBVHkCUEZ0hstdfYUU5gyMY+mLP0eq/HgSp5XEReX6RU0U 3KSielBMshs= =ipn3 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 4:49 ` David Masover @ 2005-06-25 6:15 ` Hans Reiser 2005-06-25 6:38 ` Gregory Maxwell 2005-06-25 7:33 ` Bob R. Taylor 0 siblings, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-25 6:15 UTC (permalink / raw) To: David Masover Cc: Jeff Garzik, Alan Cox, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List David Masover wrote: > The social traditions aren't uniform. In fact, if we're referring to > all of open source, go hang out on irc.freenode.net#gentoo for a great > community, whether or not you like the distro. If you want developers, > there's not too many RTFM's and "I was coding bytecode engines when you > were in dipers" attitudes in #perl, either. I would develop for BSD instead if they were anywhere near Linux in marketshare. These attitudes, when the BSD guys had them, were why I chose Linux over BSD to develop for. Now Linux has the market share, and they act like the old BSD guys used to. Sad. Somebody else will come along and challenge Linux someday. I wonder if Apple is a better social environment for developers these days than Linux? It would be fun to work with Steve Jobs, he has such a sense of vision and a delight in new things. He hires good people too; Dominic Giampaolo is really sharp. Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 6:15 ` Hans Reiser @ 2005-06-25 6:38 ` Gregory Maxwell 2005-06-25 6:47 ` David Masover 2005-06-25 7:33 ` Bob R. Taylor 1 sibling, 1 reply; 620+ messages in thread From: Gregory Maxwell @ 2005-06-25 6:38 UTC (permalink / raw) To: Hans Reiser; +Cc: ReiserFS List On 6/25/05, Hans Reiser <reiser@namesys.com> wrote: > I wonder if Apple is a better > social environment for developers these days than Linux? It would be > fun to work with Steve Jobs, he has such a sense of vision and a delight > in new things. He hires good people too; Dominic Giampaolo is really > sharp. And trade freedom for this? In the current situation you can release code even if the powers that be do not agree. Others can choose to integrate and distribute your code, .. they have and they will.. If you were at apple and some authority decided against a feature you and your team would likely not be permitted to develop it, and if developed it would not be distributable. The situation right now is somewhat unfortunate, but it is not unsolvable. Trading away freedom is seldom a good deal. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 6:38 ` Gregory Maxwell @ 2005-06-25 6:47 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-25 6:47 UTC (permalink / raw) To: Gregory Maxwell; +Cc: Hans Reiser, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Gregory Maxwell wrote: > On 6/25/05, Hans Reiser <reiser@namesys.com> wrote: > >>I wonder if Apple is a better >>social environment for developers these days than Linux? It would be >>fun to work with Steve Jobs, he has such a sense of vision and a delight >>in new things. He hires good people too; Dominic Giampaolo is really >>sharp. > > > And trade freedom for this? In the current situation you can release [...] > The situation right now is somewhat unfortunate, but it is not > unsolvable. Trading away freedom is seldom a good deal. The irony is killing me. Namesys is, I'm told, still quite in debt. Apple has money. From a certain point of view, it is a good deal... ...but to have your cake and eat it, there's Google... -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrz+DngHNmZLgCUhAQKd+A//Vif3OzRDdPHuwGVfypfJQ9ptaPUxcv/W qR8GJfGL7FNP3cGD4PJMcCJPU0tnjDVWtSDmBRP+qhWnxPq36LcLF7fIznaGcv1K zkMS7yA8UQrDtKfFwHRbi3EvOhEKsrnkCKPnuZ1SRa3aNQD2sJ89R+0641PbHnpY 2g3G7qyc2lIep59I6Eb+b4X4ZHal5BnWf3nUTFrfCt8N9pmQcWxdeTdhw0i2g+A2 UmB0nG2wL1SOjnsT8M/pFwFdF5Wft+wXrEKD5j6FCzJ/J3TRBJ90iAgWaDb5IK8b 80EcqNSKYKDzqHly0tM/m2NMDP1vwhsOPCPZlLOKoB1YvBbSuOLtpzWCOcpDuFox PuIgdBjyHVuEeY+zdPn2e2UiP5FsYcocT7VJMAwF/JJ+GVdRDci8UDqLtx1iFQ4R ukBgJ+9XbCMGOjzRe/GoCcxmsoHGqkcsbK0vaAhCxejQkwwEwr04ycj7BPXKAVMW nmmKqCw6Y+8FP7oAoZC3wsR9frF+qQbEQvlmbDQ9oKm0EzRM871V0si3imG3gyyF c37Jg1HvgxfYCrvqjkxKzHU3csUvZH9oZDfax1ThvH8D2a3pJuo1oJLdbGseedlh k8S7YxCm6hXA82RSHGV8ljo453S4j5QO42sSjlMihEfb6Ft6OsXF1P7Hqjw93uF7 s69+sVS6Rho= =2aOi -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 6:15 ` Hans Reiser 2005-06-25 6:38 ` Gregory Maxwell @ 2005-06-25 7:33 ` Bob R. Taylor 1 sibling, 0 replies; 620+ messages in thread From: Bob R. Taylor @ 2005-06-25 7:33 UTC (permalink / raw) To: linux-kernel On Fri, 2005-06-24 at 23:15, Hans Reiser wrote: > I would develop for BSD instead if they were anywhere near Linux in > marketshare. These attitudes, when the BSD guys had them, were why I > chose Linux over BSD to develop for. Now Linux has the market share, > and they act like the old BSD guys used to. Sad. Somebody else will > come along and challenge Linux someday. I wonder if Apple is a better > social environment for developers these days than Linux? It would be > fun to work with Steve Jobs, he has such a sense of vision and a delight > in new things. He hires good people too; Dominic Giampaolo is really > sharp. I hate to jump in here but this is just too much! Hans, I remember well the bellyaching you did when you insisted on the Linux developers merging Reiserfs V1 into the kernel. Did you not learn from *that* experience? You have taken *some* criticism here which may *not* have been flames at you until you flamed *first*. You have been told by at least 2 developers here what you must do *first* before consideration of merging V4. As of this writing YOU HAVE NOT ANSWERED! You obviously have not even read all articles posted here. When Jeff Garzik said "I've already listed two rather large technical reasons." You answered "They are?". You just continue your insane bickering, now followed by picking up your marbles and leaving. Grow up. BTW, I could care less whether V4 gets merged or not as I am *not* a developer. Bob -- Bob R. Taylor <brtaylor@sanfelipe.com.mx> ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 4:31 ` Hans Reiser 2005-06-25 4:43 ` Jeff Garzik 2005-06-25 4:49 ` David Masover @ 2005-06-25 14:45 ` Diego Calleja 2005-06-29 2:07 ` Matthew Frost 2 siblings, 1 reply; 620+ messages in thread From: Diego Calleja @ 2005-06-25 14:45 UTC (permalink / raw) To: Hans Reiser; +Cc: jgarzik, ninja, alan, hch, akpm, linux-kernel, reiserfs-list El Fri, 24 Jun 2005 21:31:02 -0700, Hans Reiser <reiser@namesys.com> escribió: > What exactly is not ready Jeff? As I see it, this thread is 95% > posturing, and almost no technical reasons for not merging. These > "authorities"seem perfectly content with echoing the words of someone > who skimmed the code and shot his mouth off without understanding it, > and now these "authorities" just echo him because they like him and > assume he must be right. I'm not one of those "authorities" but I doubt reiser4 will be merged at all with that attitude. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 14:45 ` Diego Calleja @ 2005-06-29 2:07 ` Matthew Frost 0 siblings, 0 replies; 620+ messages in thread From: Matthew Frost @ 2005-06-29 2:07 UTC (permalink / raw) To: Diego Calleja, Hans Reiser Cc: jgarzik, ninja, alan, hch, akpm, linux-kernel, reiserfs-list --- Diego Calleja <diegocg@gmail.com> wrote: > El Fri, 24 Jun 2005 21:31:02 -0700, > Hans Reiser <reiser@namesys.com> escribió: > > > > What exactly is not ready Jeff? As I see it, this thread is 95% > > posturing, and almost no technical reasons for not merging. These > > "authorities"seem perfectly content with echoing the words of someone > > who skimmed the code and shot his mouth off without understanding it, > > and now these "authorities" just echo him because they like him and > > assume he must be right. > > > I'm not one of those "authorities" but I doubt reiser4 will be merged > at all with that attitude. Spot on. Somebody hasn't read his Linus. It seems to me that anytime I hear whining about how unfair the system is, how arbitrary patch rejection is, the flaw is either a lack of social competence, or failure to follow the submission guidelines. Lately the number three reason, "Linus forgot," happens less and less. And linux developers are offering to help Reiser development fix what's wrong! That's above and beyond. If Hans wants to take his ball and go home, let it be understood that it is not because of a failure in the linux process. People that don't want flames, drop out of flame wars and do constructive work. People that want to talk ideology, start flame wars and perpetuate them. If you're having a code argument and you can go four posts without referring to actual code, you're no longer having a code argument. Non-demonstrable merit is not merit. Benchmarks can be necessary -- but are never alone sufficient -- merit to merge to mainline. Hans: consider that enough linux people like your stuff enough to have this argument. Performance has long since ceased to be at issue. Linux is a thing in itself. If Reiser4 is a thing in itself, and not merely a means to a further release point, you will care enough about it to merge it usably and maintainably into the kernel. Should you obsolesce 4 as you have done to 3, someone else can care enough about it, and do something with it. Features are added to Reiser3 because it is a thing in itself now that it is in the kernel. Its users care. Its maintainers, even if they still work for Namesys, care. Listen next time you hear someone talk about "vendor-abandoned projects" continuing to function through community development. Alan has said, "Free software is always late." Free software follows the itch. If Hans wants linux to have Reiser4, Reiser development will work with linux development to become compliant and explain/adapt any additional functionality. Linux does not abjectly need Reiser4 in-tree. Reiser4 is ultimately Hans's scratch for Hans's itch. It will be the more effective and widely usable once it is stable in-tree, but the mainline kernel developers aren't the ones with the itch. Protocols exist in social systems as a means to constructively enlist the help of people outside the problem. The protocols here are well established. This is a patch meritocracy. Failure to comply further increases the lateness of Reiser4 in mainline kernels. Ego involvement is noise. My observations being what they are, may I suggest that this thread die, in favor of more eyes and in-depth current code analysis help in "Re: reiser4 merging action list"? Higher utility function; higher S:N. Flames can always go down with the ship. Frosty Not a programmer, merely literate. No, I don't usually respond to trolls, but this thread, from this man, itches me. It's not worthy of his brilliance, which may be his tragic flaw. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 3:14 ` reiser4 plugins David Masover @ 2005-06-25 5:26 ` Jesper Krogh 2005-06-25 5:26 ` Jesper Krogh 2005-06-26 17:23 ` Alan Cox 2 siblings, 0 replies; 620+ messages in thread From: Jesper Krogh @ 2005-06-25 5:26 UTC (permalink / raw) To: linux-kernel; +Cc: reiserfs-list ["Followup-To:" header set to gmane.linux.kernel.] I gmane.linux.kernel, skrev David Masover: > > Most desktop users today don't have backups because there is no credible > > backup technology for 500Gb of data. They may have partial backups. Some > > Bandwidth is getting faster. And I just found a nice site for backups > called streamload.com. They don't seem to support rsync, and allow only > 100 meg downloads, but unlimited uploads. > > Few desktop users today really need to backup more than 50 megs of data. That gives tedious manual work.. and btw, won't save you if you from loosing stuff from when the backup was made until now. > > things the fs can't deal with - if the disk goes boom then thats a lower > > level concern. Also certain bits like writing to spare blocks on a > > problem write are indeed handled drive level nowdays. > > Right. And putting them in the FS is unneccesary bloat if you've got > another mechanism for dealing with it. Anyone with 500 gigs of data can > afford to do a little RAID, or at least some burned DVDs. > > DVDs are cheap nowdays: > http://www.newegg.com/Product/Product.asp?Item=N82E16817502002 Again lots of manual work.. I actually have a DAT-station.. but I'm not getting it used. People have DVD-burners, but many don't get time to do a backup anyway. A Copy-On-Write feature in the filesystem would save the average dataloss situation todag (for home users). Where they accidentally deletes stuff. > Streamload. Why, when it could be quick and transparent. And Linux is used many places where you cant let data out-of-the-house of where bandwidth "sucks". The waste-space in my diskdrives increases everyday .. and i fill up with a tar-ball of the system every now-and-then, but it would definately be better suited and more effecient (save me more times) done directly in the filesystem. > I agree it's nice to have a more corruption-proof filesystem. > Convenient. But not absolutely necessary. Thats called raid, we have that allready. But raid won't help for and: rm /etc/passwd, a Copy-On-Write filesystem (not-snapshot) would. Used on a mirrored raiddisk, with enough space on the disk, it would actually guard you from allmost anything but getting the computer stolen. Totally unrelated to reiser4 but a feature that would be nice to have in "any" filesystem. Jesper -- ./Jesper Krogh, jesper@krogh.cc ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-25 5:26 ` Jesper Krogh 0 siblings, 0 replies; 620+ messages in thread From: Jesper Krogh @ 2005-06-25 5:26 UTC (permalink / raw) To: reiserfs-list; +Cc: linux-kernel ["Followup-To:" header set to gmane.linux.kernel.] I gmane.linux.kernel, skrev David Masover: > > Most desktop users today don't have backups because there is no credible > > backup technology for 500Gb of data. They may have partial backups. Some > > Bandwidth is getting faster. And I just found a nice site for backups > called streamload.com. They don't seem to support rsync, and allow only > 100 meg downloads, but unlimited uploads. > > Few desktop users today really need to backup more than 50 megs of data. That gives tedious manual work.. and btw, won't save you if you from loosing stuff from when the backup was made until now. > > things the fs can't deal with - if the disk goes boom then thats a lower > > level concern. Also certain bits like writing to spare blocks on a > > problem write are indeed handled drive level nowdays. > > Right. And putting them in the FS is unneccesary bloat if you've got > another mechanism for dealing with it. Anyone with 500 gigs of data can > afford to do a little RAID, or at least some burned DVDs. > > DVDs are cheap nowdays: > http://www.newegg.com/Product/Product.asp?Item=N82E16817502002 Again lots of manual work.. I actually have a DAT-station.. but I'm not getting it used. People have DVD-burners, but many don't get time to do a backup anyway. A Copy-On-Write feature in the filesystem would save the average dataloss situation todag (for home users). Where they accidentally deletes stuff. > Streamload. Why, when it could be quick and transparent. And Linux is used many places where you cant let data out-of-the-house of where bandwidth "sucks". The waste-space in my diskdrives increases everyday .. and i fill up with a tar-ball of the system every now-and-then, but it would definately be better suited and more effecient (save me more times) done directly in the filesystem. > I agree it's nice to have a more corruption-proof filesystem. > Convenient. But not absolutely necessary. Thats called raid, we have that allready. But raid won't help for and: rm /etc/passwd, a Copy-On-Write filesystem (not-snapshot) would. Used on a mirrored raiddisk, with enough space on the disk, it would actually guard you from allmost anything but getting the computer stolen. Totally unrelated to reiser4 but a feature that would be nice to have in "any" filesystem. Jesper -- ./Jesper Krogh, jesper@krogh.cc ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 5:26 ` Jesper Krogh (?) @ 2005-06-25 7:46 ` David Masover -1 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-25 7:46 UTC (permalink / raw) To: Jesper Krogh; +Cc: reiserfs-list, linux-kernel -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jesper Krogh wrote: > ["Followup-To:" header set to gmane.linux.kernel.] > I gmane.linux.kernel, skrev David Masover: > >>>Most desktop users today don't have backups because there is no credible >>>backup technology for 500Gb of data. They may have partial backups. Some >> >> Bandwidth is getting faster. And I just found a nice site for backups >> called streamload.com. They don't seem to support rsync, and allow only >> 100 meg downloads, but unlimited uploads. >> >> Few desktop users today really need to backup more than 50 megs of data. > > > That gives tedious manual work.. and btw, won't save you if you from > loosing stuff from when the backup was made until now. Manual? Try scripting. For me, that's a tar command involving /home, /etc, and about one or two other files, with a few excludes, like /home/shared/video. >>>things the fs can't deal with - if the disk goes boom then thats a lower >>>level concern. Also certain bits like writing to spare blocks on a >>>problem write are indeed handled drive level nowdays. >> >> Right. And putting them in the FS is unneccesary bloat if you've got >> another mechanism for dealing with it. Anyone with 500 gigs of data can >> afford to do a little RAID, or at least some burned DVDs. >> >> DVDs are cheap nowdays: >> http://www.newegg.com/Product/Product.asp?Item=N82E16817502002 > > > Again lots of manual work.. I actually have a DAT-station.. but I'm not > getting it used. People have DVD-burners, but many don't get time to do > a backup anyway. A Copy-On-Write feature in the filesystem would save > the average dataloss situation todag (for home users). Where they > accidentally deletes stuff. A lot of the people I know keep stuff on their DVDs, like movies and music, so they can carry them around. And the rest of it is the 50 megs. >> Streamload. > > > Why, when it could be quick and transparent. And Linux is used many > places where you cant let data out-of-the-house of where bandwidth > "sucks". The waste-space in my diskdrives increases everyday .. and i > fill up with a tar-ball of the system every now-and-then, but it would > definately be better suited and more effecient (save me more times) done > directly in the filesystem. Streamload is quick and transparent for me. I put files on the fileserver, it tars them up and uploads them via Streamload's perl client. >> I agree it's nice to have a more corruption-proof filesystem. >> Convenient. But not absolutely necessary. > > > Thats called raid, we have that allready. But raid won't help for and: > rm /etc/passwd, a Copy-On-Write filesystem (not-snapshot) would. Used on > a mirrored raiddisk, with enough space on the disk, it would actually > guard you from allmost anything but getting the computer stolen. > > Totally unrelated to reiser4 but a feature that would be nice to have in > "any" filesystem. Not totally unrelated. COW has been discussed. I don't remember if the low-level stuff was done, but the main complaint was a lack of a copy system call. And RAID is an argument for Reiser4's attitude that it's not the job of the FS to be corruption-proof. Still, it's far easier to avoid deleting stuff than to avoid disk failure. First priority is to get the stuff OFF the machine. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr0LxXgHNmZLgCUhAQKKBBAAiAicaEKY42s0WctOcLg/m/ciVtphQ1jY rChlThrsRCl4xAE45GgAu4P1ZdEYGwI1574W9z2j8EpbdnghX8tBHFUDSG1/K3+f 8Ud0zyMaI46k+D/IzkXS1ENYDj1PGmPAuVbM2pAa3JW0UOMzvKRsSADewxAW7OdY V9fazSYu6l7Sn64XJxpZmXs9fkElXkqaNu/2N5d6rdH8hMmLhxs8HcsbAaJ7ch87 lz2RMruQ4Keh6H6HTHHvmoYog6XakwkD0pOY0efFZolO/EZFnmIlS9VHRIyb6mls 2xKPABDh9Qq0qQxpATgmXnVI9oh9qgrp4wqQ8nTQfEnhL5fvfBTXzJgfjOJiqXUy vX4K/PP+9wzQNoqhTj4g11JBT8ilemsA4U+gbr82qSvvkOXrHkgGTQe6wgUyo6GZ R8Ui7/jmAvNw+RvfL/p0s0s3e/EimUC7ASvN64z6z77wqNH0uUfUwRmD9SL8TbTf jPdFvcd65F84T4gXphT4Dhwjs4fVjZUq3BqB+zMl3lKJP6pJfc5bqpjvqc2Rg1Yv jfpvk3gihG7YN68IgZV+9IHJG6d5P05gi/YMqu75JDkqTXe2VznOXXQp4d58PhO0 ZKJnvAfr/xKrnuwgDUjgfaHThgquyMiC95hMo4IlSEfUquKPFGY4c9k0VKQHduqc lcZOnfkvar0= =olqO -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 3:14 ` reiser4 plugins David Masover 2005-06-25 3:18 ` Jeff Garzik 2005-06-25 5:26 ` Jesper Krogh @ 2005-06-26 17:23 ` Alan Cox 2 siblings, 0 replies; 620+ messages in thread From: Alan Cox @ 2005-06-26 17:23 UTC (permalink / raw) To: David Masover Cc: Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Sad, 2005-06-25 at 04:14, David Masover wrote: > Right, but even if I was a door geek, changing hinges costs more than > changing code. Now, if I were building a new house and I happened to Probably not, programmers are expensive 8) > DVDs are cheap nowdays: > http://www.newegg.com/Product/Product.asp?Item=N82E16817502002 > Ok, you might need two of those. Still, it's not much, unless that's At least four because the media decay patterns are not well understood. So you need raid DVD too 8) > > 1. It must work > > 2. It must be clean code that follows the kernel style > > 3. It must not break other stuff > > 4. It needs a maintainer who won't get bored 12 months later and only > > support reiser5 > It's #2 and #3 that I'm worried about. #4 is a little unfair, and I can > verify that #1 is satisfied. Please review the 2.5 history of reiserfs3 if you believe so ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 3:17 ` David Masover 2005-06-24 3:34 ` Horst von Brand 2005-06-24 11:34 ` Alan Cox @ 2005-06-24 12:49 ` Olivier Galibert 2005-06-25 2:52 ` David Masover 2005-06-29 16:36 ` Pavel Machek 2005-06-24 19:46 ` Hans Reiser 3 siblings, 2 replies; 620+ messages in thread From: Olivier Galibert @ 2005-06-24 12:49 UTC (permalink / raw) To: David Masover Cc: Alan Cox, Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Thu, Jun 23, 2005 at 10:17:06PM -0500, David Masover wrote: > I was able to recover from bad blocks, though of course no Reiser that I > know of has had bad block relocation built in... But I got all my files > off of it, fortunately. My experience shows that you've been very, very lucky. I hope r4 is better in that regard. If you want to try with r3, take a well-used partition[1] and copy it at block level to another partition or a file. Then zero some random spans of blocks in the copy and reiserfsck --rebuild-tree it. My experience is that you'll usually get the files names and directory tree but their contents will have been scattered all over the place. OG. [1] I suspect a minimum of fragmentation is in order to see the problem ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 12:49 ` Olivier Galibert @ 2005-06-25 2:52 ` David Masover 2005-06-29 16:36 ` Pavel Machek 1 sibling, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-25 2:52 UTC (permalink / raw) To: Olivier Galibert Cc: Alan Cox, Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Olivier Galibert wrote: > On Thu, Jun 23, 2005 at 10:17:06PM -0500, David Masover wrote: > >>I was able to recover from bad blocks, though of course no Reiser that I >>know of has had bad block relocation built in... But I got all my files >>off of it, fortunately. > > > My experience shows that you've been very, very lucky. I hope r4 is > better in that regard. I was on Reiser4. And as Hans is loudly protesting, v3 seems to be stable now, including being somewhat resilient to disk breakage. But I do better backups now. > If you want to try with r3, take a well-used partition[1] and copy it > at block level to another partition or a file. Then zero some random > spans of blocks in the copy and reiserfsck --rebuild-tree it. My > experience is that you'll usually get the files names and directory > tree but their contents will have been scattered all over the place. Sometime I'll try that. I do remember mounting read-only *before* I tried the --rebuild-tree, because I didn't want to give it a chance to screw up more that it already was. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrzG3XgHNmZLgCUhAQKayQ//Z4R/1lKkjaiHfIrIpVXmLrGfcGUuVXLG FeQb+50orlA6pfaHDJ5JkJKKyHnTsnNSvn+VHdxqvz25/ijWDoewe368syXSggOA JkSUSvdQlcKn+dnpT8tPFY8OU3Ve87zb53fAI9J145DRLpXXmWjW5/R1OUzlPnmr +nn2n32UvbSyUj8VAM9ECtJ3+HaxXKm0ASbQLKgLoLmJ5bO/u2xml7rGtB1kdaCr xJIxugAaWnA/sytoK2vIsdnH4QFQqbuqW4bdHa3ziORS1w/DQ08PMFtxu/3ChKVQ 5d0cGtMsZkVIz+k4zE+3RD2QXZVLRvZCTwPqQciitvXCRqz3d9Wc73D+fC8FwLKx t7aARQKprPDA/bgg4SPAoJSVxZMTJDATuJ5f6XrpDB/FVUmHYQfqNLhOD/Vl5IAA 9aZCPjDXMknClydasw/SiRu3DUiOIKYlXXUN1Tnsx4l9lbVJpca25TBem+0Val2P IsruPiJ8L3XztRNwQJOY4RGH1CMobPnwEA4F9PFKqVF+wwJS5sNNqbkoD7XMrOT6 UYn3KC0DixgVCkl75nP6ORs8F9LjIuDtFd/+KNQC+AlcU5MX+BeaJD1QhT5G/1qd GjRcQaN+95jVxLVjkqnb552dQ7+CMt/DzszFDtfKlJepobflVMtX1hoZ0phudAgR PqjDS3tIROk= =lkfJ -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 12:49 ` Olivier Galibert 2005-06-25 2:52 ` David Masover @ 2005-06-29 16:36 ` Pavel Machek 1 sibling, 0 replies; 620+ messages in thread From: Pavel Machek @ 2005-06-29 16:36 UTC (permalink / raw) To: Olivier Galibert, David Masover, Alan Cox, Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Hi! > > I was able to recover from bad blocks, though of course no Reiser that I > > know of has had bad block relocation built in... But I got all my files > > off of it, fortunately. > > My experience shows that you've been very, very lucky. I hope r4 is > better in that regard. > > If you want to try with r3, take a well-used partition[1] and copy it > at block level to another partition or a file. Then zero some random > spans of blocks in the copy and reiserfsck --rebuild-tree it. My > experience is that you'll usually get the files names and directory > tree but their contents will have been scattered all over the place. Actually it would be nice to have a script to automate this, and test various filesystems for robustness :-). I done something similar, and it uncovered few bugs in ext2fsck and vfatfsck; perhaps it is time to try it on other filesystems, too? Pavel #!/bin/bash # # fscktest # # Usage: # Make sure output is logged somewhere # First, run fscktest -p as root # Then you can run fscktest as normal user... # prepare() { SIZE=100000 echo "Creating file..." cat /dev/zero | head -c $[$SIZE*1024] > test echo "Making filesystem..." /sbin/mkfs.$FS test echo "Mounting..." mount test -o loop /mnt || exit "Cant mount" echo "Copying files..." cp -a /bin /mnt cp -a /usr/bin /mnt cp -a /usr/src/linux /mnt echo "Syncing..." sync echo "Unmounting..." umount /mnt echo "Moving..." mv test fsck.okay echo "All done." } FS=ext2 if [ .$1 == .-p ]; then prepare exit fi RUN=0 while true; do RUN=$[$RUN+1] echo "Run #$RUN" echo Preparing... cat fsck.okay > fsck.damaged echo Damaging... dd if=/dev/urandom of=fsck.damaged count=10240 seek=7 conv=notrunc cp fsck.damaged fsck.test echo First check... /sbin/fsck.$FS -fy fsck.damaged RESULT=$? if [ $RESULT != 1 -a $RESULT != 2 -a $RESULT != 0 ]; then echo "Fsck failed in bad way (result = $RESULT)" exit fi echo Second check... /sbin/fsck.$FS -fy fsck.damaged RESULT=$? if [ $RESULT != 0 ]; then echo "Fsck lied about its success (result = $RESULT)" exit fi done -- teflon -- maybe it is a trademark, but it should not be. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 3:17 ` David Masover ` (2 preceding siblings ...) 2005-06-24 12:49 ` Olivier Galibert @ 2005-06-24 19:46 ` Hans Reiser 2005-06-27 20:52 ` Vitaly Fertman 3 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-24 19:46 UTC (permalink / raw) To: David Masover, vitaly Cc: Alan Cox, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List David Masover wrote: > > > I was able to recover from bad blocks, though of course no Reiser that I > know of has had bad block relocation built in... there was a patch somewhere. Vitaly, please comment. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 19:46 ` Hans Reiser @ 2005-06-27 20:52 ` Vitaly Fertman 2005-06-27 21:07 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Vitaly Fertman @ 2005-06-27 20:52 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, vitaly, Alan Cox, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Friday 24 June 2005 23:46, Hans Reiser wrote: > David Masover wrote: > > > > > > > > I was able to recover from bad blocks, though of course no Reiser that I > > know of has had bad block relocation built in... > > there was a patch somewhere. Vitaly, please comment. http://www.namesys.com/bad-block-handling.html describes how reiserfs handles bad blocks. -- Thanks, Vitaly Fertman ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 20:52 ` Vitaly Fertman @ 2005-06-27 21:07 ` David Masover 2005-06-28 8:32 ` Vitaly Fertman 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-27 21:07 UTC (permalink / raw) To: Vitaly Fertman Cc: Hans Reiser, vitaly, Alan Cox, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Vitaly Fertman wrote: > On Friday 24 June 2005 23:46, Hans Reiser wrote: > >>David Masover wrote: >> >> >> >>> >>>I was able to recover from bad blocks, though of course no Reiser that I >>>know of has had bad block relocation built in... >> >>there was a patch somewhere. Vitaly, please comment. > > > http://www.namesys.com/bad-block-handling.html describes > how reiserfs handles bad blocks. Anything like this for v4? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsBqhHgHNmZLgCUhAQLnqQ/8CNGlg/b2rS5TJi6Qzbi5E3eQMDXpU6GO bQZCrZ60dhPGI0z5AqM+kpd4tfMqvF9pW83Pa4xPN9Snvs4RxJt79JutRss5JByF ZrBdwt2Whq0Hmv5MxbIlCKof92thg3AL7HVmuKVgApAy85cOLH3C0CTMS00KjHqn oVDO6gAYlZxJXd/Z+VqCILRTorcTJQXi7NaZy0ptSYPZlna/1JL+0JUOKOAQ8U++ 8xK/ETwMwnWRd0/yZfrjE7Y1hU1bFlM5QAHKoRHnXfXEqXNm5eg+idsfCbrv80uw +QuQy46Xl0puumcB1G6kp+WLboQe+u4fCn85osiWo8OzVcU4Q4hkqCjV6kOo6WH5 VYn/W3kEgdEpgxnkuDEG4uxiootzpUPUVLpVsK2GSqo5pdiNz63IGvt40WYQOnkc I6/YPDTlHdmrFbxpASY05gMdtHYDEkvI7cw8MBeaK35Fr7S5VvCeRkEFI48CZHTw Yz+wljq6v22tPmkdKZByXqIY+8A5hgSf2eBXGG4HBL9ogBFbj+IkueqhRQ9Z023J EL7qYVlO0EDmpxb+8MbDYvugMXk6PlzebZLNu4Zw7S0sbkZFUhdkCQS3bnn22kJ4 UxYfsxy7fTUl5Rbdk+TzPbUX39U/fO9seZi6euEt6Dc3ak4g9lVnnzIo8GkwPBNF 7KRZCgN3wCw= =7bqs -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 21:07 ` David Masover @ 2005-06-28 8:32 ` Vitaly Fertman 2005-06-28 19:14 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Vitaly Fertman @ 2005-06-28 8:32 UTC (permalink / raw) To: reiserfs-list Cc: David Masover, Hans Reiser, Alan Cox, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Tuesday 28 June 2005 01:07, David Masover wrote: > Vitaly Fertman wrote: > > On Friday 24 June 2005 23:46, Hans Reiser wrote: > > > >>David Masover wrote: > >> > >> > >> > >>> > >>>I was able to recover from bad blocks, though of course no Reiser that I > >>>know of has had bad block relocation built in... > >> > >>there was a patch somewhere. Vitaly, please comment. > > > > > > http://www.namesys.com/bad-block-handling.html describes > > how reiserfs handles bad blocks. > > Anything like this for v4? in todo for v4, not implemented yet. -- Thanks, Vitaly Fertman ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 8:32 ` Vitaly Fertman @ 2005-06-28 19:14 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-28 19:14 UTC (permalink / raw) To: Vitaly Fertman Cc: reiserfs-list, Hans Reiser, Alan Cox, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Vitaly Fertman wrote: > On Tuesday 28 June 2005 01:07, David Masover wrote: > >>Vitaly Fertman wrote: >> >>>On Friday 24 June 2005 23:46, Hans Reiser wrote: >>> >>> >>>>David Masover wrote: >>>> >>>> >>>> >>>> >>>>>I was able to recover from bad blocks, though of course no Reiser that I >>>>>know of has had bad block relocation built in... >>>> >>>>there was a patch somewhere. Vitaly, please comment. >>> >>> >>>http://www.namesys.com/bad-block-handling.html describes >>>how reiserfs handles bad blocks. >> >>Anything like this for v4? > > > in todo for v4, not implemented yet. Is it significantly different that there'd be another doc I can read? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsGhjXgHNmZLgCUhAQJaeg/7BeQVRKc0m0AEKiwp6B/N/R6/32JcIvk9 0jZq3A6wYsSxmRSDfId+FVoHApAoZldOtw5CZ7W3TwUJBxzGwFZ7IYM4mQaZ5eRa 7CdAoX9LrG85sY/2M9qypE5cW6BQ/0DJFPDOiree/+6mNomg8uOUJ6FeO93j2JW3 jdPrWqsmYBPSDo4vKha+xoAZpCH/sMLwDDBUIFfC2iz4B4yTxYNfEX+mJ4T2mEls 3YqSiTMkgnFbsVTdUiIjqVJY+jJ9ALhDKdc/hkNNrNir3Iib6aNbg9J9J5cDgNHU zzvNyp6WsCgQjtCvfMojVewKAMu94eg1sXt6ESNz4dKbdPBSdo+lBnATntnMjFCN 7nS9K/kKcTKwSVQjQdyEhbm51C9SXYLxj4CMcru9ZyVtdlyzCLhBda1FvaxWbatJ iGCh8tg43Xkd1N4H+LlTQpJrfbkJBXu4U9oYIuNSQuolMfz6IbJqcfNzV0hm9drp lGLODFSFEWFqHUgPqLymk8/pBW9xP4XYwwWFcY8x0Dw4UxpWRB43EZ8Nydg4PBEk SrLnq1FNSP+QLHZvWwF+8vpWj1jyeiBU2hFpt/p9crU5uy4OWGbKEh8GuEHFgp9P BdKzp4iwTFwgjKR/uM3JnCfN4ct0yyyBZVMNf0N8w2Idu4OkBiS2edg/70A8/OqV U0JVANH5qxc= =sd/Q -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 22:04 ` David Masover 2005-06-23 23:43 ` Alan Cox @ 2005-06-24 1:02 ` Hans Reiser 2005-06-24 2:41 ` Horst von Brand 2 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-24 1:02 UTC (permalink / raw) To: David Masover Cc: Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover wrote: > > > But, there are some things Reiser does better and faster than ext3, even > if you don't count file-as-directory and other toys. There's nothing > ext3 does better than Reiser, unless you count the compatibility with > random bootloaders and low-level tools. In fairness, there are some things that I am aware of. In Reiser4 fsync performance needs optimizing, truly random modifications and anything else that write twice logging is best for needs optimizing, workloads dominated by dirtying more mmap'd pages than can fit into RAM need optimizing. > > >> Not everyone will want > >>to reformat at once, but as the reiser4 code matures and proves itself > >>(even more than it already has), > > > >I for one have seen mainly people with wild claims that it will make > their > >machines much faster, and coming back later asking how they can recover > >their thrashed partitions... > > > You know how many I've had thrashed on Reiser4? Two. The first one was > with a VERY early alpha/beta, and the second one was when I dropped a > laptop and the disk failed. > > And it does make certain things faster. For one thing, "emerge sync" on > Gentoo is twice to four times as fast, and /usr/portage is 75% as big, > as on ReiserFS (3). Wow. You know, Nikita complains about my benchmarks, but what I usually hear from users is that their performance when they use their favorite app is mostly similar to my benchmarks. I would be really happy if our compression code was working before we went in. Half the space and thus twice the speed to disk. Edward could use some help with it. If only we could work on that plus fsync plus mmap dirtying plus random modification optimized journaling rather than these flamefests. Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 22:04 ` David Masover 2005-06-23 23:43 ` Alan Cox 2005-06-24 1:02 ` Hans Reiser @ 2005-06-24 2:41 ` Horst von Brand 2005-06-24 18:42 ` Hubert Chan 2005-06-25 4:10 ` David Masover 2 siblings, 2 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-24 2:41 UTC (permalink / raw) To: David Masover Cc: Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover <ninja@slaphack.com> wrote: > Horst von Brand wrote: > > David Masover <ninja@slaphack.com> wrote: > >>Hans Reiser wrote: > >>>Jeff Garzik wrote: > > [...] > >>>You missed his point. He is saying ext3 code should migrate towards > >>>becoming reiser4 plugins, and reiser4 should be merged now so that the > >>>migration can get started. > >>Sort of. > >> > >>I think ext3 would be nice as a reiser4 plugin. > > What for? It works just fine as it stands, AFAICS. > So does DOS. I'm sorry? > Do you use DOS? Good riddance, no! Not for something like 15 years. > I don't even use DOS to run DOS programs. Haven't seen one in quite some time, to be honest. Oops, sorry, I just lied. Booted it from floppy to run a program to whack the BIOS password on a PC some two weeks ago. Only use in a long time. > "Ain't broke" is the battle cry of stagnation. I see it as the battle cry of those that are looking for /real/ problems to solve. > But, there are some things Reiser does better and faster than ext3, Yes, I've heard it is supposed to be faster on huge directories, and doesn't run out of inodes. And it is more efficient spacewise on small files. But then again, space is extremely cheap today... And again, on a list around here I've seen several cries for help with completely hosed filesystems, all ReiserFS. No solution has ever come forth. > even > if you don't count file-as-directory and other toys. There's nothing > ext3 does better than Reiser, unless you count the compatibility with > random bootloaders and low-level tools. For me, those are quite critical... > >> Not everyone will want > >>to reformat at once, but as the reiser4 code matures and proves itself > >>(even more than it already has), > > I for one have seen mainly people with wild claims that it will make their > > machines much faster, and coming back later asking how they can recover > > their thrashed partitions... > You know how many I've had thrashed on Reiser4? Two. The first one was > with a VERY early alpha/beta, and the second one was when I dropped a > laptop and the disk failed. OK. Know how many I thrashed with ext2/3? I remember 3, could have been as many as 5. One was due to a failed disk, another one because of DMA to a disk causing slow corruption. Another one I believe was due to a odd kernel compiled with a snapshot gcc a long time back, plus power loss at the wrong time. And that is from using ext2/3 since they were in early beta. On several machines at the same time, over years. I'd have to say that there isn't much of a difference. > And it does make certain things faster. For one thing, "emerge sync" on > Gentoo is twice to four times as fast, and /usr/portage is 75% as big, > as on ReiserFS (3). That can't all be due to on-disk format. > >> the ext3 people may find themselves > >>wanting some of the more generic optimizations. > > They'll filch them in due time, don't worry. > Duplication of effort. With plugins, we can optimize the upper layers > of ALL filesystems, regardless of the lower layers, in such a way that > it is optional. Generic optimizations how, if they need VFS support?! > I'm sure it's far easier to write a Reiser storage > plugin than a brand new FS. Comparing apples and oranges tells you what? > Eventually, once competition is only based on storage format, we could > end up with just one format. Just one filesystem! (except for > fat/ntfs/iso/udf/network...) And in the open source world, sometimes a > single product is a good thing. Sorry, I don't think this will come to pass in our lifetime, if ever. There are different requirements, and the way to cater to them is different solutions. That has always been what Linux (as opposed to the propietary and even some open source systems) is all about... > >>But, I don't think that will realistically happen at all. > >>Instead, what will probably happen is that once Reiser4 is in the > >>mainstream kernel, it will become more popular and noticable. Other > >>FSes will take note. ext3 people may decide they want > >>file-as-directory, > > That idea is even much older than Linux itself, and no other (Unix) > > filesystem has implemented it. Ever. Wonder why... > >> and vfat people may decide they want cryptocompress, > > I'm sure they don't, because it is mostly for Windows and assorted devices > > (pendrives, digital cameras, ...) compatibility. > I, for one, would like to use a pendrive and have certain files be > encrypted transparently, only for use on Linux, but others be ready to > transfer to a Windows box. And that would surely break Windows compatibility (because you have to keep the data on what to encrypt one the filesystem itself). Besides, having pgp, or gpg, or crypt, or my own whacky encryption proggie do the job in /userland/, and not shoving into the kernel and so down the next user's throat, is in the end a largeish part of what Unix is all about for me. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 2:41 ` Horst von Brand @ 2005-06-24 18:42 ` Hubert Chan 2005-06-25 4:10 ` David Masover 1 sibling, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-24 18:42 UTC (permalink / raw) To: linux-kernel; +Cc: reiserfs-list On Thu, 23 Jun 2005 22:41:01 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: > David Masover <ninja@slaphack.com> wrote: >> I, for one, would like to use a pendrive and have certain files be >> encrypted transparently, only for use on Linux, but others be ready >> to transfer to a Windows box. > And that would surely break Windows compatibility (because you have to > keep the data on what to encrypt one the filesystem itself). Easily solved, using the same method OS/2 added Extended Attribute support to FAT. Of course, things may break if you try to manipulate the encrypted file under Windows (e.g. deleting those files would leave some garbage blocks behind), but breakage should be minor and relatively easy to fix. > Besides, having pgp, or gpg, or crypt, or my own whacky encryption > proggie do the job in /userland/, and not shoving into the kernel and > so down the next user's throat, is in the end a largeish part of what > Unix is all about for me. Meh. It's nice to have encrypting to be transparent. Unless your editor supports gpg/crypt/etc., you'll need to decrypt the file to a temporary area, edit the file, and then re-encrypt. Much easier to make a mistake and lose data, or leave traces of the cleartext data. Besides, the encryption would be optional, and hence not being shoved down the user's throat. If you don't like it, don't use it. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-24 18:42 ` Hubert Chan 0 siblings, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-24 18:42 UTC (permalink / raw) To: reiserfs-list; +Cc: linux-kernel On Thu, 23 Jun 2005 22:41:01 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: > David Masover <ninja@slaphack.com> wrote: >> I, for one, would like to use a pendrive and have certain files be >> encrypted transparently, only for use on Linux, but others be ready >> to transfer to a Windows box. > And that would surely break Windows compatibility (because you have to > keep the data on what to encrypt one the filesystem itself). Easily solved, using the same method OS/2 added Extended Attribute support to FAT. Of course, things may break if you try to manipulate the encrypted file under Windows (e.g. deleting those files would leave some garbage blocks behind), but breakage should be minor and relatively easy to fix. > Besides, having pgp, or gpg, or crypt, or my own whacky encryption > proggie do the job in /userland/, and not shoving into the kernel and > so down the next user's throat, is in the end a largeish part of what > Unix is all about for me. Meh. It's nice to have encrypting to be transparent. Unless your editor supports gpg/crypt/etc., you'll need to decrypt the file to a temporary area, edit the file, and then re-encrypt. Much easier to make a mistake and lose data, or leave traces of the cleartext data. Besides, the encryption would be optional, and hence not being shoved down the user's throat. If you don't like it, don't use it. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 2:41 ` Horst von Brand 2005-06-24 18:42 ` Hubert Chan @ 2005-06-25 4:10 ` David Masover 2005-06-25 14:20 ` Valdis.Kletnieks 2005-06-26 4:01 ` Horst von Brand 1 sibling, 2 replies; 620+ messages in thread From: David Masover @ 2005-06-25 4:10 UTC (permalink / raw) To: Horst von Brand Cc: Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Horst von Brand wrote: > David Masover <ninja@slaphack.com> wrote: > >>Horst von Brand wrote: >> >>>David Masover <ninja@slaphack.com> wrote: >>> >>>>Hans Reiser wrote: >>>> >>>>>Jeff Garzik wrote: > > >>>[...] > > >>>>>You missed his point. He is saying ext3 code should migrate towards >>>>>becoming reiser4 plugins, and reiser4 should be merged now so that the >>>>>migration can get started. > > >>>>Sort of. >>>> >>>>I think ext3 would be nice as a reiser4 plugin. > > >>>What for? It works just fine as it stands, AFAICS. > > >>So does DOS. > > > I'm sorry? > > >> Do you use DOS? > > > Good riddance, no! Not for something like 15 years. [...] >>"Ain't broke" is the battle cry of stagnation. > > > I see it as the battle cry of those that are looking for /real/ problems to > solve. I'll refer you to my other rant about stagnation and oil. And, listen to yourself. "Good riddance, no, I don't use DOS" but "Ain't broke is the battle cry of those looking for real problems to solve." You can solve real problems in DOS, it's usable, it ain't broke, there are even some decent games (doom) and windowing systems (win3.1 and others) for it. But Linux is better. DOS ain't broke, but Linux is better. So maybe VFS ain't broke, but plugins would be better. I guess we'll only know if we let Reiser4 merge... >>But, there are some things Reiser does better and faster than ext3, > > > Yes, I've heard it is supposed to be faster on huge directories, and > doesn't run out of inodes. And it is more efficient spacewise on small > files. But then again, space is extremely cheap today... Speed isn't. CPU is, but not disk speed. And packing stuff more efficiently, without actually compressing it, will give you some of that speed. Also, space is not so cheap that I won't take 25% more. > And again, on a list around here I've seen several cries for help with > completely hosed filesystems, all ReiserFS. No solution has ever come > forth. I haven't been counting, but I've seen a number of solutions fly around reiserfs-list for people with reiser4 problems. >> even >>if you don't count file-as-directory and other toys. There's nothing >>ext3 does better than Reiser, unless you count the compatibility with >>random bootloaders and low-level tools. > > > For me, those are quite critical... There have been patches... >>>> Not everyone will want >>>>to reformat at once, but as the reiser4 code matures and proves itself >>>>(even more than it already has), > > >>>I for one have seen mainly people with wild claims that it will make their >>>machines much faster, and coming back later asking how they can recover >>>their thrashed partitions... > > >>You know how many I've had thrashed on Reiser4? Two. The first one was >>with a VERY early alpha/beta, and the second one was when I dropped a >>laptop and the disk failed. > > > OK. Know how many I thrashed with ext2/3? I remember 3, could have been as > many as 5. One was due to a failed disk, another one because of DMA to a > disk causing slow corruption. Another one I believe was due to a odd kernel > compiled with a snapshot gcc a long time back, plus power loss at the wrong > time. And that is from using ext2/3 since they were in early beta. On > several machines at the same time, over years. I'd have to say that there > isn't much of a difference. > > >>And it does make certain things faster. For one thing, "emerge sync" on >>Gentoo is twice to four times as fast, and /usr/portage is 75% as big, >>as on ReiserFS (3). > > > That can't all be due to on-disk format. It's not compression. What do you think it is? To be fair, it's gotten a bit slower recently. Wish that repacker was done. I think it's actually gotten just a little slower than it ever was on v3, but v3 was awhile ago for me, and Gentoo has been growing a lot. /usr/portage is probably twice as big now as it was then. If you want, I can do some unscientific benchmarks. >>>> the ext3 people may find themselves >>>>wanting some of the more generic optimizations. > > >>>They'll filch them in due time, don't worry. > > >>Duplication of effort. With plugins, we can optimize the upper layers >>of ALL filesystems, regardless of the lower layers, in such a way that >>it is optional. > > > Generic optimizations how, if they need VFS support?! This was about a hypothetical ext3 format as a reiser4 storage plugin. I'm not sure how this ties into the VFS stuff. >> I'm sure it's far easier to write a Reiser storage >>plugin than a brand new FS. > > > Comparing apples and oranges tells you what? That I'd rather have oranges? A lot of what people like about ext3 is its stability and fairly universally accepted format. A lot of what people like about XFS is its stability and speed, mainly with large files. A lot of what people like about Reiser4 (as it is today) is its speed, with large and especially with small files. Those are broad and somewhat uneducated statements, but I doubt most people care what FS they are using if the stability and performance is what they want. In that case, why have so much duplicated effort between different filesystems -- even between ISO and UDF and Reiser and XFS -- when most of what's really different is the on-disk format and the optimization? So, in this hypothetical situation where ext3 is a reiser4 plugin, suddenly all the ext3 developers are trying to improve the speed and reliability of reiser4, which benefits both ext3 and reiser4, instead of just ext3. >>Eventually, once competition is only based on storage format, we could >>end up with just one format. Just one filesystem! (except for >>fat/ntfs/iso/udf/network...) And in the open source world, sometimes a >>single product is a good thing. > > > Sorry, I don't think this will come to pass in our lifetime, if ever. There > are different requirements, and the way to cater to them is different > solutions. That has always been what Linux (as opposed to the propietary > and even some open source systems) is all about... egrep and fgrep are the same program. A lot of the code is shared, I'm sure. But you change some settings and they suddenly solve entirely different problems. You don't need a wholly different solution if you can adapt an existing one. That's what plugins are all about -- adapting one solution to any number of requirements, so that people can focus on making that one solution better. But, you may be right about it not happening in our lifetime, as I said: >>>>But, I don't think that will realistically happen at all. People are stubborn. [...] >>>> and vfat people may decide they want cryptocompress, > > >>>I'm sure they don't, because it is mostly for Windows and assorted devices >>>(pendrives, digital cameras, ...) compatibility. > > >>I, for one, would like to use a pendrive and have certain files be >>encrypted transparently, only for use on Linux, but others be ready to >>transfer to a Windows box. > > > And that would surely break Windows compatibility (because you have to keep > the data on what to encrypt one the filesystem itself). Besides, having Aside from what someone else already said about this, why not just have support for accessing, say, a .gpg file as transparently decrypted? You don't even need file-as-directory, just create a file called foo which is really the decrypted version of foo.gpg. No need to change the format, just the filesystem. > pgp, or gpg, or crypt, or my own whacky encryption proggie do the job in > /userland/, and not shoving into the kernel and so down the next user's > throat, is in the end a largeish part of what Unix is all about for me. Do you use ipv6? I don't. And it's not shoved down my throat, either, although it's in the kernel. I simply disable it, and you would (I'm sure) disable the crypto stuff. Plus, as someone else said, it's much easier to do $ vim /some/encrypted/file than $ gpg --decrypt /some/encrypted/file > /some/decrypted/file $ vim /some/decrypted/file $ gpg --encrypt /some/decrypted/file > /some/encrypted/file $ shred /some/decrypted/file Not to mention, shred doesn't work on modern filesystems, so you need to either patch vim (and any other program you might want to use), create an actual ramdisk, or deactivate swap and mount a tmpfs. On doing stuff in userland: I am all for moving tons of stuff to userland, but not until someone does a microkernel right, if it's even possible. And even then, I'd probably still use Linux and argue for some stuff in the kernel, because Linux has developers. Developers. Developers. Developers. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrzZOngHNmZLgCUhAQLk/w//dwfRaR5mtrznECpLMtpfNTm/n0EK2gqC ABYClsvcyKPzFtRTYnhZmBmVvAIueJpaReRmrR3QryESoeD6zftrxpuyvgcxxFRA BUIZcXKtqDF7ILmwCgSN0kB3ltD859q0E9ceJz5/lQZJwIJ2JN1huDM9xsOtw4sW 5HZEwSd6C46HArHD5fnnljppCgxV9ozVDfVzWs2GHU9r9cYZ+y9FU/kWIiuw78Ub fSSF7UqXd+QP0PWkkC7bAA9EtePT4Xd875WX7etgUmJRqVbf6erzPNq34b8V4r0r 6o2QFd5j4ZjpUvAeUzlMivSwFl10Yta+I9E3MHkYIm/86pX3GmPY5W24YfkDKsdS Y1odr3HYHt16QIJF3MPO3724PsSax+c9vmF7ANU+2QadtyQFrU0aasKxehTPnp9M PjShkWpWLGCG2YyesM3lq2HzydbljBraFUfzK9LKKoTqBk0jd6kcUrP8869NPo94 /stcyLSa/1gSAsSX+97Gs8CpQZ9JeJp7cMyDdrbR68h0xAg+UGK7tDez5j7VY9kc cSNo+T+W8OBiRPfz7Zg39ltHIeaPT/8THPKRmOLHWXK2duW+RmZBTDPK22Cwbv83 0f9RgRtm1q1FYG1q4gH373V/hjmvZIXGrN+/I5mU6P43iVrLuz/7MIWAGkQyXS3M nOCoZV7WiHY= =XUC1 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 4:10 ` David Masover @ 2005-06-25 14:20 ` Valdis.Kletnieks 2005-06-25 18:33 ` David Masover 2005-06-26 4:01 ` Horst von Brand 1 sibling, 1 reply; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-25 14:20 UTC (permalink / raw) To: David Masover Cc: Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 3628 bytes --] On Fri, 24 Jun 2005 23:10:35 CDT, David Masover said: > But Linux is better. DOS ain't broke, but Linux is better. So maybe > VFS ain't broke, but plugins would be better. I guess we'll only know > if we let Reiser4 merge... No, we'll only know if we merge something that does plugins at the VFS level in a well-designed way. > This was about a hypothetical ext3 format as a reiser4 storage plugin. > I'm not sure how this ties into the VFS stuff. Very poorly. There's only two interpretations of "ext3 as a reiser4 plugin" that make *any* sense. The first is that reiser4 is totally violating the VFS layer boundary, and the second is that reiser4 is trying to be an all-singing all-dancing wankfest. Later on, you say: > A lot of what people like about ext3 is its stability and fairly > universally accepted format. A lot of what people like about XFS is its > stability and speed, mainly with large files. A lot of what people like > about Reiser4 (as it is today) is its speed, with large and especially > with small files. Now *think* for a moment - how does a hypothetical Reiser4 using ext3 format gain any speed advantage with small files, when the speed advantage is based on using a format other than ext3? As I said, either it's violating the VFS boundary, or it's busy wanking. The Reiser4 proponents would be well served to disavow that particular hypothetical example - I have yet to see *anything* that does more damage to the Reiser4 cause. > So, in this hypothetical situation where ext3 is a reiser4 plugin, > suddenly all the ext3 developers are trying to improve the speed and > reliability of reiser4, which benefits both ext3 and reiser4, instead of > just ext3. Or we can do what *should* be done, which is: a) Put the crack pipe down. b) Tell reiser4 to get its grubby little paws off the VFS if it ever intends to have a chance of being merged in mainline. c) Have a *separate* project to improve the speed/reliability/function of the VFS layer, which is the only way that your vision of having the ext3 and reiser developers cooperating will ever happen. Yes, the VFS could probably use an overhaul. But that *will* happen like this: 1) A patch is submitted and passes review to change the VFS. 2) If appropriate, a patch for reiser4 (if it gets merged) is also submitted (possibly by the same people) to be the first user of the new API/functionality. There's a *reason* why we see patch streams that look like: Patch 1/3: Add moby_foo_init function to nautical core. Patch 2/3: Modify white_whale driver to use moby_foo_init Patch 3/3: Modify captain_ahab driver to use moby_foo_init > Aside from what someone else already said about this, why not just have > support for accessing, say, a .gpg file as transparently decrypted? You > don't even need file-as-directory, just create a file called foo which > is really the decrypted version of foo.gpg. No need to change the > format, just the filesystem. I don't think this is what they mean by "Linux gives you enough rope to shoot yourself in the foot with"... > Plus, as someone else said, it's much easier to do > $ vim /some/encrypted/file > than > $ gpg --decrypt /some/encrypted/file > /some/decrypted/file > $ vim /some/decrypted/file > $ gpg --encrypt /some/decrypted/file > /some/encrypted/file > $ shred /some/decrypted/file You've totally failed to understand that the whole *point* of PGP is that 'vim /some/encrypted/file' *isnt* easy to do. A better example might be the various crypto-loop-ish variants or Microsoft's EFS, where the key management model is more tractable to this sort of automation. [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 14:20 ` Valdis.Kletnieks @ 2005-06-25 18:33 ` David Masover 2005-06-25 20:31 ` Valdis.Kletnieks 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-25 18:33 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Fri, 24 Jun 2005 23:10:35 CDT, David Masover said: > > >>But Linux is better. DOS ain't broke, but Linux is better. So maybe >>VFS ain't broke, but plugins would be better. I guess we'll only know >>if we let Reiser4 merge... > > > No, we'll only know if we merge something that does plugins at the VFS > level in a well-designed way. Which will never happen, the way you see it. (see below.) Or maybe you could see this in action if you look at the *current* code, not the hypothetical future code where reiser4 plugins are removed from the FS, which is then merged, then plugins are added to VFS, then back into reiser... (see below) >>This was about a hypothetical ext3 format as a reiser4 storage plugin. >>I'm not sure how this ties into the VFS stuff. > > > Very poorly. There's only two interpretations of "ext3 as a reiser4 plugin" > that make *any* sense. The first is that reiser4 is totally violating the VFS > layer boundary, and the second is that reiser4 is trying to be an all-singing > all-dancing wankfest. Later on, you say: You'll have to be more specific and less insulting. Actually, the way I'd like to see that is if we >>A lot of what people like about ext3 is its stability and fairly >>universally accepted format. A lot of what people like about XFS is its >>stability and speed, mainly with large files. A lot of what people like >>about Reiser4 (as it is today) is its speed, with large and especially >>with small files. > > > Now *think* for a moment - how does a hypothetical Reiser4 using ext3 format > gain any speed advantage with small files, when the speed advantage is based > on using a format other than ext3? It doesn't. It gains in other ways. Transparent cryptocompress, oddball other plugins, not to mention some speed optimizations that happen in RAM. If you do a ton of work with a dataset that stays in RAM, Reiser probably performs as well or better than a ramdisk, because changes that get overwritten while still in RAM never actually touch the disk. Reiser also doesn't fragment as quickly as ext3, and I don't think that has anything to do with its format. > The Reiser4 proponents would be well served to disavow that particular > hypothetical example - I have yet to see *anything* that does more damage > to the Reiser4 cause. It is my example. I don't speak for Namesys. I don't work at Namesys. If it does "damage", that is only in the eyes of those not paying attention. >>So, in this hypothetical situation where ext3 is a reiser4 plugin, >>suddenly all the ext3 developers are trying to improve the speed and >>reliability of reiser4, which benefits both ext3 and reiser4, instead of >>just ext3. > > > Or we can do what *should* be done, which is: > > a) Put the crack pipe down. That was unneccesary. Come back when you can debate technical reasons, not imaginary personal ones. > b) Tell reiser4 to get its grubby little paws off the VFS if it ever intends > to have a chance of being merged in mainline. You are saying Reiser isn't in because it shouldn't touch VFS. Every single other person in this thread says Reiser isn't in because it *should* touch VFS. > c) Have a *separate* project to improve the speed/reliability/function of > the VFS layer, which is the only way that your vision of having the ext3 and > reiser developers cooperating will ever happen. Why does it have to be a separate project if it is already *done* as part of Reiser4? Or is the name Reiser just cursed that way? > Yes, the VFS could probably use an overhaul. But that *will* happen like this: > > 1) A patch is submitted and passes review to change the VFS. > 2) If appropriate, a patch for reiser4 (if it gets merged) is also submitted > (possibly by the same people) to be the first user of the new API/functionality. The FS that gets merged ahead of time without plugins would no longer be Reiser4. Go read the whitepaper, or tell me why I'm wrong, but even symlinks are implemented as plugins. > There's a *reason* why we see patch streams that look like: > > Patch 1/3: Add moby_foo_init function to nautical core. > Patch 2/3: Modify white_whale driver to use moby_foo_init > Patch 3/3: Modify captain_ahab driver to use moby_foo_init I'm not arguing that at all. But if you've got an entirely new driver, why not do: Patch 1/2: Add white_whale driver, which also adds moby_foo_init to nautical core. Patch 2/2: Modify captain_ahab driver to use moby_foo_init >>Plus, as someone else said, it's much easier to do >>$ vim /some/encrypted/file >>than >>$ gpg --decrypt /some/encrypted/file > /some/decrypted/file >>$ vim /some/decrypted/file >>$ gpg --encrypt /some/decrypted/file > /some/encrypted/file >>$ shred /some/decrypted/file > > > You've totally failed to understand that the whole *point* of PGP is that 'vim > /some/encrypted/file' *isnt* easy to do. A better example might be the various > crypto-loop-ish variants or Microsoft's EFS, where the key management model is > more tractable to this sort of automation. Actually, plugins are just as easy or easier than crypto-loop or dm-crypt. And why shouldn't my crypto be easy? Most users are insecure in all kinds of ways because of that attitude -- security is HARD, so I won't do it. If security is transparent, just enter a password and go, then more people would do it. If more people are secure in more ways, there are less DDOS attacks and the world is a better place. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr2jd3gHNmZLgCUhAQIeihAAgifgKxYw2FECNKLsspQ7sMOmBaT1jcev 5CJdDCiDjiMEMMcsh1h749tZX38YTjqL/w8VQQ18ys/2QqRERbjNEn/lGWLjO4Hs UwFFv/usVMv3RmRModH62P2j7QAxWhjTcfWYeIexdh20whMJLJA+BpIhnoKZ6Xjq WQMbdc9X/NW/lS09zkthWUju8JXtjTjD5rsMF3yQ47Scijec2i67ATsnzNxdXB5/ fZmjOfWKkahHh8qdO/IXX/6nvyEEzmVThr1EskCjUwoRYUSfb9bLQMkntjvEWSDh mDIBbwAUg88k3dM947uS55iV3QGoVkgDRvF0cpWxeaOEP+FcKmF9gPiaW0wm5jYw n60tw7P48FVchBrAPZTwvq5po78Evj3bB09NmS3i68in56rhRI3hTB7KNR/vIbS1 9dKvYuilV0C+thGD+sfE81qqFyP/O66sjvpxRi/YhjL8hU8N3x2f5tTIkKli7Y2C bZSxCiLiDRzKYAgCq0Rr+mrn/TZb5Ld5W8Ys062qF5l00M0I6cDO0sPiH72RexM7 PMM5tWNBDqVacUWxNYReULC5O7Q6c8FYTWcZz/zC1boLTeGpvlI8uVkbxP6K/grY j1u7vHbjgAqsztYhA8qmw3nl5vHtXCGHCRFQJ36g14Qt3atFcMgvfE5yphWyC1P9 hYZm8EWvMqc= =GvIJ -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 18:33 ` David Masover @ 2005-06-25 20:31 ` Valdis.Kletnieks 2005-06-25 20:52 ` Hans Reiser ` (2 more replies) 0 siblings, 3 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-25 20:31 UTC (permalink / raw) To: David Masover Cc: Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 4924 bytes --] On Sat, 25 Jun 2005 13:33:27 CDT, David Masover said: > > Now *think* for a moment - how does a hypothetical Reiser4 using ext3 format > > gain any speed advantage with small files, when the speed advantage is based > > on using a format other than ext3? > happen in RAM. If you do a ton of work with a dataset that stays in > RAM, Reiser probably performs as well or better than a ramdisk, because > changes that get overwritten while still in RAM never actually touch the > disk. At that point, since the actual buffer management is being done at the VFS level (see fs/buffer.c and friends) what you're really comparing is the speed of journalling metadata - at which point you need to be *very* careful to specify exactly what configuration you're talking about. If you don't believe me, investigate why mounting a filesystem with 'noatime,nodiratime' can make a dramatic difference totally independent of the underlying filesystem, but the actual amount of gain is dependent on format (hint - how far do the heads have to move to record 3 atime updates against 3 random inodes on an ext2, an ext3, and a VFAT filesystem, assuming no other disk activity), and why ext3 has 3 different modes data=ordered/writeback/journal. > Reiser also doesn't fragment as quickly as ext3, and I don't > think that has anything to do with its format. Care to explain why it's not format-dependent? > > b) Tell reiser4 to get its grubby little paws off the VFS if it ever intends > > to have a chance of being merged in mainline. > > You are saying Reiser isn't in because it shouldn't touch VFS. Every > single other person in this thread says Reiser isn't in because it > *should* touch VFS. Hmm.. let's see.. I said Reiser isn't in because it shouldn't be screwing with the VFS, and said stuff should be done separate from the Reiser4 filesystem. Everybody else said that to achieve all the goals that Hans wants will require changes to the VFS, and the way reiser4 gets there isn't acceptable. Seems like myself and everybody else are saying this needs to be factored into 2 pieces, a FS piece and a VFS piece, and moved forward separately. > > c) Have a *separate* project to improve the speed/reliability/function of > > the VFS layer, which is the only way that your vision of having the ext3 and > > reiser developers cooperating will ever happen. > > Why does it have to be a separate project if it is already *done* as > part of Reiser4? Or is the name Reiser just cursed that way? Because it's done *as a part of reiser4*, and not as a separately reviewed change to the VFS. > The FS that gets merged ahead of time without plugins would no longer be > Reiser4. Go read the whitepaper, or tell me why I'm wrong, but even > symlinks are implemented as plugins. Which is another way of saying Reiser4 can't be merged in its present form. > I'm not arguing that at all. But if you've got an entirely new driver, > why not do: > > Patch 1/2: Add white_whale driver, which also adds moby_foo_init to > nautical core. You don't do this because the rule is "one patch, one logical change", and "which also" implies more than one change. > Actually, plugins are just as easy or easier than crypto-loop or > dm-crypt. And why shouldn't my crypto be easy? Most users are insecure > in all kinds of ways because of that attitude -- security is HARD, so I There's a vast distinction between "easy for implementors" and "easy for users". Jaari Russo's loop-aes stuff does a wonderful job of being "easy for users" - just say "mount", answer the passphrase, and you're good to go. The underlying arguing about the crypto involved is complicated enough keep professional crypto jocks busy for years (is the watermark attack Jaari is concerned about a real threat? You tell me. ;) Meanwhile, PGP was designed to be used in an environment where you could do this: "Today's secret plans are AES256 encrypted. The key is the next key in your one-time-pad book, XOR'ed with your 128-bit secret key - do it in your head". (And yes, you can easily memorize a 16-digit hex number and learn to do an XOR with another 16-digit number, if failing to do so means you could end up dead). This is inconvenient for the user, but intractable for an attacker to create a scenario where they can just 'vi /each/decrypted/file' ;) > won't do it. If security is transparent, just enter a password and go, > then more people would do it. "Just enter a password and go, then more people would do it". Two words: "phishing e-mail". Why does phishing work at all? Because it's simple for the user, and the user isn't aware of the totally busticated underlying security model of SMTP (namely, there isn't one) and the mostly busticated security model of most browsers (the misleading concept that if an SSL site is identified by the SSL cert, that this implies the site can be trusted, and other similar misfeatures). [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 20:31 ` Valdis.Kletnieks @ 2005-06-25 20:52 ` Hans Reiser 2005-06-26 4:59 ` Lincoln Dale 2005-06-26 0:24 ` Hubert Chan 2005-06-26 3:14 ` David Masover 2 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-25 20:52 UTC (permalink / raw) To: Valdis.Kletnieks Cc: David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Valdis.Kletnieks@vt.edu wrote: > Hmm.. let's see.. I said Reiser isn't in because it shouldn't be > screwing with > >the VFS, and said stuff should be done separate from the Reiser4 filesystem. > > We don't touch a line of VFS code. We look like a normal fs at the interface. Shall we send in a file labeled reiservfs.c containing the plugin layer? That will really test whether the Reiser name is cursed that way, yes? There has been no response to the technical email asking for what exactly it is that is duplicative, and what exactly it is that ought to be changed in how the code works, as opposed to what file the code is placed in, or who is considered its maintainer. There has been no response to the questions about whether the difference between class and instance makes our layer non-duplicative. Perhaps no response was possible? Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 20:52 ` Hans Reiser @ 2005-06-26 4:59 ` Lincoln Dale 2005-06-26 5:07 ` Gregory Maxwell 2005-06-26 5:18 ` David Masover 0 siblings, 2 replies; 620+ messages in thread From: Lincoln Dale @ 2005-06-26 4:59 UTC (permalink / raw) To: Hans Reiser Cc: Valdis.Kletnieks, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hans Reiser wrote: >There has been no response to the technical email asking for what >exactly it is that is duplicative, and what exactly it is that ought to >be changed in how the code works, as opposed to what file the code is >placed in, or who is considered its maintainer. There has been no >response to the questions about whether the difference between class and >instance makes our layer non-duplicative. > >Perhaps no response was possible? > > Hans, the l-k community have asked YOU may times. any lack of response isn't because of the kernel cabal .. its because YOU are refusing to entertain any notion that what Reiser4 has implemented is unpalatable to the kernel community. you can threaten all you want to take your code and move it to BSD. or fork the kernel. whatever. but if you want to get your work into the mainline kernel, you need to provide answers to the question that keeps being asked of you - and which you patently keep ignoring time & time again. in short, as per Message-ID: <42BBC710.8010906@cisco.com>: posting to l-k on "why Reiser4 cannot use VFS infrastructure for [crypto,compression,blahblah] plugins" - ideally, for each plugin. or again, in Message-ID: <42BBB1FA.7070400@cisco.com>: [..] but instead just understand that this is the framework that you have to work in to get it into the mainline kernel. if you don't want to go down that path, you're free to do so. its open source, you can keep your own linux-kernel fork. but if you want to get your code into mainline, i don't think its so much a case of l-k folks telling you how to make your code work under VFS. the onus is on you to say WHY your code and plugins could never be put into VFS. or further back in Message-ID: <42BB7083.2070107@cisco.com> you know that VFS is the mechanism in Linux. you know (i hope..) how it works. it isn't so hard to see how many of the Reiser4 "plug-ins" could be tied into VFS calls. OR, if they cannot TODAY, propose how VFS _COULD_ be made to do this. NB. it doesn't matter what David thinks. this is linux-kernel, not linux-users. cheers, lincoln. > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 4:59 ` Lincoln Dale @ 2005-06-26 5:07 ` Gregory Maxwell 2005-06-26 7:16 ` Lincoln Dale 2005-06-26 5:18 ` David Masover 1 sibling, 1 reply; 620+ messages in thread From: Gregory Maxwell @ 2005-06-26 5:07 UTC (permalink / raw) To: Lincoln Dale Cc: Hans Reiser, Valdis.Kletnieks, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On 6/26/05, Lincoln Dale <ltd@cisco.com> wrote: > the l-k community have asked YOU may times. any lack of response isn't > because of the kernel cabal .. its because YOU are refusing to entertain > any notion that what Reiser4 has implemented is unpalatable to the > kernel community. A lot of this is based on misconceptions, for example in recent times reiser4 is faulted for layering violations.. But it doesn't have them, it neither duplicates nor modifies the VFS. It has also been requested that reiser4 be changed to move some of it's operations above the VFS... not only would that not make sense for the currently provided functions, but merging was put off previously because of changes to the VFS.... now that it doesn't change the VFS we are asking hans to push it off until it does?? It's a filesysem for gods sake. Hans and his team have worked hard to minimize its impact and they are still willing to accept more guidance, even if their patience has started to run a little thin. The acceptance of reiser4 into the mainline shouldn't be any larger deal than any other filesystem, but yet it is... ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 5:07 ` Gregory Maxwell @ 2005-06-26 7:16 ` Lincoln Dale 2005-06-26 7:48 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Lincoln Dale @ 2005-06-26 7:16 UTC (permalink / raw) To: Gregory Maxwell Cc: Hans Reiser, Valdis.Kletnieks, David Masover, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Gregory Maxwell wrote: >On 6/26/05, Lincoln Dale <ltd@cisco.com> wrote: > > >>the l-k community have asked YOU may times. any lack of response isn't >>because of the kernel cabal .. its because YOU are refusing to entertain >>any notion that what Reiser4 has implemented is unpalatable to the >>kernel community. >> >> > >A lot of this is based on misconceptions, for example in recent times >reiser4 is faulted for layering violations.. But it doesn't have them, >it neither duplicates nor modifies the VFS. > >It has also been requested that reiser4 be changed to move some of >it's operations above the VFS... not only would that not make sense >for the currently provided functions, but merging was put off >previously because of changes to the VFS.... now that it doesn't >change the VFS we are asking hans to push it off until it does?? > > <sigh> it has NEVER been a case of Reiser4 not being merged because "it required changes to VFS". the whole point of VFS is to provide a standard API for data to/from individual filesystems. over the course of history, VFS itself hasn't been a static thing - it has had to adapt and change as a result of the needs of filesystems. but it hasn't ever been a case of individual filesystems doing 'proprietary' things (i.e. there isn't a sys_ext3() system call) - where it has made sense to do things at a VFS layer, VFS itself has been adapted to handle those things. a semi-recent example of this is extended attributes. it is with some irony that on my desktop i make use of the excellent open-source desktop search tool 'beagle'. it, however, uses extended attributes for storing things - and Reiser4's EAs are incompatible with the "standard" EAs such that Reiser4 is incompatible with beagle. this is the WHOLE point of standardization .. i don't think its that Reiser4's EAs offer any more or less capabilities than standard EAs - BUT they haven't used the standard mechanisms available for implementing them, such for Beagle to work on Reiser4, there now needs to be logic added to Beagle to do so. lets take this a step further. what about compression? do we accept that each filesystem can implement its own proprietary compression via its own API - and now we need individual user-space tools to understand each of these APIs? how about encryption? ... and so-on. suddenly every user app out there needs to have specialized knowledge of each type of filesystem. Hans should be applauded for the 'plug-in' concept and showing how it can be used. however, from an implementation stand-point, it really shouldn't come as any great surprise that numerous kernel developers are pushing back saying 'layering violation' and "why can't this be done at the VFS layer". none of this is rocket-science. its just plain common sense. >It's a filesysem for gods sake. Hans and his team have worked hard to >minimize its impact and they are still willing to accept more >guidance, > i don't see any acceptance at this point. simply lots of hot air that smells like marketing & PR. cheers, lincoln. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 7:16 ` Lincoln Dale @ 2005-06-26 7:48 ` David Masover 2005-06-26 8:26 ` Lincoln Dale ` (2 more replies) 0 siblings, 3 replies; 620+ messages in thread From: David Masover @ 2005-06-26 7:48 UTC (permalink / raw) To: Lincoln Dale Cc: Gregory Maxwell, Hans Reiser, Valdis.Kletnieks, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Lincoln Dale wrote: [...] > this is the WHOLE point of standardization .. i don't think its that > Reiser4's EAs offer any more or less capabilities than standard EAs - They do. Reiser4's EAs can look like any other object -- files, folders, symlinks, whatever. This is important, especially for transparency. For one thing, can I access a Beagle search as a folder? > BUT they haven't used the standard mechanisms available for implementing > them, such for Beagle to work on Reiser4, there now needs to be logic > added to Beagle to do so. Well, ideally, I'd like to see people stop bickering, come up with something better than sys_reiser4, add an emulation layer for xattrs and mark them obsolete. But I don't speak for Namesys. > lets take this a step further. what about compression? do we accept > that each filesystem can implement its own proprietary compression via > its own API - and now we need individual user-space tools to understand No, that's the beauty of these "EAs" in Reiser4. The API is standard write(2) commands. sys_reiser4 supposedly implements an interface to make this scale better, but otherwise have the same semantics. And who said anything about proprietary compression? I think we were planning on the kernel's zlib, though we might have been planning to make it a bit more seekable... > each of these APIs? So, the API becomes something like: cat crypto/inflated/foo # transparently decompressed cat crypto/raw/foo.gz # raw, gzip-compressed Another possibility, if you like file-as-a-directory: cat foo.gz # raw cat foo.gz/inflated # decompressed One could easily imagine things like these two potentially equivalent commands: cp foo bar.zip/ zip bar foo The whole point is to have less userland tools, not more. I'm not saying we move zip into the kernel, just that the user now has one less command to remember. But, back to reality. file-as-directory probably won't happen, at least not for awhile, so imagine more along the lines of my first example. > how about encryption? About the same, only now you have a key file that you write to in order to unlock the decrypted files. > ... and so-on. > suddenly every user app out there needs to have specialized knowledge of > each type of filesystem. Not really. More like, every app that cares to has generalized knowledge, if that. > none of this is rocket-science. its just plain common sense. I could say the same of my stuff, but lots of people seem to disagree with me, or at least fail to see it. I guess I can say the same of your comments. >> It's a filesysem for gods sake. Hans and his team have worked hard to >> minimize its impact and they are still willing to accept more >> guidance, >> > i don't see any acceptance at this point. simply lots of hot air that > smells like marketing & PR. They do keep asking for specifically what they need to do to put this stuff in VFS. Or am I wrong? Or maybe it should be obvious to them? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr5dtngHNmZLgCUhAQIBGxAAiLK6EyHnLRhEA+rUIDCwacM4K89wlE7X +dcw3xv3Pc9tZZqVVAd7Y27whEzjmNOwfGkvPkzPk/ATQditnyt+7xHcuXpqORNU j7zHc5zS8MGDRU8Re4MXTO6jCXDgtTwQHjcdg4i8KYWLMPT7LpO+DHY/mZyQEgpD kZGE4WJePA+aNlHAzySW9u/atnwp5hSRvmfuF4zzN8ng5tf8SSMvbfoCyjYSue8l N6jvcGnt+yItmbHVaij0IdHUw1/9/u6b3Q0Ut39NBk8fUKXJcASHmKtjwLTAoWW+ hiYVhLdZQGkWo2d6XdzdNY2OgE3kWVnLBqrOuTo7zCjMojvWIrEGE/x3Yh/E6Hs8 cAPVRebG5yUQBxJk1lcDeJOBozutIpCVyTzwBnKU1nz3KArqanU51oT++3cTjVha 1tBnaLS4RLcdy8UD1ewS+VHj61VSlcBjv2abCrYw4DC0anUFrYUSciNjx3tdYJRx o7l/pEn7UYPpGaXgHyBdVDIRlRNdOoTRZp3aIY2Z2v6/jyi3TeufMUjGtpuQHl2k BuYm7tV4l1Ec/QZLM+PbAyVU9qqlz9BuHlI1U7z1p3gYeAzz0guAeDHfi1l95sUn l+bFCOfmXi3qRAxVZyidczqeOtGtCed3nIUH+1z+siuFzH3jecjzUpGWKcGxzVlc jkUS+tlihfg= =C4UP -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 7:48 ` David Masover @ 2005-06-26 8:26 ` Lincoln Dale 2005-06-26 9:35 ` David Masover 2005-06-26 18:16 ` Valdis.Kletnieks 2005-06-29 19:32 ` Thomas Rösner 2 siblings, 1 reply; 620+ messages in thread From: Lincoln Dale @ 2005-06-26 8:26 UTC (permalink / raw) To: David Masover Cc: Gregory Maxwell, Hans Reiser, Valdis.Kletnieks, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >Lincoln Dale wrote: > >[...] > > > >>this is the WHOLE point of standardization .. i don't think its that >>Reiser4's EAs offer any more or less capabilities than standard EAs - >> >> > >They do. Reiser4's EAs can look like any other object -- files, >folders, symlinks, whatever. This is important, especially for >transparency. > > it was accepted not so long ago that 'file-as-directory' and 'EA' are two different things, predominantly because existing tools and apps won't necessarily "do the right thing<tm>". this has been discussed to death previously. oh what a surprise. http://lwn.net/Articles/100271/ cheers, lincoln. cheers, lincoln. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 8:26 ` Lincoln Dale @ 2005-06-26 9:35 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-26 9:35 UTC (permalink / raw) To: Lincoln Dale Cc: Gregory Maxwell, Hans Reiser, Valdis.Kletnieks, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Lincoln Dale wrote: > > > David Masover wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Lincoln Dale wrote: >> >> [...] >> >> >> >>> this is the WHOLE point of standardization .. i don't think its that >>> Reiser4's EAs offer any more or less capabilities than standard EAs - >>> >> >> >> They do. Reiser4's EAs can look like any other object -- files, >> folders, symlinks, whatever. This is important, especially for >> transparency. >> >> > it was accepted not so long ago that 'file-as-directory' and 'EA' are > two different things, predominantly because existing tools and apps > won't necessarily "do the right thing<tm>". They are different, but not quite -- what's the word -- discrete? Rather, file-as-directory is one logical conclusion of EA, and one possible representation of EA. Anyway, Reiser's EAs currently (I think) could appear as files, but not necessarily the file-as-directory madness we were talking about then. They could also be accessed by openat. And they can also be hierarchical, which I would think makes them cleaner and faster. > this has been discussed to death previously. oh what a surprise. > http://lwn.net/Articles/100271/ I was there. That was fun to watch, actually, once it got over my head. The three points he lists are only relevent if you try to do file-as-directory. Right now, I like the idea of file attributes as showing up in separate magic directories, or just creating magic directories first and filling them with files. Symlinks could help clean up messiness, and allow you to do things like encrypt /etc/passwd - -- symlink to /etc/crypto/passwd/decrypted or something. Yes, I know about shadow, and there are still some paranoid reasons to encrypt passwd. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr521XgHNmZLgCUhAQKSSA/9HK8JiG6wW40jQDUXsyKPz+l9UvoaNuLs hqIXo3sdQj1CWAuwa1BKY16w91tJBN3uX75jLgwR36ix4A1jXQ5v1sRjvfkLKR/7 3a51v9UsRAhAisiqWFGWYTrbXgAh+S5B51xHuXv3qTs/wqC8sxuJtAHwuCldPaBr cR1WzPtGvgd+ESqx5avllZIfNCy9tcH3P5fUsaIFCCm30E6u9PVO6xBOHylFWZKS Pywv+wGUTxbgCFSmLG07/zhwVF94fAHPIWXQXQGmhPrGf3Wtt7VTMiRpkpyONyso eoY1hFiwyh2YMrIPxdL0Uo+mcDvErWFFZyTcCGqIMp6x+QSe0Ww9Ie90afZPvvS0 Q4DmdST2JEHEinal5MCiqr3S83wanv3+h9ywTCEkTcl3mEK1iwtc4dwmvRNVHfkx f34CAxM1rBfu++kd3xgL+Kb/ao4LOCDhVL3XY6pNDglX6Y+Kk5dRSJjOm4kAU2fB j66uGOUZOiCCzSMLvVrCAPnNXmUavLKUXDKXKL2gTCYM6TL+7RkAXEMrqu5YxdZM ihIbfmGc7FzItbQCDZhhozG51IMkLCotU9U9CaotnhfUaosibmwEnmeY8FWY75ba MOs+VH3UJUZObRBwmBHSX4pwg5sDhSjyqPR05M26A5xz8nGh4kukD/E6QZYCB9EP lVdDus4iJUM= =hsOJ -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 7:48 ` David Masover 2005-06-26 8:26 ` Lincoln Dale @ 2005-06-26 18:16 ` Valdis.Kletnieks 2005-06-26 19:58 ` David Masover 2005-06-29 19:32 ` Thomas Rösner 2 siblings, 1 reply; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-26 18:16 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 3314 bytes --] On Sun, 26 Jun 2005 02:48:06 CDT, David Masover said: > Lincoln Dale wrote: > > this is the WHOLE point of standardization .. i don't think its that > > Reiser4's EAs offer any more or less capabilities than standard EAs - > > They do. Reiser4's EAs can look like any other object -- files, > folders, symlinks, whatever. This is important, especially for > transparency. No, you want them to look like the same objects that {get|set}xattr() manage currently. You don't want programs to have to guess what an EA looks like this week, with this user's combination of plugins that's different from everybody else's. > > lets take this a step further. what about compression? do we accept > > that each filesystem can implement its own proprietary compression via > > its own API - and now we need individual user-space tools to understand > > No, that's the beauty of these "EAs" in Reiser4. The API is standard > write(2) commands. sys_reiser4 supposedly implements an interface to > make this scale better, but otherwise have the same semantics. And who > said anything about proprietary compression? I think we were planning > on the kernel's zlib, though we might have been planning to make it a > bit more seekable... > > > each of these APIs? > > So, the API becomes something like: > > cat crypto/inflated/foo # transparently decompressed > cat crypto/raw/foo.gz # raw, gzip-compressed And 'cat crypto/raw/foo' or 'crypto/inflated/foo.gz' gets you what, exactly? Now throw some .bz2 and .zip files into the mix... ;) > Another possibility, if you like file-as-a-directory: > > cat foo.gz # raw > cat foo.gz/inflated # decompressed > > One could easily imagine things like these two potentially equivalent > commands: > > cp foo bar.zip/ > zip bar foo Unless of course the user had done 'mkdir sorted.by.city.zip' to make a directory of files containing data sorted by USPS Zip code. And what happens if the user has a file 'bar' that's not a ZIP file, and a directory 'bar.zip' isn't a view into 'bar'? Most of the time, if I have a file 'linux-2.6.12.tar.bz2' and a directory 'linux-2.6.12', what is under the directory is *NOT* the same data as what's in the .bz2 - I've done 'make oldconfig' and a few builds and some variable amount of patching, usually with rejects, and I *don't* want that .bz2 being updated during all this (hint - what's my next command after 'rm -rf linux-2.6.12' likely to be, and why, and what expectations do I have when I do it?) You want to think this sort of thing through *really* thoroughly, because there's a *lot* of things, both users and programs, that have expectations about The Way Things Work. > The whole point is to have less userland tools, not more. I'm not > saying we move zip into the kernel, just that the user now has one less > command to remember. But now instead of having to remember the one meme "I can manage any compressed-archive format that's stored in a file, and put other files in it, and all I need is the appropriate userspace tool", they have to remember "the cp trick works for .zip and .tar, but I'll get a "not a directory" error if I try it with a .hqx file, and that other file format may or may not work, because I can't remember if this kernel has a working out-of-tree module for this kernel...." [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 18:16 ` Valdis.Kletnieks @ 2005-06-26 19:58 ` David Masover 2005-06-26 21:05 ` Valdis.Kletnieks 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-26 19:58 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Sun, 26 Jun 2005 02:48:06 CDT, David Masover said: > > >>Lincoln Dale wrote: > > >>>this is the WHOLE point of standardization .. i don't think its that >>>Reiser4's EAs offer any more or less capabilities than standard EAs - >> >>They do. Reiser4's EAs can look like any other object -- files, >>folders, symlinks, whatever. This is important, especially for >>transparency. > > > No, you want them to look like the same objects that {get|set}xattr() manage > currently. You don't want programs to have to guess what an EA looks like > this week, with this user's combination of plugins that's different from > everybody else's. "Plugins" is a bad word. This user's combination of plugins is most likely identical to other users', it's just which ones are enabled, and which aren't? If they are all included, I assume they play nice. And just because they are called "plugins" doesn't mean the EA looks different every week. I cannot stress this enough. In Reiser4, EVERYTHING is some sort of plugin. We now have a standard disk format plugin, and a standard set of plugins providing UNIX filesystem-like things (vanilla files, directories, symlinks, device nodes, named pipes...) which has been stable for some time. I see no problems with providing at least as consistent an EA interface as any other FS, only more extensible because it's not flat. >>>lets take this a step further. what about compression? do we accept >>>that each filesystem can implement its own proprietary compression via >>>its own API - and now we need individual user-space tools to understand >> >>No, that's the beauty of these "EAs" in Reiser4. The API is standard >>write(2) commands. sys_reiser4 supposedly implements an interface to >>make this scale better, but otherwise have the same semantics. And who >>said anything about proprietary compression? I think we were planning >>on the kernel's zlib, though we might have been planning to make it a >>bit more seekable... >> >> >>>each of these APIs? >> >>So, the API becomes something like: >> >>cat crypto/inflated/foo # transparently decompressed >>cat crypto/raw/foo.gz # raw, gzip-compressed > > > And 'cat crypto/raw/foo' or 'crypto/inflated/foo.gz' gets you what, exactly? > > Now throw some .bz2 and .zip files into the mix... ;) Interface is the same. Only, zip files aren't just compression, so maybe the interface changes a little there. Point is, now you have a standard interface for any program to access any simple lossless compression, transparently. >>Another possibility, if you like file-as-a-directory: >> >>cat foo.gz # raw >>cat foo.gz/inflated # decompressed >> >>One could easily imagine things like these two potentially equivalent >>commands: >> >>cp foo bar.zip/ >>zip bar foo > > > Unless of course the user had done 'mkdir sorted.by.city.zip' to make > a directory of files containing data sorted by USPS Zip code. What's this got to do with anything? > And what happens if the user has a file 'bar' that's not a ZIP file, > and a directory 'bar.zip' isn't a view into 'bar'? In file-as-a-directory (which is probably NOT happening soon), bar.zip is both the actual zipfile and the view inside, depending on whether you try to open() it directly or peek inside it as a directory. I've used that interface for some simpler things like file permissions. I could do things like "echo 0 > some_file/metas/uid". The interface is gone for now, because it broke some things, I'm told, but nothing serious for me while I was using it. I don't think the problem is insurmountable. However, let's not discuss this now. I do NOT want to start another "silent semantic changes with reiser4" thread. File-as-directory is not happening this time, so don't worry about it -- this time. > Most of the time, if I have a file 'linux-2.6.12.tar.bz2' and a > directory 'linux-2.6.12', what is under the directory is *NOT* the same > data as what's in the .bz2 - I've done 'make oldconfig' and a few builds > and some variable amount of patching, usually with rejects, and I *don't* > want that .bz2 being updated during all this (hint - what's my next command > after 'rm -rf linux-2.6.12' likely to be, and why, and what expectations > do I have when I do it?) You're misunderstanding. man zip. $ zip bar foo creates/modifies a file named "bar.zip", not "bar", which contains the file "foo". > You want to think this sort of thing through *really* thoroughly, because > there's a *lot* of things, both users and programs, that have expectations > about The Way Things Work. Or, I can avoid those issues altogether, and simply delegate this kind of stuff to user-created-but-magic directories. For instance, I could have a directory called "/foo" which contains encrypted files, and "/foo/decrypted" which has transparently decrypted representations of them. >>The whole point is to have less userland tools, not more. I'm not >>saying we move zip into the kernel, just that the user now has one less >>command to remember. > > > But now instead of having to remember the one meme "I can manage any > compressed-archive format that's stored in a file, and put other files in it, > and all I need is the appropriate userspace tool", they have to remember "the > cp trick works for .zip and .tar, but I'll get a "not a directory" error if I > try it with a .hqx file, and that other file format may or may not work, > because I can't remember if this kernel has a working out-of-tree module for > this kernel...." True, for now. But for me, it's not hard at all to try "cp" first, and after it fails, Google and install the required tool. After all, most users already have to do that with GUI interfaces, this just takes the GUI way of doing things (like in file-roller) and makes it perform better and work with all programs. Not necessarily all formats, but all programs. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr8IzngHNmZLgCUhAQKS9A/+K+Nkq0sLynA0ltiA7o+BnA4cRSLYnu0t LdVI+yvYMUocvZt1HcIuU1LN2y3//Xm7hh7Lsyf3moNCIFMkDUN/xuzUcmfpUs5W NpyYPS4M3h8bX7nAMI9SmJ2KPoQNJxiEMToi5cYkQvafAeAlR2xw+z9nQ7Cv2PAi PCKROYSbZ1MdyCVLDPYkIpk9Z9EnOZ7zE+nbMaippqAnwfoqMoe8OMBKt1qxBROS tSPn4h1+JQZTs5OY18Cg/km7J6SM3SmHEWtsicil9/AVMFoOCpUXkeuiBLgVZ7Vk hh79SLjmuAOgU5hTBhEwvtEjB5uGqbjRtp+quIWNO4tmHqPuSv+TP0YaOgMTqbf7 9Cr+iq/YrWMDCA3fYrnq/TiSCNK/YHGNlQG4rZtnKDXUFUZRuItMhbRaiYYfv/Cl wQjBrknKci4xFNB9gIOoykS9Vs3JDdZc8oj26rFhHGqkGE+xo6b0z1EaZMvBfes9 L+VqU6B7YnyDWnhDbK6TiLZ/Qg83SigTyoLORVIDzyo3k2SVA8kmy8NgdA9NlmIT ko/g0+7zHtu64LuEtQEdL/FULnI/YcMtxQDjYuhCqtUo/x4y3hi5685P00iZDfdp RpuZnOT/esiw/RLashgGRK3xsVik9R+WX2psrEXp7HvNYvr5X4BpFovyVYeTxSnj jV201bzTv6A= =z+xB -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 19:58 ` David Masover @ 2005-06-26 21:05 ` Valdis.Kletnieks 2005-06-26 22:35 ` David Masover 2005-06-26 22:54 ` Hans Reiser 0 siblings, 2 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-26 21:05 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 4149 bytes --] On Sun, 26 Jun 2005 14:58:07 CDT, David Masover said: > "Plugins" is a bad word. This user's combination of plugins is most > likely identical to other users', it's just which ones are enabled, and > which aren't? If they are all included, I assume they play nice. Which ones are enabled. Exactly. > And just because they are called "plugins" doesn't mean the EA looks > different every week. They do if the one enabled this week is "make EAs look like symlinks", and last week's was "make EAs look like folders". (Don't blame me, *you're* the one that said "EAs can look like any other object"..) > > And 'cat crypto/raw/foo' or 'crypto/inflated/foo.gz' gets you what, exactly ? > > > > Now throw some .bz2 and .zip files into the mix... ;) > > Interface is the same. Only, zip files aren't just compression, so > maybe the interface changes a little there. Right. So please explain what crypto/raw/foo and crypto/inflated/foo.gz give you. > Point is, now you have a standard interface for any program to access > any simple lossless compression, transparently. > > >>Another possibility, if you like file-as-a-directory: > >> > >>cat foo.gz # raw > >>cat foo.gz/inflated # decompressed > >> > >>One could easily imagine things like these two potentially equivalent > >>commands: > >> > >>cp foo bar.zip/ > >>zip bar foo > > Unless of course the user had done 'mkdir sorted.by.city.zip' to make > > a directory of files containing data sorted by USPS Zip code. > > What's this got to do with anything? It's got a *LOT* to do with it if I created a *DIRECTORY*, to use *AS A DIRECTORY*, the way Unix-style systems have done for 3 decades, and suddenly my system is running like a pig because the kernel decided that it's a .zip file. > > And what happens if the user has a file 'bar' that's not a ZIP file, > > and a directory 'bar.zip' isn't a view into 'bar'? > > In file-as-a-directory (which is probably NOT happening soon), bar.zip > is both the actual zipfile and the view inside, depending on whether you > try to open() it directly or peek inside it as a directory. Ahem. "bar.zip' is a *DIRECTORY*. I said 'mkdir bar.zip' - why is it not acting like a directory? > However, let's not discuss this now. I do NOT want to start another > "silent semantic changes with reiser4" thread. File-as-directory is not > happening this time, so don't worry about it -- this time. Fish or cut bait. You are the one who started handwaving the 'file-as-directory'. If you don't want it discussed, don't mention it. > > Most of the time, if I have a file 'linux-2.6.12.tar.bz2' and a > > directory 'linux-2.6.12', what is under the directory is *NOT* the same > > data as what's in the .bz2 - I've done 'make oldconfig' and a few builds > > and some variable amount of patching, usually with rejects, and I *don't* > > want that .bz2 being updated during all this (hint - what's my next command > > after 'rm -rf linux-2.6.12' likely to be, and why, and what expectations > > do I have when I do it?) > > You're misunderstanding. man zip. > $ zip bar foo > creates/modifies a file named "bar.zip", not "bar", which contains the > file "foo". No. *YOU* are misunderstanding. I have a directory 'linux-2.6.12', and I have a file 'linux-2.6.12.tar.bz2', and I do *NOT* want directory operations to be silently converted into "let's scribble into the middle of this tar file and then compress it". (Hint - work out how long a kernel 'make' would take if you were doing it inside a .tar.bz2). > > You want to think this sort of thing through *really* thoroughly, because > > there's a *lot* of things, both users and programs, that have expectations > > about The Way Things Work. > > Or, I can avoid those issues altogether, and simply delegate this kind > of stuff to user-created-but-magic directories. For instance, I could > have a directory called "/foo" which contains encrypted files, and > "/foo/decrypted" which has transparently decrypted representations of them. So rather than everything working in a funky manner, a program gets to guess how funky, and in what direction, a given magical directory is.... [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 21:05 ` Valdis.Kletnieks @ 2005-06-26 22:35 ` David Masover 2005-06-26 23:43 ` Hans Reiser 2005-06-27 0:40 ` Valdis.Kletnieks 2005-06-26 22:54 ` Hans Reiser 1 sibling, 2 replies; 620+ messages in thread From: David Masover @ 2005-06-26 22:35 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Sun, 26 Jun 2005 14:58:07 CDT, David Masover said: > > >>"Plugins" is a bad word. This user's combination of plugins is most >>likely identical to other users', it's just which ones are enabled, and >>which aren't? If they are all included, I assume they play nice. > > > Which ones are enabled. Exactly. I doubt there will be duplicate plugins, once things settle down. So you just have the program demand certain ones to be enabled. >>And just because they are called "plugins" doesn't mean the EA looks >>different every week. > > > They do if the one enabled this week is "make EAs look like symlinks", and > last week's was "make EAs look like folders". We could just as easily do that with other extended attributes. Sure, the details would be different -- maybe we just randomly add a XA onto the beginning of every string. > (Don't blame me, *you're* the one that said "EAs can look like any other object"..) Doesn't mean we can't guarentee a certain kind in a certain configuration will always be available for a certain plugin, once it's been accepted. > > > >>>And 'cat crypto/raw/foo' or 'crypto/inflated/foo.gz' gets you what, exactly > > ? > >>>Now throw some .bz2 and .zip files into the mix... ;) >> >>Interface is the same. Only, zip files aren't just compression, so >>maybe the interface changes a little there. > > > Right. So please explain what crypto/raw/foo and crypto/inflated/foo.gz give you. In that example (shouldn't have used the name "crypto", but oh well), it should be crypto/raw/foo.gz and crypto/inflated/foo -- where foo.gz is the gzip'ed file and foo is the transparently compressed/decompressed file. Basically, these are equivalent: $ zcat crypto/raw/foo.gz $ cat crypto/inflated/foo >>Point is, now you have a standard interface for any program to access >>any simple lossless compression, transparently. >> >> >>>>Another possibility, if you like file-as-a-directory: >>>> >>>>cat foo.gz # raw >>>>cat foo.gz/inflated # decompressed >>>> >>>>One could easily imagine things like these two potentially equivalent >>>>commands: >>>> >>>>cp foo bar.zip/ >>>>zip bar foo > > >>>Unless of course the user had done 'mkdir sorted.by.city.zip' to make >>>a directory of files containing data sorted by USPS Zip code. >> >>What's this got to do with anything? > > > It's got a *LOT* to do with it if I created a *DIRECTORY*, to use *AS A DIRECTORY*, > the way Unix-style systems have done for 3 decades, and suddenly my system is > running like a pig because the kernel decided that it's a .zip file. The kernel does not decide that. You do. And it doesn't automatically decide that every time you create a file. You have to use some interface to trigger the plugins. Originally, this was file-as-a-directory. Now? I'm not sure, I guess we could use a separate delimiter. Something like: foo # file ...foo # directory containing xattrs of file, list of plugins used. foo/ # directory foo/... # directory containing xattrs of file, list of plugins used. I guess I need a new name for this approach. That's three possible ways of doing this? >>>And what happens if the user has a file 'bar' that's not a ZIP file, >>>and a directory 'bar.zip' isn't a view into 'bar'? >> >>In file-as-a-directory (which is probably NOT happening soon), bar.zip >>is both the actual zipfile and the view inside, depending on whether you >>try to open() it directly or peek inside it as a directory. > > > Ahem. "bar.zip' is a *DIRECTORY*. I said 'mkdir bar.zip' - why is it not > acting like a directory? If you said "mkdir", it would act like a directory. More likely than foo.zip/bar would be foo.zip/.../contents/bar. Which would also work for tarballs. But would not automatically compress anything you didn't want it to. >>However, let's not discuss this now. I do NOT want to start another >>"silent semantic changes with reiser4" thread. File-as-directory is not >>happening this time, so don't worry about it -- this time. > > > Fish or cut bait. You are the one who started handwaving the 'file-as-directory'. > If you don't want it discussed, don't mention it. I do want it discussed. I'm not sure it's a good idea now. But looks like you got me... >>>Most of the time, if I have a file 'linux-2.6.12.tar.bz2' and a >>>directory 'linux-2.6.12', what is under the directory is *NOT* the same >>>data as what's in the .bz2 - I've done 'make oldconfig' and a few builds >>>and some variable amount of patching, usually with rejects, and I *don't* >>>want that .bz2 being updated during all this (hint - what's my next command >>>after 'rm -rf linux-2.6.12' likely to be, and why, and what expectations >>>do I have when I do it?) >> >>You're misunderstanding. man zip. >>$ zip bar foo >>creates/modifies a file named "bar.zip", not "bar", which contains the >>file "foo". > > > No. *YOU* are misunderstanding. I have a directory 'linux-2.6.12', and > I have a file 'linux-2.6.12.tar.bz2', and I do *NOT* want directory operations > to be silently converted into "let's scribble into the middle of this tar file > and then compress it". (Hint - work out how long a kernel 'make' would take > if you were doing it inside a .tar.bz2). I remember discussing that, actually. It wouldn't automatically do this if you didn't want it to, but it would be nice if, say, it was something truly seekable like linux-2.6.12.zip, and linux-2.6.12 was a user-created symlink to linux-2.6.12.zip/.../contents, and we had a nice caching system... This is nice because then you get exactly the same performance during "make" as you would with "unzip && make", only better, because files you don't ever use (lots of arch, for instance) are not unpacked. This is all something that's been discussed before. I think the consensus was that it sounded cool, but not useful enough by itself to justify file-as-a-directory. You can always go re-read "Silent Semantic Changes..." >>>You want to think this sort of thing through *really* thoroughly, because >>>there's a *lot* of things, both users and programs, that have expectations >>>about The Way Things Work. >> >>Or, I can avoid those issues altogether, and simply delegate this kind >>of stuff to user-created-but-magic directories. For instance, I could >>have a directory called "/foo" which contains encrypted files, and >>"/foo/decrypted" which has transparently decrypted representations of them. > > > So rather than everything working in a funky manner, a program gets to guess > how funky, and in what direction, a given magical directory is.... It doesn't have to guess, it can know if it needs to. Although this probably wreaks havok on traditional backup systems. You'd have to implement a somewhat new backup system, but it's not impossible, and certainly doesn't require that your new backup system know about every plugin -- just make sure plugins know about backup. Of course, file-as-directory avoids all that -- you just have to patch programs to know that if foo/ exists or foo responds to opendir, foo is not necessarily a directory. You probably just drop the trailing / and use stat(). Both (all three?) appraches have cans-of-worms and workarounds. But I think both (all three?) are doable. I know user-created-but-magic is doable. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr8txHgHNmZLgCUhAQKOxA/+KP1lmUgmrbEH+kPKriSFF3qFfFBgbeJh iebHVaOY4AMa99a3yHy8iA5ZyE5N2b3CEueoq44ygYs5gqeft1IF14SJweNk/6or IgGt+OsAkqDMdbbpKZdDGo8lZK0bJq/Ni+PMzx44sMGg4vWIHGAQfDGZ9FYRmrRT fLNprUrkUsIzVEDHcuYjrZfPHvqr4HK8KXa5IJIKj6WkCYj5gxkQlH3g9zEaWXlN Mfsqr6uioK34vKEgQw9HvpDpMZaB5NpZ7RWr5vEP3G3AV120VNKIEqU09Pex/IJD oiOaroR7y1OZUaLuTSyut+4MGjHJ7k6TNr8H/CFdRfpLs7mP5tsJG42rzAhqTdyP 2D2m7sgpEN8ALkGd+IU3ihLOVQCrSq2T0jpr6tFhDe5uTenP3fXBeGgu8ynRzPFD kkpyZhT094UWKoq8n+IcHQrIcba4j6X/jDJt7SDx8IwfZaPoMpcBGQH1//Q+lTuC q5qM7olI5OzcSr4AhDld7OOjqO0pLXRfH5y8hy8mgWEiGgomK+N5OFTqXrQsuB/g BopfBpqHRRYVpI1VkAutmTpyNjm3dQjrHmRWXX2sFe9lxExpWo9VX4By9ytv9EXQ UFfSBF9TVe+EkZ4XxO+JMCcahNUsBh45rRXY3wVpH1qPwsOVp8FxYcOmuzQSTha9 mgOxWbRfqkQ= =jrll -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 22:35 ` David Masover @ 2005-06-26 23:43 ` Hans Reiser 2005-06-27 0:16 ` David Masover 2005-06-27 0:40 ` Valdis.Kletnieks 1 sibling, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-26 23:43 UTC (permalink / raw) To: David Masover Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover wrote: > Valdis.Kletnieks@vt.edu wrote: > > >On Sun, 26 Jun 2005 14:58:07 CDT, David Masover said: > > > >>"Plugins" is a bad word. This user's combination of plugins is most > >>likely identical to other users', it's just which ones are enabled, and > >>which aren't? If they are all included, I assume they play nice. > > > >Which ones are enabled. Exactly. Reiser4 plugins are not per user, but per kernel. They are compiled in. The model is intended to ease the development process, nothing more. Apologies if the naming suggests more. Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 23:43 ` Hans Reiser @ 2005-06-27 0:16 ` David Masover 2005-06-27 0:36 ` Valdis.Kletnieks ` (2 more replies) 0 siblings, 3 replies; 620+ messages in thread From: David Masover @ 2005-06-27 0:16 UTC (permalink / raw) To: Hans Reiser Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hans Reiser wrote: > David Masover wrote: > > >>Valdis.Kletnieks@vt.edu wrote: >> >> >>>On Sun, 26 Jun 2005 14:58:07 CDT, David Masover said: >> >> >>>>"Plugins" is a bad word. This user's combination of plugins is most >>>>likely identical to other users', it's just which ones are enabled, and >>>>which aren't? If they are all included, I assume they play nice. >> >> >>>Which ones are enabled. Exactly. > > > Reiser4 plugins are not per user, but per kernel. They are compiled > in. The model is intended to ease the development process, nothing > more. Apologies if the naming suggests more. But, to avoid confusion, the inclusion of a crytocompress plugin in a given kernel doesn't mean that all files accessed from that kernel are encrypted and compressed. It just means that you can pick an individual file and set it to be transparently encrypted/compressed. That is what I meant by "enabled". Not per-user, but per-file. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr9FcHgHNmZLgCUhAQJ2+BAAgnqAcEg41VXTUmpUUZWMduWeJKKYDE16 QvWore6MK/FWoU8hqFMddTzJVpzKNH8NoZ1Jfm2nEC51nndckwtBDsuIlMb1CGC+ I+wCrpe1kjXTz8sTs++CwDg29YojHP1/Wa5MwnUgTEBkZxacPI7r0VXyt1oO+f/N Eu6Jpgl4IN3ba52/PS+dXCQrmOJDhLeHAb+l4rtRsK+LASJitqmGmpf4PRwEejM8 dWZyjEYdiAcMx6mkeB/lho5IIcp1Xi5QBACf6S8SHXvHxRW50cKeouAISR25Ttk2 Oa1vKhWPGo4IBREjK7fzgj6GwfpIBnUfE25ZRjrBRupWumwekaCAY91JOoRkdRI+ Lw70OZhqqIKQ/O46xqhyCrroP/6is5ZxLyIRe1q3qsQfsWUfHBOONRUBdtaTQlaa 3OWzAU49cxn4Jv9S9UEYEyKx9ggqJ5hd94wZpXPq4GogWt4S8cYK3d4ZtRIyXEBr qOrLoJoTF9WeT254u+uq5gLP5kxq7Z7J51aMXbGF+4luuGJPq/50Y4hbapAMQWFA v0Z8YNWoOnOJgYji/+u8qJhsfzBdi/dmWlhhy9Te1e1b/1+zHcsbCslgsIrG2spk 3uF1GOv5NorO2n7RhierhkUrkUsLLlFqf0vPmgMqJ9DG4h6wl3bOJkYShBKTYQB+ ldYcxERKMZ4= =jv9S -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 0:16 ` David Masover @ 2005-06-27 0:36 ` Valdis.Kletnieks 2005-06-27 3:48 ` Hans Reiser 2005-06-27 5:05 ` Horst von Brand 2 siblings, 0 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 0:36 UTC (permalink / raw) To: David Masover Cc: Hans Reiser, Lincoln Dale, Gregory Maxwell, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 470 bytes --] On Sun, 26 Jun 2005 19:16:48 CDT, David Masover said: > But, to avoid confusion, the inclusion of a crytocompress plugin in a > given kernel doesn't mean that all files accessed from that kernel are > encrypted and compressed. It just means that you can pick an individual > file and set it to be transparently encrypted/compressed. > > That is what I meant by "enabled". Not per-user, but per-file. Doing key management in a secure manner is going to be *fun*. :) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 0:16 ` David Masover 2005-06-27 0:36 ` Valdis.Kletnieks @ 2005-06-27 3:48 ` Hans Reiser 2005-06-27 5:05 ` Horst von Brand 2 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-27 3:48 UTC (permalink / raw) To: David Masover Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover wrote: > Hans Reiser wrote: > > >David Masover wrote: > > > >>Valdis.Kletnieks@vt.edu wrote: > >> > >> > >>>On Sun, 26 Jun 2005 14:58:07 CDT, David Masover said: > >> > >> > >>>>"Plugins" is a bad word. This user's combination of plugins is most > >>>>likely identical to other users', it's just which ones are > enabled, and > >>>>which aren't? If they are all included, I assume they play nice. > >> > >> > >>>Which ones are enabled. Exactly. > > > >Reiser4 plugins are not per user, but per kernel. They are compiled > >in. The model is intended to ease the development process, nothing > >more. Apologies if the naming suggests more. > > > But, to avoid confusion, the inclusion of a crytocompress plugin in a > given kernel doesn't mean that all files accessed from that kernel are > encrypted and compressed. It just means that you can pick an individual > file and set it to be transparently encrypted/compressed. > > That is what I meant by "enabled". Not per-user, but per-file. > You are correct. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 0:16 ` David Masover 2005-06-27 0:36 ` Valdis.Kletnieks 2005-06-27 3:48 ` Hans Reiser @ 2005-06-27 5:05 ` Horst von Brand 2005-06-27 5:52 ` Gregory Maxwell 2005-06-27 6:22 ` David Masover 2 siblings, 2 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-27 5:05 UTC (permalink / raw) To: David Masover Cc: Hans Reiser, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover <ninja@slaphack.com> wrote: > Hans Reiser wrote: [...] > > Reiser4 plugins are not per user, but per kernel. They are compiled > > in. The model is intended to ease the development process, nothing > > more. Apologies if the naming suggests more. What do you gain by this? It is just a kernel configuration option of sorts? Just name mangling of existing mechanisms for no good reason at all? > But, to avoid confusion, the inclusion of a crytocompress plugin in a > given kernel doesn't mean that all files accessed from that kernel are > encrypted and compressed. It just means that you can pick an individual > file and set it to be transparently encrypted/compressed. > > That is what I meant by "enabled". Not per-user, but per-file. Wonderful! I carefully "transparently encrypt" my secret files, so /everybody/ can read them! Now /that/ is progress! -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 5:05 ` Horst von Brand @ 2005-06-27 5:52 ` Gregory Maxwell 2005-06-27 6:22 ` David Masover 1 sibling, 0 replies; 620+ messages in thread From: Gregory Maxwell @ 2005-06-27 5:52 UTC (permalink / raw) To: Horst von Brand Cc: David Masover, Hans Reiser, Valdis.Kletnieks, Lincoln Dale, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On 6/27/05, Horst von Brand <vonbrand@inf.utfsm.cl> wrote: > Wonderful! I carefully "transparently encrypt" my secret files, so > /everybody/ can read them! Now /that/ is progress! All of this side feature argument is completely offtopic for the inclusion of reiser4, but oh well. In any case, the real use for encrypted files (vs encrypted partitions) would be for doing things like tying keying into the login process so that your files are only accessible while you are logged in. This would be a very nice feature on a multiuser system. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 5:05 ` Horst von Brand 2005-06-27 5:52 ` Gregory Maxwell @ 2005-06-27 6:22 ` David Masover 1 sibling, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-27 6:22 UTC (permalink / raw) To: Horst von Brand Cc: Hans Reiser, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Horst von Brand wrote: > David Masover <ninja@slaphack.com> wrote: > >>Hans Reiser wrote: > > > [...] > > >>>Reiser4 plugins are not per user, but per kernel. They are compiled >>>in. The model is intended to ease the development process, nothing >>>more. Apologies if the naming suggests more. > > > What do you gain by this? It is just a kernel configuration option of > sorts? Just name mangling of existing mechanisms for no good reason at all? I don't know why it's named "plugins", but the mechanism itself gains a lot. For one thing, certain new "plugins" for new kinds of files (cryptocompressed files, for instance) can be almost as easy as new plugins for (say) XMMS. Still, Hans, it's high time to either rename these "plugins" or start talking about bytecode engines... >>But, to avoid confusion, the inclusion of a crytocompress plugin in a >>given kernel doesn't mean that all files accessed from that kernel are >>encrypted and compressed. It just means that you can pick an individual >>file and set it to be transparently encrypted/compressed. >> >>That is what I meant by "enabled". Not per-user, but per-file. > > > Wonderful! I carefully "transparently encrypt" my secret files, so > /everybody/ can read them! Now /that/ is progress! Explain to me how this is worse than how you currently encrypt your secret files? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr+bCHgHNmZLgCUhAQKnag/8DF1PjfQwEYaJWUOOY0roqn6b+/L8X/X1 kgFE3SyPJkNRuY6X/J3EQUht5v/EBEbk2ZE/oTWB+HGEeUv1juUmG7z2rWCvi8sY tXPG28iqOAjwvZa+2aq8o4ptW0+gaXVIbrNGEaWTXPgbxdxtoVMOAakBJl3FUchK pwZWJD+msuQuPFc42zQ3Zdvp+4/NhhC6qcSpZ9DGXHO9uDJDDdl3Xgf9+qOD+W7Y Bz4sGfNphIbJjydHs0DdcZPh49MhS+qd48Qw1WFskalF9dhZhHFq3MKIBFbg8bA8 MX9yY5XHrNPJqJXF96YtcJV9nZcXh4qEz81/JC7V+VYm/BME20a0jtUt5bMtGRjj kC8FxzVvamOmeXnzRnU2DKIlGlfTUmK1u6wQpMsoiwPZr9WH7WRxiP/tHPVYQbBc gPpg7POOz3M6Q3guZMGTxeYu7kHJm8s/zJVwXm04fz+03hrkab1q/w4cT0SaX4Pv naipO7nkdr5qoRzF2sBSZAsnauxGa0ViR0TP715FN+oX0ZRiSQcgEGc+zXz6qPb5 bSz29jG1iGkrepB6LU88gMZiyxc2qK1SHpZlKjn9iPKqpxpwEtpQToUBhRsAlFBv 9Z6KxrQYrUYfJ/Z9ktIUuvsvN3zbnnc7tmZGYNisqm03dq61kguAwOV66INSlYDj gFaw23Z9X58= =xsgf -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 22:35 ` David Masover 2005-06-26 23:43 ` Hans Reiser @ 2005-06-27 0:40 ` Valdis.Kletnieks 2005-06-27 2:37 ` David Masover ` (2 more replies) 1 sibling, 3 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 0:40 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 2841 bytes --] On Sun, 26 Jun 2005 17:35:48 CDT, David Masover said: > > Right. So please explain what crypto/raw/foo and crypto/inflated/foo.gz give you. > > In that example (shouldn't have used the name "crypto", but oh well), it > should be crypto/raw/foo.gz and crypto/inflated/foo -- where foo.gz is > the gzip'ed file and foo is the transparently compressed/decompressed > file. Basically, these are equivalent: > > $ zcat crypto/raw/foo.gz > $ cat crypto/inflated/foo I'm *quite* aware of what your preconceived notions think it *should* be. Maybe the two examples I asked for have *real-world* meanings that you should be allowing for. Like, for instance, on a mail server, where the A/V software may need to unzip a file 5 or 6 times to find out if there's malicious content. Or seeing if it's a ".zip bomb", where a small .zip will decompress to gigabytes. Or I'm testing a new compression algorithm, to see if multiple compressions help (yes, I know that it *shouldn't* help - but I've seen real-world cases where the algorithm could only look at a 4K or 8K window at a time, and if you hit a *very* long run of duplicate 4K segments, a second compression would compress all the identical or near-identical "this is a 4K chunk" tokens...) > > It's got a *LOT* to do with it if I created a *DIRECTORY*, to use *AS A DIRECTORY*, > > the way Unix-style systems have done for 3 decades, and suddenly my system is > > running like a pig because the kernel decided that it's a .zip file. > > The kernel does not decide that. You do. And it doesn't automatically > decide that every time you create a file. You have to use some > interface to trigger the plugins. Oh, I'm waiting for the fun the first time somebody deploys a plugin that has similar semantics to 'chmod g+s dirname/' ;) > I guess I need a new name for this approach. That's three possible ways > of doing this? I *said* you need to think this through in detail, didn't I? ;) > I remember discussing that, actually. It wouldn't automatically do this > if you didn't want it to, but it would be nice if, say, it was something > truly seekable like linux-2.6.12.zip, and linux-2.6.12 was a > user-created symlink to linux-2.6.12.zip/.../contents, and we had a nice > caching system... I think you're highly deluded as to just how much or little performance gain this will get you. Model what happens with a kernel 'make' on a 256M machine with and without all that zipping and compressing, and assume that a constant 48M is available for caching of the linux-2.6.12/ tree. > This is nice because then you get exactly the same performance during > "make" as you would with "unzip && make", only better, because files you > don't ever use (lots of arch, for instance) are not unpacked. Go read http://www.tux.org/lkml/#s7-7 and ponder until enlightenment arrives. [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 0:40 ` Valdis.Kletnieks @ 2005-06-27 2:37 ` David Masover 2005-06-27 3:10 ` Kyle Moffett ` (2 more replies) 2005-06-27 2:49 ` David Masover 2005-06-27 3:10 ` Hubert Chan 2 siblings, 3 replies; 620+ messages in thread From: David Masover @ 2005-06-27 2:37 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Sun, 26 Jun 2005 17:35:48 CDT, David Masover said: > > >>>Right. So please explain what crypto/raw/foo and crypto/inflated/foo.gz give you. >> >>In that example (shouldn't have used the name "crypto", but oh well), it >>should be crypto/raw/foo.gz and crypto/inflated/foo -- where foo.gz is >>the gzip'ed file and foo is the transparently compressed/decompressed >>file. Basically, these are equivalent: >> >>$ zcat crypto/raw/foo.gz >>$ cat crypto/inflated/foo > > > I'm *quite* aware of what your preconceived notions think it *should* be. > > Maybe the two examples I asked for have *real-world* meanings that you should > be allowing for. Like, for instance, on a mail server, where the A/V software > may need to unzip a file 5 or 6 times to find out if there's malicious content. > > Or seeing if it's a ".zip bomb", where a small .zip will decompress to gigabytes. > > Or I'm testing a new compression algorithm, to see if multiple compressions help > (yes, I know that it *shouldn't* help - but I've seen real-world cases where the > algorithm could only look at a 4K or 8K window at a time, and if you hit a *very* > long run of duplicate 4K segments, a second compression would compress all the > identical or near-identical "this is a 4K chunk" tokens...) > > > >>>It's got a *LOT* to do with it if I created a *DIRECTORY*, to use *AS A DIRECTORY*, >>>the way Unix-style systems have done for 3 decades, and suddenly my system is >>>running like a pig because the kernel decided that it's a .zip file. >> >>The kernel does not decide that. You do. And it doesn't automatically >>decide that every time you create a file. You have to use some >>interface to trigger the plugins. > > > Oh, I'm waiting for the fun the first time somebody deploys a plugin that > has similar semantics to 'chmod g+s dirname/' ;) > > >>I guess I need a new name for this approach. That's three possible ways >>of doing this? > > > I *said* you need to think this through in detail, didn't I? ;) > > >>I remember discussing that, actually. It wouldn't automatically do this >>if you didn't want it to, but it would be nice if, say, it was something >>truly seekable like linux-2.6.12.zip, and linux-2.6.12 was a >>user-created symlink to linux-2.6.12.zip/.../contents, and we had a nice >>caching system... > > > I think you're highly deluded as to just how much or little performance gain > this will get you. Model what happens with a kernel 'make' on a 256M machine > with and without all that zipping and compressing, and assume that a constant > 48M is available for caching of the linux-2.6.12/ tree. Ignoring Hans' point, there is still a performance gain. Assume we can do on-disk caching, similar to fscache/cachefs for nfs. Now, benchmark: $ unzip linux-2.6.12.zip && make -C linux-2.6.12 versus the hypothetical $ make -C linux-2.6.12.zip/.../contents This is an automatic performance gain, in theory, because the second command is identical to unzipping just the parts you need into linux-2.6.12, then running "make". The one disadvantage is that because the unzipping is done on demand, it only really performs well if you can keep the "zip" binary cached. Even on most embedded systems, 54K of RAM is really not much to ask these days. Also, once you've run "make" once, you get to run it as many times as you like, and so long as the on-disk cache of the zipfile is still there and valid, you never have the overhead of unzipping again. Of course, this probably saves only a minute or two at most per kernel compile. But that doesn't mean there aren't real-world situations where this kind of architecture would be much more beneficial. >>This is nice because then you get exactly the same performance during >>"make" as you would with "unzip && make", only better, because files you >>don't ever use (lots of arch, for instance) are not unpacked. > > > Go read http://www.tux.org/lkml/#s7-7 and ponder until enlightenment arrives. So what? I don't intend to convince anyone based on how much slower/faster their kernel compiles. It's meant to illustrate the principle of the thing. Besides, your point was that you could not run make inside of a kernel tarball/zipfile. Nobody ever suggested that you would actually want to. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr9mfHgHNmZLgCUhAQJi0g//RGxFXWi8Om4EnKsZHcI0J7X3G6T9pj2a nwkWwjLdyg6jykdt3a5MTELBgOM2uT87k7CeO7TasA/T1ZkZ/y2Yw7x50YIgrb7j W1u5N/vfDLw3C2Nd6O2fe/b4ygReyBB6HQtNTw/k+XxDswxtEp0mcZHpNxt+W9B4 s0naezawRjF51P4ISqa6HoRo7vZUkXv+3CwuZinC5m8KnW2Us8Ww5uDjtNBLpJGR zs79w24zaj6VSHjF8lO6CuMKQLjSelMDXKDEkFHaybJgz7AckkcZg5c6VDTrc3/t m8HM5oyHWfRqVeQt9cMdLdcBZnhdbspSwQaNQkdkrZx1IX96mPoMNDUwk1B/TIi7 ++iqpkDYeOdg+DWzGPVpwQ+5LQC+7m8vRHv5dROIM6T89TnlUck2clBiPovzSP+8 KMR1R6F7qBPxJMkPcy2MNOVMkjN+VLSOCzOeOXVEUkNYdXjrJB5h3XIN5MyRR7C/ pRmjB2crzPTUz2yBatP+QUFNUMadikMqj44sEc/ie8iHbo9pfxQC/wY4M4VkJcf2 03spe2e8M+k3txj63O9TmJYgobkjx+d+cJ5Zm7DbKJmGlVaAGqmCXeNjxhTtBSwU yP2Jrz6Awu+nDxFxMAsN4GP17/0/aXdBhIYLPGyAJ9/HV7KHENIqIjnvD4e3bXnU shBrM+G1N5E= =BXCi -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 2:37 ` David Masover @ 2005-06-27 3:10 ` Kyle Moffett 2005-06-27 3:24 ` David Masover 2005-06-27 4:23 ` Valdis.Kletnieks 2005-06-27 4:27 ` Valdis.Kletnieks 2 siblings, 1 reply; 620+ messages in thread From: Kyle Moffett @ 2005-06-27 3:10 UTC (permalink / raw) To: David Masover Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Jun 26, 2005, at 22:37:48, David Masover wrote: > $ make -C linux-2.6.12.zip/.../contents I've yet to see how such automatic unzipping does not inherently introduce system insecurity into _existing_ applications by changing POSIX and UNIX semantics. When I do this in my suid perl script: open my $fh, '<', $ARGV[0]; I do _not_ mean this: system '/bin/zip', 'x', $path1, $path2; Neither do I want the kernel to unzip it, because that just introduces the kernel to a whole series of normally application-level vulnerabilities. I've seen situations where a specially doctored tarball can expand to a result file over 1000x the size of the original. Likewise, there have been cases where crafted tarballs have taken advantage of buffer overflows. Can you illustrate for me with precise, clear, and unambiguous arguments how this can avoid all possible pitfalls, especially in cases where it's likely to break existing working privileged apps? Such extended operations, including the automatic encryption/decryption and most other non- standard filesystem features (Basically the whole '...' directory), should probably be left out of any patch submitted for inclusion until they can be _proven_ (or at least thoroughly checked) not to have undesirable results. Cheers, Kyle Moffett -- Somone asked me why I work on this free (http://www.fsf.org/philosophy/) software stuff and not get a real job. Charles Shultz had the best answer: "Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life." -- Charles Shultz ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 3:10 ` Kyle Moffett @ 2005-06-27 3:24 ` David Masover 2005-06-27 3:40 ` Kyle Moffett 2005-06-27 5:38 ` Horst von Brand 0 siblings, 2 replies; 620+ messages in thread From: David Masover @ 2005-06-27 3:24 UTC (permalink / raw) To: Kyle Moffett Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Kyle Moffett wrote: > On Jun 26, 2005, at 22:37:48, David Masover wrote: > >> $ make -C linux-2.6.12.zip/.../contents > > > I've yet to see how such automatic unzipping does not inherently introduce > system insecurity into _existing_ applications by changing POSIX and UNIX > semantics. > > When I do this in my suid perl script: > > open my $fh, '<', $ARGV[0]; > > I do _not_ mean this: > system '/bin/zip', 'x', $path1, $path2; > > Neither do I want the kernel to unzip it, because that just introduces the > kernel to a whole series of normally application-level vulnerabilities. That just means the zip plugin has to be more carefully written than I thought. It would have to be sandboxed and limited to prevent buffer overruns and zip bombs. I think that you could create a situation where an untrusted zip could be explored, and the worst side effect would be that the files you see inside the untrusted zip wouldn't be what you expect. But that's no surprise -- if the zip is really malicious, there's no amount of sandboxing that would give you valid files anyway. I was probably taking this too lightly, though. Remember that zip, at least, would not be in the kernel. I think what is going in the kernel is gzip and lzo, and it's being done so transparently that you never actually see the compressed version. > Can you illustrate for me with precise, clear, and unambiguous arguments That will take some time. And some thinking. > how this can avoid all possible pitfalls, Especially if you want perfection. > including the automatic encryption/decryption and most other non- standard > filesystem features (Basically the whole '...' directory), should probably > be left out of any patch submitted for inclusion until they can be > _proven_ > (or at least thoroughly checked) not to have undesirable results. I am doing that out of habit. Actually, it probably ends up more as: .../foo.zip/ instead of foo.zip/.../ But, it is left out. At least that interface. Now, the cryptocompress as it currently stands does not involve ... and does not introduce any new security holes in the way that you are describing. There might be some issues with key management (someone hinted vaguely at that), but nothing insurmountable. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr9xZ3gHNmZLgCUhAQKmOA/9EGus4fGXKnBPoK4Rpd9K/6tgy0A7QFIO EeF50BSgl7M5sot9Vp28Dg1DA9X8gXHf/BxWIUse2doEdrbYKy3HFNd4ZChPFXCS j4ZtJo/eGYQntIFZ+exNJG2gDeprSBUH9AnMxF9xBfG8CxBdl6WL1dx8d7kc7ip/ UYGiu+9WmC9LanEb0ezhsO8I0KBvjx73Q3FTbD9N0cMIzK1Drv7p9r9rhswsoIzg eZnKrT2Z0u8BASbd0GfCAT3DH3Zn6zHf6Zk9FOaPaqcLwWlXbELaTbFvBp+2rpnH 9j3NwKHTtCKX5Z2BOQqtiEDEE8QInuDlcEV2K/y4g9YM1mMw3TNoXswaZrbPWWeF RD/YyzknaDhfgQdz9kQ2bfHJM7//Y9IUJZp/3NmWhvhc2+QBhXBBoTb8UEnRl7tK D9UIgsDFDVHlLcO9KPKokjyf3fRL57dd0ictHFORvicVIK6NFSfFP9LY/ZsmUXF9 Fbqwwuu8/5n9z5j9IyIqf5m7XJkHcZCeLFDkn89VS4ZM8W1aB0g3yRIlhSEDDkVt 0nZZRzvIlRHCPoPMZuoFhWSngi50ZAYXlRicKrHQERP8f7ECkB1JvY2rLSFqM+FX BGUZpXxgHe/zgg806ACNbfdnny5WgZ7qXl/IXD9svQa3lIgxY6We8ACj8Oi6c7eu 5ooBcT7AmRE= =pBA8 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 3:24 ` David Masover @ 2005-06-27 3:40 ` Kyle Moffett 2005-06-27 21:19 ` David Masover 2005-06-27 5:38 ` Horst von Brand 1 sibling, 1 reply; 620+ messages in thread From: Kyle Moffett @ 2005-06-27 3:40 UTC (permalink / raw) To: David Masover Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Jun 26, 2005, at 23:24:23, David Masover wrote: >> Neither do I want the kernel to unzip it, because that just >> introduces the >> kernel to a whole series of normally application-level >> vulnerabilities. > > That just means the zip plugin has to be more carefully written than I > thought. It would have to be sandboxed and limited to prevent buffer > overruns and zip bombs. > Remember that zip, at least, would not be in the kernel. I think what > is going in the kernel is gzip and lzo, and it's being done so > transparently that you never actually see the compressed version. >> Can you illustrate for me with precise, clear, and unambiguous >> arguments > > That will take some time. And some thinking. Precisely. Come back when you have a good sturdy set of arguments. See the FUSE project (Also discussed in this thread), for better ideas on how to add strange filesystem semantics. NOTE: Even the FUSE project, which is in _userspace_ (as opposed to the massive kernelspace mess of reiser4), is taking a lot of flak for changing UNIX semantics, so I see no reason why Reiser4 should be special. >> how this can avoid all possible pitfalls, > > Especially if you want perfection. It doesn't need perfection, it just needs to convince a large segment of the developers on this list (and especially linus). >> including the automatic encryption/decryption and most other non- >> standard >> filesystem features (Basically the whole '...' directory), should >> probably >> be left out of any patch submitted for inclusion until they can be >> _proven_ (or at least thoroughly checked) not to have undesirable >> results. > > I am doing that out of habit. Actually, it probably ends up more as: > .../foo.zip/ > instead of > foo.zip/.../ > > But, it is left out. At least that interface. Ok, good. That's one of the first issues. A _lot_ of applications would get themselves thoroughly confused at that '...' interface, not to mention a lot of sysadmins :-D. > Now, the cryptocompress as it currently stands does not involve ... > and > does not introduce any new security holes in the way that you are > describing. Good. I think that we can all agree that, in the event cryptocompress is included, it would be a nice feature to have in all filesystems. Therefore, we should attempt to come up with a clean interface with which it could be added _inside_ the VFS. > There might be some issues with key management (someone > hinted vaguely at that), but nothing insurmountable. Likewise, this should be handled in a common VFS interface that all FSs can use. There already exists a key management system that would not be particularly difficult to interface with, but it would take thought and discussion. Cheers, Kyle Moffett -- There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 3:40 ` Kyle Moffett @ 2005-06-27 21:19 ` David Masover 2005-06-27 23:03 ` Kyle Moffett 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-27 21:19 UTC (permalink / raw) To: Kyle Moffett Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Kyle Moffett wrote: > On Jun 26, 2005, at 23:24:23, David Masover wrote: > >>> Neither do I want the kernel to unzip it, because that just >>> introduces the >>> kernel to a whole series of normally application-level vulnerabilities. >> >> >> That just means the zip plugin has to be more carefully written than I >> thought. It would have to be sandboxed and limited to prevent buffer >> overruns and zip bombs. > > > > >> Remember that zip, at least, would not be in the kernel. I think what >> is going in the kernel is gzip and lzo, and it's being done so >> transparently that you never actually see the compressed version. > > > >>> Can you illustrate for me with precise, clear, and unambiguous >>> arguments >> >> >> That will take some time. And some thinking. > > > Precisely. Come back when you have a good sturdy set of arguments. See > the FUSE project (Also discussed in this thread), for better ideas on how > to add strange filesystem semantics. If I didn't care about performance. I will read up on how FUSE works, though, to see if there's anything in the _kernel_ code that would be useful. > NOTE: Even the FUSE project, which > is in _userspace_ (as opposed to the massive kernelspace mess of reiser4), > is taking a lot of flak for changing UNIX semantics, so I see no reason > why Reiser4 should be special. Right. Kernel people hate change. Got it. No, seriously, I have to respect the history involved, which is why I'm not pretending to know more than I do. > Ok, good. That's one of the first issues. A _lot_ of applications > would get > themselves thoroughly confused at that '...' interface, not to mention > a lot > of sysadmins :-D. Not as much as you think. Unless a lot of applications can't handle opening or saving files that have '...' in the pathname? The sysadmin problem doesn't worry me so much. >> Now, the cryptocompress as it currently stands does not involve ... and >> does not introduce any new security holes in the way that you are >> describing. > > > Good. I think that we can all agree that, in the event cryptocompress is > included, it would be a nice feature to have in all filesystems. > Therefore, > we should attempt to come up with a clean interface with which it could be > added _inside_ the VFS. Agreed. I don't know enough about VFS to propose one, though. I think others are working on that, finally. >> There might be some issues with key management (someone >> hinted vaguely at that), but nothing insurmountable. > > > Likewise, this should be handled in a common VFS interface that all FSs > can use. There already exists a key management system that would not be > particularly difficult to interface with, but it would take thought and > discussion. Not too much, I hope. Although being able to use different keys on a per-file basis would be nice. I'm not sure if that would work in the existing system. Not to mention that keys would also be handlable with '...', if/when it happens. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsBtWXgHNmZLgCUhAQI5gw/8DXFk/v3faGbAtv0vdeMmy5aZ4MSpr3pp +rjjb8Wx6twCZelmPQgwTBtSQec9XUEipW3ucHVeP5Y2LM8hZHfbxEpHCzDydlH0 AppxZfHp/FBlqROSYajcc4qX062QQ7Wc/VfePmvVKcC7Plo1n1UnF30RRFLR67/r SOwb4/GPDnYyY0SQQt8XmuFmu1ngRB5M/kFgGOkiGLOsRiPzx4zVnYsxnOp8vH/q l0SxCsjev4+dL0TRnFBhx/uw7GerZJhqjjB9DZZgMIXILXt8oTHj1ubEXD0WRQxr iwiXruO2rO3vFbJVXzdlhFnUj98ViQR9nALa+CwT779Zus9hx8/KAxScPXEZPzVz jdF7y7GUvBL9vR+t5TTJjlpSqyfiyYKEw038/li/uBB1ThXtjTx4uL0QLzWkaaOh mfdX5RFwyinrsmscBaYgUPHwqLTjgcIWwqSfmGQ9RvYbMkHy9ZOzHTP9chJIeD5F cuJPluPEOMpeQduF/S7sG/Y00eQaMx0nBxtDprkwonnhy3Qw2jSSgsXqEIG+x45P 3qjiK7/4cZSkN/Q/VkObEwD0GE2FD31yfqSkrkGRMkhOg22YWBVqrwvmcm2RyQk9 h9S9C3HzNXGAuW9HfCmWLqg1yAQCilkTdGRpc38unBa364nRlBzii7PkA3XLol3S zChIED+O964= =shbE -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 21:19 ` David Masover @ 2005-06-27 23:03 ` Kyle Moffett 2005-06-27 23:27 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Kyle Moffett @ 2005-06-27 23:03 UTC (permalink / raw) To: David Masover Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Jun 27, 2005, at 17:19:21, David Masover wrote: >> Precisely. Come back when you have a good sturdy set of >> arguments. See >> the FUSE project (Also discussed in this thread), for better ideas >> on how >> to add strange filesystem semantics. > > If I didn't care about performance. I will read up on how FUSE works, > though, to see if there's anything in the _kernel_ code that would be > useful. A number of projects do their performance critical things in userspace, including real-time audio and video editing. The kernel is *NOT* for performance-critical things, unless they cannot be done efficiently in userspace. It is for _security_critical_ and _stability_critical_ things, and even then, some of those are pushed to userspace as well. >> NOTE: Even the FUSE project, which >> is in _userspace_ (as opposed to the massive kernelspace mess of >> reiser4), >> is taking a lot of flak for changing UNIX semantics, so I see no >> reason >> why Reiser4 should be special. > > Right. Kernel people hate change. Got it. > > No, seriously, I have to respect the history involved, which is why > I'm > not pretending to know more than I do. There is a good reason to hate change (at least without thorough evidence to back it up). Even small changes can break things in subtle ways. Big changes can tend to wreak kernel-global (and even userspace-global) havoc. >> Ok, good. That's one of the first issues. A _lot_ of applications >> would get themselves thoroughly confused at that '...' interface, not >> to mention a lot of sysadmins :-D. >> > > Not as much as you think. Unless a lot of applications can't handle > opening or saving files that have '...' in the pathname? If those files are as special as you lead me to believe, even a simple "tar -cjvf foo.tar.bz2 foo" would break horribly, because it would tar up all sorts of strange special files that shouldn't be included. Worse, when I untar that archive on the filesystem, it will overwrite those files, which probably should be overwritten even less than they should be stored in an archive. > The sysadmin problem doesn't worry me so much. Personally, I know several sysadmins who, never having heard of Reiser4 but using it anyways, would get scared when all of the '...' directories started showing up, thinking that some cracker had gotten a module loaded in their kernel. If you can, please avoid overloading existing semantics in filesystems. You can either claim that your filesystem is POSIXy, (similar to ext3, xfs, etc), or you can claim that it's not compatible by design (AFS, Coda) or you can claim that it's a completely virtualized interface (procfs, sysfs, CKRM fs, etc) and doesn't care about POSIX in any case. Reiser4 definitely isn't the third, but neither is it either of the first two, because it would break standard operations. > Agreed. I don't know enough about VFS to propose one, though. I > think > others are working on that, finally. When the VFS work gets done, this discussion can move on, otherwise, we are stuck at an impasse. > Not too much, I hope. > > Although being able to use different keys on a per-file basis would be > nice. I'm not sure if that would work in the existing system. Not to > mention that keys would also be handlable with '...', if/when it > happens. I think the '...' is just a bad idea in general, because it breaks tar and such. An interface like this might be simpler to design and use: # mkdir /attr # mount -t attrfs attrfs /attr /attr/fd/$FD_NUM == '...' directory for a filedescriptor /attr/fs/$DEV_NUM/$INODE_NUM == '...' directory for an inode It would be usable from a shell with a simple "getattrpath" command which returns "fs/$DEV_NUM/$INODE_NUM" by stat-ing any given path. Cheers, Kyle Moffett -- There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 23:03 ` Kyle Moffett @ 2005-06-27 23:27 ` David Masover 2005-06-28 2:21 ` Hubert Chan 2005-06-28 2:34 ` Lee Revell 0 siblings, 2 replies; 620+ messages in thread From: David Masover @ 2005-06-27 23:27 UTC (permalink / raw) To: Kyle Moffett Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Kyle Moffett wrote: > On Jun 27, 2005, at 17:19:21, David Masover wrote: > >>> Precisely. Come back when you have a good sturdy set of arguments. >>> See >>> the FUSE project (Also discussed in this thread), for better ideas >>> on how >>> to add strange filesystem semantics. >> >> >> If I didn't care about performance. I will read up on how FUSE works, >> though, to see if there's anything in the _kernel_ code that would be >> useful. > > > A number of projects do their performance critical things in userspace, > including real-time audio and video editing. The kernel is *NOT* for > performance-critical things, unless they cannot be done efficiently in > userspace. I don't think transparent compress-on-flush can be done efficiently in userspace. > It is for _security_critical_ and _stability_critical_ things, > and even then, some of those are pushed to userspace as well. There's more to it than that. It is also for certain interfaces that makes more sense in kernel space, or are there for historical reasons. For instance, a filesystem could be done in userspace, with a userspace library, but even FUSE uses the kernel for its interface, rather than (say) glibc. >>> NOTE: Even the FUSE project, which >>> is in _userspace_ (as opposed to the massive kernelspace mess of >>> reiser4), >>> is taking a lot of flak for changing UNIX semantics, so I see no reason >>> why Reiser4 should be special. >> >> >> Right. Kernel people hate change. Got it. >> >> No, seriously, I have to respect the history involved, which is why I'm >> not pretending to know more than I do. > > There is a good reason to hate change (at least without thorough evidence > to back it up). Even small changes can break things in subtle ways. Big > changes can tend to wreak kernel-global (and even userspace-global) havoc. Right on all points. Just remember that some change is good. Why do we have ALSA now? Everything a user can do with ALSA, they can do with OSS, AFAIK. >>> Ok, good. That's one of the first issues. A _lot_ of applications >>> would get themselves thoroughly confused at that '...' interface, not >>> to mention a lot of sysadmins :-D. >>> >> >> Not as much as you think. Unless a lot of applications can't handle >> opening or saving files that have '...' in the pathname? > > > If those files are as special as you lead me to believe, even a simple > "tar -cjvf foo.tar.bz2 foo" would break horribly, because it would tar up > all sorts of strange special files that shouldn't be included. Worse, > when I untar that archive on the filesystem, it will overwrite those > files, > which probably should be overwritten even less than they should be stored > in an archive. Point well taken, although sometimes that's exactly what you want. You no longer need the -p flag to Tar that way, although you would want its opposite, so that you can ignore '...' > >> The sysadmin problem doesn't worry me so much. > > > Personally, I know several sysadmins who, never having heard of Reiser4 > but using it anyways, would get scared when all of the '...' directories > started showing up, thinking that some cracker had gotten a module loaded > in their kernel. I'm not sure if the '...' or '..metas' directory is currently available in Reiser4, but at one point it was the default, then it got changed to a mount option which is off by default. > If you can, please avoid overloading existing semantics > in filesystems. You can either claim that your filesystem is POSIXy, > (similar to ext3, xfs, etc), or you can claim that it's not compatible by > design (AFS, Coda) or you can claim that it's a completely virtualized > interface (procfs, sysfs, CKRM fs, etc) and doesn't care about POSIX in > any case. Reiser4 definitely isn't the third, but neither is it either of > the first two, because it would break standard operations. Reiser4 is attempting to extend POSIX without breaking it too much. >> Agreed. I don't know enough about VFS to propose one, though. I think >> others are working on that, finally. > > When the VFS work gets done, this discussion can move on, otherwise, we > are stuck at an impasse. Not my department. It seems that people are agreeing on what needs to be done, though. >> Not too much, I hope. >> >> Although being able to use different keys on a per-file basis would be >> nice. I'm not sure if that would work in the existing system. Not to >> mention that keys would also be handlable with '...', if/when it >> happens. > > > I think the '...' is just a bad idea in general, because it breaks tar > and such. An interface like this might be simpler to design and use: > > # mkdir /attr > # mount -t attrfs attrfs /attr > > /attr/fd/$FD_NUM == '...' directory for a filedescriptor > /attr/fs/$DEV_NUM/$INODE_NUM == '...' directory for an inode > > It would be usable from a shell with a simple "getattrpath" command > which returns "fs/$DEV_NUM/$INODE_NUM" by stat-ing any given path. Still pretty annoying, but maybe a good idea, especially if the shell gets extended to support it. Not sure I like using inode numbers, though -- I like the idea of being able to symlink to stuff inside the meta-file dir. Actually, I like this. Give me some time to let it percolate. Hans, thoughts? Seems to be namespace fragmentation, but seems usable, less breakage, and so on. And should it be /attr or /meta? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsCLXXgHNmZLgCUhAQIXjQ/+OQc+KqDl+ybutM6Ba2KmaiwAJTdpIi40 DAO6n/vRkdO87r0vLU+g+3kpWyTqkIMrvGmbvm2dle/d+Q5yZc3fc3Lmg9tuwXyO GDMA/Lhx/1ckVX3EtcQ8PUomhcCvCFebR/y8IuNgEEthsHcBHPaZD32xkke9Q7u7 0zUbFCry5mgiN3EuSGMy5q4ZvvETUteK/xoEcTGAhh3K3JMTp0N8dHGR8XkGb+Z9 qFG3mYu8g2uRAsZ6hFScQ/rf4j3fu9012Q0SxvEZoTLTNJjFnZhSRqyNSY4i24iV OVYfstJrnpEWEAeEn29sTk0Pp5cHc8dh/Ure0MCOVHZUnMVD9xy259+3ivavePPn N27Z7dMmqvf4bKdjqa35tK7bao78Of8EkGeW3v61VJe3qsG45SwpiLmYG/uEvtIw u/9e/RlWQYyNS9k8eIxY6rjcrPjL4HErFiHmcw36wUmjyDqNYDlKtfsq0+KDGfOe 7I1G7TQc7K3uJu41XEqsG3IVQpY3o6bW8m7WyicRoOg3Qco+g/cjBBERmfAZh65W t+t7tzk4p+UZfvSiqSfkRnm2CITrqFC0VW2c1pk42BmY0rJPfzRmVU5oBGlH1Avd j75UFJi8qwm4d2S1InX8bxPoLaZpSjQGPAbpcKpZaaiM5mB3pfj62UVQV5hks9lt QNzi7/gMmto= =I2R/ -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 23:27 ` David Masover @ 2005-06-28 2:21 ` Hubert Chan 2005-06-28 2:59 ` Kyle Moffett 2005-06-28 20:22 ` David Masover 2005-06-28 2:34 ` Lee Revell 1 sibling, 2 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-28 2:21 UTC (permalink / raw) To: David Masover Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Mon, 27 Jun 2005 18:27:26 -0500, David Masover <ninja@slaphack.com> said: > Kyle Moffett wrote: >> I think the '...' is just a bad idea in general, because it breaks >> tar and such. An interface like this might be simpler to design and >> use: >> >> # mkdir /attr >> # mount -t attrfs attrfs /attr >> >> /attr/fd/$FD_NUM == '...' directory for a filedescriptor >> /attr/fs/$DEV_NUM/$INODE_NUM == '...' directory for an inode The most obvious (at least obvious for me) complaint about this is that the attributes are separate from the file. (In other words, it's a bit ugly. ;-) ) So if you want to backup a file, along with all its (extended) attributes (or substreams, or ... etc. ...), you need to backup the file, and find the appropriate /attr/fs/$DEV_NUM/$INODE_NUM and back that up. If I want to edit an attribute, I need to find $DEV_NUM and $INODE_NUM (which I have no idea how to do), rather than just "edit foo/.../extended_attribute". (Or using your "getattrpath" command, it would be a two-step process -- run getattrpath, then run editor -- rather than a one-step process. Of course, this is only mildly annoying.) It also exposes a difference between attributes and regular files. e.g. can I add attributes to an attribute? (say, I have a thumbnail attribute for the file ~/foo, and I want to add a mime-type attribute to that thumbnail attribute.) If you want to allow it, you have to be careful not to run into a big loop generating an infinite number of inodes in the attrfs. (e.g. /attr/fs/$(getattrpath /attr/fs/$(getattrpath ~/foo)/thumbnail)/mimetype -- attrfs shouldn't generate the inode for that until /attr/fs/$(getattrpath~/foo)/thumbnail is accessed, and maybe not even then...) That said, I prefer that interface over xattr or openat. >> It would be usable from a shell with a simple "getattrpath" command >> which returns "fs/$DEV_NUM/$INODE_NUM" by stat-ing any given path. > Still pretty annoying, but maybe a good idea, especially if the shell > gets extended to support it. Not sure I like using inode numbers, > though -- I like the idea of being able to symlink to stuff inside the > meta-file dir. The advantage of using inodes rather than pathnames is that it is robust with respect to file renaming/moving. It also allows things like adding attributes to symlinks (since the inode number for a symlink is different from the inode number for the file that it points to). You can still symlink. It just takes a little more effort to figure out what $DEV_NUM/$INODE_NUM to use. > Actually, I like this. Give me some time to let it percolate. > Hans, thoughts? Seems to be namespace fragmentation, but seems > usable, less breakage, and so on. And should it be /attr or /meta? For the mount point, it doesn't matter; it's up to the user. It's the attrfs or metafs or ???fs that matters (but which will greatly influence whether people user /attr or /meta). -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 2:21 ` Hubert Chan @ 2005-06-28 2:59 ` Kyle Moffett 2005-06-28 3:45 ` Hubert Chan 2005-06-28 20:07 ` David Masover 2005-06-28 20:22 ` David Masover 1 sibling, 2 replies; 620+ messages in thread From: Kyle Moffett @ 2005-06-28 2:59 UTC (permalink / raw) To: Hubert Chan Cc: David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Jun 27, 2005, at 22:21:35, Hubert Chan wrote: > On Mon, 27 Jun 2005 18:27:26 -0500, David Masover > <ninja@slaphack.com> said: >> Kyle Moffett wrote: >> >>> I think the '...' is just a bad idea in general, because it breaks >>> tar and such. An interface like this might be simpler to design and >>> use: >>> >>> # mkdir /attr >>> # mount -t attrfs attrfs /attr >>> >>> /attr/fd/$FD_NUM == '...' directory for a filedescriptor >>> /attr/fs/$DEV_NUM/$INODE_NUM == '...' directory for an inode > > The most obvious (at least obvious for me) complaint about this is > that > the attributes are separate from the file. (In other words, it's a > bit > ugly. ;-) ) So if you want to backup a file, along with all its > (extended) attributes (or substreams, or ... etc. ...), you need to > backup the file, and find the appropriate /attr/fs/$DEV_NUM/$INODE_NUM > and back that up. If I want to edit an attribute, I need to find > $DEV_NUM and $INODE_NUM (which I have no idea how to do), rather than > just "edit foo/.../extended_attribute". (Or using your "getattrpath" > command, it would be a two-step process -- run getattrpath, then run > editor -- rather than a one-step process. Of course, this is only > mildly annoying.) These are not really "attributes" so much as they are "metadata", for example, a "contents" subdirectory, if one existed, would be based on the original file, and therefore non-unique, but would be looked up based on information about the original file. > It also exposes a difference between attributes and regular files. > e.g. can I add attributes to an attribute? (say, I have a thumbnail > attribute for the file ~/foo, and I want to add a mime-type > attribute to > that thumbnail attribute.) If you want to allow it, you have to be > careful not to run into a big loop generating an infinite number of > inodes in the attrfs. (e.g. > /attr/fs/$(getattrpath /attr/fs/$(getattrpath ~/foo)/thumbnail)/ > mimetype > -- attrfs shouldn't generate the inode for that until > /attr/fs/$(getattrpath~/foo)/thumbnail is accessed, and maybe not even > then...) Agreed. I discuss this more below > That said, I prefer that interface over xattr or openat. Absolutely. On the other hand, that's not to say it can't be improved. > >>> It would be usable from a shell with a simple "getattrpath" command >>> which returns "fs/$DEV_NUM/$INODE_NUM" by stat-ing any given path. > >> Still pretty annoying, but maybe a good idea, especially if the shell >> gets extended to support it. Not sure I like using inode numbers, >> though -- I like the idea of being able to symlink to stuff inside >> the >> meta-file dir. >> > > The advantage of using inodes rather than pathnames is that it is > robust > with respect to file renaming/moving. It also allows things like > adding attributes to symlinks (since the inode number for a symlink is > different from the inode number for the file that it points to). > > You can still symlink. It just takes a little more effort to > figure out > what $DEV_NUM/$INODE_NUM to use. Also, unlike a symlink, if the path doesn't change and the file does, it will break, also, if the file is removed and another created elsewhere, it will be redirected improperly. Perhaps a new symlink syntax is needed to allow attribute specification (Ick, more changes :-\). > >> Hans, thoughts? Seems to be namespace fragmentation, but seems >> usable, less breakage, and so on. And should it be /attr or /meta? > > For the mount point, it doesn't matter; it's up to the user. It's the > attrfs or metafs or ???fs that matters (but which will greatly > influence > whether people user /attr or /meta). "meta" seems the more descriptive name. There should also probably be a somewhat standardized location for this, such that programs can locate it without much trouble. This mechanism would be usable from all FSs, and could be built into the VFS. Also, it would allow one to access the meta data of meta data (if supported by the filesystem, and possibly only through the file descriptor lookup, due to numbering limitations.) Cheers, Kyle Moffett -- There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 2:59 ` Kyle Moffett @ 2005-06-28 3:45 ` Hubert Chan 2005-06-28 4:38 ` Kyle Moffett 2005-06-28 20:07 ` David Masover 1 sibling, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-06-28 3:45 UTC (permalink / raw) To: Kyle Moffett Cc: David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Mon, 27 Jun 2005 22:59:24 -0400, Kyle Moffett <mrmacman_g4@mac.com> said: > On Jun 27, 2005, at 22:21:35, Hubert Chan wrote: >> On Mon, 27 Jun 2005 18:27:26 -0500, David Masover >> <ninja@slaphack.com> said: >>> Kyle Moffett wrote: [...] >>>> /attr/fd/$FD_NUM == '...' directory for a filedescriptor >>>> /attr/fs/$DEV_NUM/$INODE_NUM == '...' directory for an inode [...] > These are not really "attributes" so much as they are "metadata", for > example, a "contents" subdirectory, if one existed, would be based on > the original file, and therefore non-unique, but would be looked up > based on information about the original file. I think for most people on the reiser-fs list, the '...' directory represents an interface to many things including - automatic aggregating/tar/untar/compress - a different interface to stat data - adding extended attributes/substreams/acls/etc. - anything else you might imagine (I think this is your understanding too? Just double-checking.) So some of that stuff would be separate from the file. (Separate in the sense that it's not generated from the file's binary data.) Personally, I don't like it all being in one directory, but it's not that important. >> You can still symlink. It just takes a little more effort to figure >> out what $DEV_NUM/$INODE_NUM to use. > Also, unlike a symlink, if the path doesn't change and the file does, > it will break, also, if the file is removed and another created > elsewhere, it will be redirected improperly. Perhaps a new symlink > syntax is needed to allow attribute specification (Ick, more changes > :-\). I think those breakages are acceptable. (IMHO) In other words, I think it would not occur very often without the user being aware of it, and should very rarely result in catastrophic effects. One other minor annoyance is it isn't easy to go backwards from the ... directory to the file. e.g. if I have a symlink that points to /attr/fs/2/92036, I don't know what file's attributes that refers to. Hopefully I'm sane enough to give the symlink a descriptive enough name... >>> Hans, thoughts? Seems to be namespace fragmentation, but seems >>> usable, less breakage, and so on. And should it be /attr or /meta? >> For the mount point, it doesn't matter; it's up to the user. It's >> the attrfs or metafs or ???fs that matters (but which will greatly >> influence whether people user /attr or /meta). > "meta" seems the more descriptive name. There should also probably be > a somewhat standardized location for this, such that programs can > locate it without much trouble. Agreed. Ultimately, it's the user's decision where to put it, but probably 99.99999% of all people will put it in the same place. Just like you could put procfs or sysfs somewhere other than /proc or /sys if you really wanted to, but nobody does that. > This mechanism would be usable from all FSs, and could be built into > the VFS. Also, it would allow one to access the meta data of meta > data (if supported by the filesystem, and possibly only through the > file descriptor lookup, due to numbering limitations.) One other issue is that the attrfs/metafs needs to communicate with the other filesystem somehow. It needs to know if the filesystem can handle storing of extended attributes/substreams/etc. so that it knows whether or not to allow those interfaces. In my /attr/fs/$(getattrpath /attr/fs/$(getattrpath ~/foo)/thumbnail)/mimetype example, it needs to be smart enough to store that in ~/foo's filesystem. etc. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 3:45 ` Hubert Chan @ 2005-06-28 4:38 ` Kyle Moffett 2005-06-28 5:30 ` Hubert Chan 0 siblings, 1 reply; 620+ messages in thread From: Kyle Moffett @ 2005-06-28 4:38 UTC (permalink / raw) To: Hubert Chan Cc: David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Jun 27, 2005, at 23:45:00, Hubert Chan wrote: > On Mon, 27 Jun 2005 22:59:24 -0400, Kyle Moffett > <mrmacman_g4@mac.com> said: >> /attr/fd/$FD_NUM == '...' directory for a filedescriptor >> /attr/fs/$DEV_NUM/$INODE_NUM == '...' directory for an inode > > [...] > >> These are not really "attributes" so much as they are "metadata", for >> example, a "contents" subdirectory, if one existed, would be based on >> the original file, and therefore non-unique, but would be looked up >> based on information about the original file. > > I think for most people on the reiser-fs list, the '...' directory > represents an interface to many things including > - automatic aggregating/tar/untar/compress > - a different interface to stat data > - adding extended attributes/substreams/acls/etc. > - anything else you might imagine > (I think this is your understanding too? Just double-checking.) > So some of that stuff would be separate from the file. (Separate > in the > sense that it's not generated from the file's binary data.) Ok. Those things should probably be divided up. Stuff like POSIX extended attributes and such that have existing interfaces should use those. Other types of data should chose other interfaces that make the most sense for that type of data. I think that the /meta fs should probably only be used when the data is generated from the existing file or directory, and perhaps a few other cases. > Personally, I don't like it all being in one directory, but it's not > that important. I agree with the first, but I think that the implementation and usage confusion will be one of the barriers preventing '...' or metafs merge. >> Also, unlike a symlink, if the path doesn't change and the file does, >> it will break, also, if the file is removed and another created >> elsewhere, it will be redirected improperly. Perhaps a new symlink >> syntax is needed to allow attribute specification (Ick, more changes >> :-\). > > I think those breakages are acceptable. (IMHO) In other words, I > think > it would not occur very often without the user being aware of it, and > should very rarely result in catastrophic effects. Agreed. On the other hand, perhaps filesystems that support such meta data should internally assign the appropriate numbering to make it work nicely. > One other minor annoyance is it isn't easy to go backwards from the > ... directory to the file. e.g. if I have a symlink that points to > /attr/fs/2/92036, I don't know what file's attributes that refers to. > Hopefully I'm sane enough to give the symlink a descriptive enough > name... I don't see this occurring often enough to be a major problem, and in any case inodes are not path-exclusive (think hardlinks). If they have the filedescriptor open, however, they could use /proc/$PID/* to figure out where the file is. >> "meta" seems the more descriptive name. There should also >> probably be >> a somewhat standardized location for this, such that programs can >> locate it without much trouble. > > Agreed. Ultimately, it's the user's decision where to put it, but > probably 99.99999% of all people will put it in the same place. Just > like you could put procfs or sysfs somewhere other than /proc or / > sys if > you really wanted to, but nobody does that. Yeah. I was referring more to calling it "metafs" than "attrfs", because it seems it would be more than just attributes. > One other issue is that the attrfs/metafs needs to communicate with > the > other filesystem somehow. It needs to know if the filesystem can > handle > storing of extended attributes/substreams/etc. so that it knows > whether > or not to allow those interfaces. Those would be tied to file descriptor (/meta/fd) or filesystem device ID (/meta/fs/$DEV), and would therefore be accessible from metafs in a similar way to how filedescriptor path info is available from procfs. > In my > /attr/fs/$(getattrpath /attr/fs/$(getattrpath ~/foo)/thumbnail)/ > mimetype > example, it needs to be smart enough to store that in ~/foo's > filesystem. etc. Agreed. On another note, it's nice to see the flamewar has died out and several technical discussions are taking place on various levels :-D. Cheers, Kyle Moffett -- There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 4:38 ` Kyle Moffett @ 2005-06-28 5:30 ` Hubert Chan 2005-06-28 6:01 ` Kyle Moffett 0 siblings, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-06-28 5:30 UTC (permalink / raw) To: Kyle Moffett Cc: David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Agree with most of the stuff you wrote. On Tue, 28 Jun 2005 00:38:38 -0400, Kyle Moffett <mrmacman_g4@mac.com> said: > On Jun 27, 2005, at 23:45:00, Hubert Chan wrote: >> I think for most people on the reiser-fs list, the '...' directory >> represents an interface to many things including >> - automatic aggregating/tar/untar/compress >> - a different interface to stat data >> - adding extended attributes/substreams/acls/etc. >> - anything else you might imagine > Ok. Those things should probably be divided up. Stuff like POSIX > extended attributes and such that have existing interfaces should use > those. One of the claimed advantages of the '...' interface is that everything is editable as a file. So if someone wanted to edit the description extended attribute for foo.txt, he would just run "[editor] foo.txt/.../description" (or "[editor] /meta/fs/$(getattrpath foo.txt)/description"), instead of needing to use some special purpose editor. It works well for things like being able to use Gimp to edit a thumbnail or icon attribute. Of course, it shouldn't be too hard to provide both interfaces, as long as you get locking and caching right... The Reiser4 file-as-dir interface is also supposed to be more flexible than POSIX extended attributes. For example adding attributes to attributes (e.g. adding a mimetype attribute to a thumbnail attribute). The point of it is to present many different things (extended attributes, file manipulation magic like automatic compress, etc.) through a single interface that everyone knows how to use (i.e. regular files) so that people can use regular programs to edit or manipulate them. The inspiration, I think, was the MacOS X/NeXTSTEP bundle format. For example, MacOS X/NeXTSTEP .app file is actually a directory that behaves much like an executable file (double-clicking a .app file in the Finder launches the application, instead of opening the directory). However, it is in reality a directory that contains many things that could be thought of as extended attributes (such as the application icon, information about the application, etc.). Since the application icon is a real file, it can be edited by normal graphics editors (not like Windows programs, where you need a special icon editor). And since it's inside the .app directory, it's attached to the application (not like Linux, where the program is in /usr/bin, and the icon is in /usr/share/pixmaps), so it makes package management easier (to delete an application, just delete the .app file -- don't need to look in /usr/share/pixmaps for the icon and delete it). > Other types of data should chose other interfaces that make the most > sense for that type of data. I think that the /meta fs should > probably only be used when the data is generated from the existing > file or directory, and perhaps a few other cases. [...] >> One other minor annoyance is it isn't easy to go backwards from the >> ... directory to the file. e.g. if I have a symlink that points to >> /attr/fs/2/92036, I don't know what file's attributes that refers to. >> Hopefully I'm sane enough to give the symlink a descriptive enough >> name... > I don't see this occurring often enough to be a major problem, I don't see that this should be a major problem either, but I thought it was worth bringing up. > and in any case inodes are not path-exclusive (think hardlinks). If > they have the filedescriptor open, however, they could use > /proc/$PID/* to figure out where the file is. Although I guess if they have a file descriptor open, they probably have the original filename lying around somewhere... [...] > On another note, it's nice to see the flamewar has died out and > several technical discussions are taking place on various levels :-D. Agreed! :-D -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 5:30 ` Hubert Chan @ 2005-06-28 6:01 ` Kyle Moffett 2005-06-28 17:51 ` Hubert Chan 0 siblings, 1 reply; 620+ messages in thread From: Kyle Moffett @ 2005-06-28 6:01 UTC (permalink / raw) To: Hubert Chan Cc: David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Jun 28, 2005, at 01:30:08, Hubert Chan wrote: > On Tue, 28 Jun 2005 00:38:38 -0400, Kyle Moffett > <mrmacman_g4@mac.com> said: >> Ok. Those things should probably be divided up. Stuff like POSIX >> extended attributes and such that have existing interfaces should use >> those. > > One of the claimed advantages of the '...' interface is that > everything > is editable as a file. So if someone wanted to edit the description > extended attribute for foo.txt, he would just run > "[editor] foo.txt/.../description" (or > "[editor] /meta/fs/$(getattrpath foo.txt)/description"), instead of > needing to use some special purpose editor. It works well for things > like being able to use Gimp to edit a thumbnail or icon attribute. I don't disagree with the thumbnail/icon/description, but things like POSIX acls and extended attributes have _existing_ interfaces which should be used. I don't deny them the right to add other interfaces later, but such should be a secondary or tertiary patch, after the rest of the stuff is in. In any case, if we were to provide an interface by which one could $EDITOR the POSIX ACLs, it should be done in the VFS where all filesystems can share it. > The inspiration, I think, was the MacOS X/NeXTSTEP bundle format. For > example, MacOS X/NeXTSTEP .app file is actually a directory that > behaves > much like an executable file (double-clicking a .app file in the > Finder > launches the application, instead of opening the directory). However, > it is in reality a directory that contains many things that could be > thought of as extended attributes (such as the application icon, > information about the application, etc.). Since the application > icon is > a real file, it can be edited by normal graphics editors (not like > Windows programs, where you need a special icon editor). And since > it's > inside the .app directory, it's attached to the application (not like > Linux, where the program is in /usr/bin, and the icon is in > /usr/share/pixmaps), so it makes package management easier (to > delete an > application, just delete the .app file -- don't need to look in > /usr/share/pixmaps for the icon and delete it). The key difference here is that Mac OS X does all of the bundle mess in userspace where it belongs. :-D (I know, I use it daily) I think that part of Reiser4 needs more review than it's been given at present. Cheers, Kyle Moffett -- There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 6:01 ` Kyle Moffett @ 2005-06-28 17:51 ` Hubert Chan 2005-06-28 19:32 ` David Masover 2005-06-28 19:59 ` Kyle Moffett 0 siblings, 2 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-28 17:51 UTC (permalink / raw) To: Kyle Moffett Cc: David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Tue, 28 Jun 2005 02:01:12 -0400, Kyle Moffett <mrmacman_g4@mac.com> said: > On Jun 28, 2005, at 01:30:08, Hubert Chan wrote: >> On Tue, 28 Jun 2005 00:38:38 -0400, Kyle Moffett >> <mrmacman_g4@mac.com> said: >>> Ok. Those things should probably be divided up. Stuff like POSIX >>> extended attributes and such that have existing interfaces should >>> use those. >> One of the claimed advantages of the '...' interface is that >> everything is editable as a file. So if someone wanted to edit the >> description extended attribute for foo.txt, he would just run >> "[editor] foo.txt/.../description" (or "[editor] >> /meta/fs/$(getattrpath foo.txt)/description"), instead of needing to >> use some special purpose editor. It works well for things like being >> able to use Gimp to edit a thumbnail or icon attribute. > I don't disagree with the thumbnail/icon/description, but things like > POSIX acls and extended attributes have _existing_ interfaces which > should be used. OK. I agree with that. Of course, Reiser4 can always present both interfaces, just like it presented two interfaces to the stat data -- the regular interface and the metas (now '...') interface -- before file-as-dir got disabled by default. > I don't deny them the right to add other interfaces later, but such > should be a secondary or tertiary patch, after the rest of the stuff > is in. In any case, if we were to provide an interface by which one > could $EDITOR the POSIX ACLs, it should be done in the VFS where all > filesystems can share it. I don't know if VFS is the right place for it, but I agree that it would be good to make it accessible to all filesystems. >> The inspiration, I think, was the MacOS X/NeXTSTEP bundle format. >> For example, MacOS X/NeXTSTEP .app file is actually a directory that >> behaves much like an executable file (double-clicking a .app file in >> the Finder launches the application, instead of opening the >> directory). However, it is in reality a directory that contains many >> things that could be thought of as extended attributes (such as the >> application icon, information about the application, etc.). Since >> the application icon is a real file, it can be edited by normal >> graphics editors (not like Windows programs, where you need a special >> icon editor). And since it's inside the .app directory, it's >> attached to the application (not like Linux, where the program is in >> /usr/bin, and the icon is in /usr/share/pixmaps), so it makes package >> management easier (to delete an application, just delete the .app >> file -- don't need to look in /usr/share/pixmaps for the icon and >> delete it). > The key difference here is that Mac OS X does all of the bundle mess > in userspace where it belongs. :-D (I know, I use it daily) Yes. It's handled by NSWorkspace which is approximately equivalent to this sort of thing being handled by GnomeVFS or the KDE equivalent. Of course the problem with handling it in userspace is that behaviour isn't uniform -- applications that don't use NSWorkspace (e.g. some command-line utils, programs ported over from UNIX, etc.) won't have the same behaviour. Whether or not that is an actual problem seems to be debatable. (I don't use MacOS X, but I've done some programming in GNUstep.) Another problem is that it only works with bundle files. e.g. I can't add an icon to a regular txt file. Tiger now supports xattrs, which you could use for that functionality, but then we run into the problem again of not being able to edit it with regular applications. > I think that part of Reiser4 needs more review than it's been given at > present. Looking forward to the flamewar that happens when Namesys decides that file-as-dir is ready to be turned on again. ;-) -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 17:51 ` Hubert Chan @ 2005-06-28 19:32 ` David Masover 2005-06-28 19:57 ` Hubert Chan 2005-06-28 20:03 ` Kyle Moffett 2005-06-28 19:59 ` Kyle Moffett 1 sibling, 2 replies; 620+ messages in thread From: David Masover @ 2005-06-28 19:32 UTC (permalink / raw) To: Hubert Chan Cc: Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hubert Chan wrote: > On Tue, 28 Jun 2005 02:01:12 -0400, Kyle Moffett <mrmacman_g4@mac.com> said: > > >>On Jun 28, 2005, at 01:30:08, Hubert Chan wrote: >> >>>On Tue, 28 Jun 2005 00:38:38 -0400, Kyle Moffett >>><mrmacman_g4@mac.com> said: >>> >>>>Ok. Those things should probably be divided up. Stuff like POSIX >>>>extended attributes and such that have existing interfaces should >>>>use those. >>> >>> One of the claimed advantages of the '...' interface is that >>>everything is editable as a file. So if someone wanted to edit the >>>description extended attribute for foo.txt, he would just run >>>"[editor] foo.txt/.../description" (or "[editor] >>>/meta/fs/$(getattrpath foo.txt)/description"), instead of needing to >>>use some special purpose editor. It works well for things like being >>>able to use Gimp to edit a thumbnail or icon attribute. > > >>I don't disagree with the thumbnail/icon/description, but things like >>POSIX acls and extended attributes have _existing_ interfaces which >>should be used. Any existing interface should be supported, but Reiser4 seems to want to replace all existing interfaces except a direct open() and a proposed new system call which opens lots of small files at once, efficiently. The two approaches are not mutually exclusive, though. > OK. I agree with that. Of course, Reiser4 can always present both > interfaces, just like it presented two interfaces to the stat data -- > the regular interface and the metas (now '...') interface -- before > file-as-dir got disabled by default. The proposal I like right now is /meta, as a separate FS, which provides access to what would be the '...' interface. Check the archives. >>I don't deny them the right to add other interfaces later, but such >>should be a secondary or tertiary patch, after the rest of the stuff >>is in. In any case, if we were to provide an interface by which one >>could $EDITOR the POSIX ACLs, it should be done in the VFS where all >>filesystems can share it. > > > I don't know if VFS is the right place for it, but I agree that it would > be good to make it accessible to all filesystems. You put it in /meta, which is available to all filesystems. >>I think that part of Reiser4 needs more review than it's been given at >>present. > > > Looking forward to the flamewar that happens when Namesys decides that > file-as-dir is ready to be turned on again. ;-) Namesys still hasn't commented directly on the /meta proposal, have they? That would avoid the flamewar altogether. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsGl6HgHNmZLgCUhAQK+OQ//Zhj9DPyXRaASbTmnj2oPcWYWHzY5+M8O n0e7PsFPtVpb6mduDPYu91WTGYooPnicVOBBQezWUOVFYMN4SWxyVunTpDYbvTGn xVwSLTjb5/AZ7dIuklVfverN72nwmLURCJjGZ41iqyylHJT/nCufYZHhQIPa15Pu o0IzYlQahS3uxuYVJubDOms0G/K2pDuoFGuJJkk+JTPCbf9+2UEqjiUrL3FTqeG+ 5RQ6wVeinz5CFPrUWmxl0CdipGqx3quCCYlBav31uRDJm2VwwqZNTxeEHWvZN/Y9 tXnkcVSVJkoaJtbGHUvu61g/TAs8ckxlEA9lnaG3SEhNNU5ZSm0aprZJTeRvs81e iFxzlOKdgZA11GNx8rnj04Ii5MJapgQhDlz5CbOCxvOm6SkZZul+3n6VHxlCJzH2 pQD/GFcynPAbZYitOAyTWAU9K+fZ98dbCkDKJRLMbT8uiZwoc/m59m9ZniDVEKAn IjUMFrIllpq9F8AZefEFqRV6l9vjM2vam5OS2HC4R+v753JEVKuvTjbJkR7mB9iv zEGPBciVOjZAWuWmeQVartnDwx4ic2SECxv25w32wcPcrXUs9w0Z/hpIUfG7EAUl z4dbGlKlXWybq8JLC6ojmL+cgKFAYs+4yfZvE73H9XHlhh1HrGbMc6sd342NCFJN WjYRbkxGym0= =LZpe -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 19:32 ` David Masover @ 2005-06-28 19:57 ` Hubert Chan 2005-06-28 20:03 ` Kyle Moffett 1 sibling, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-28 19:57 UTC (permalink / raw) To: David Masover Cc: Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Tue, 28 Jun 2005 14:32:56 -0500, David Masover <ninja@slaphack.com> said: > Hubert Chan wrote: >> On Tue, 28 Jun 2005 02:01:12 -0400, Kyle Moffett >> <mrmacman_g4@mac.com> said: >>> I don't deny them the right to add other interfaces later, but such >>> should be a secondary or tertiary patch, after the rest of the stuff >>> is in. In any case, if we were to provide an interface by which one >>> could $EDITOR the POSIX ACLs, it should be done in the VFS where all >>> filesystems can share it. >> I don't know if VFS is the right place for it, but I agree that it >> would be good to make it accessible to all filesystems. Sorry, "accessible to all filesystems" is incorrect terminology (it's backwards). What I meant was something more like "able to access extended data from all filesystems". But I think everyone understands what I meant. > You put it in /meta, which is available to all filesystems. >> Looking forward to the flamewar that happens when Namesys decides >> that file-as-dir is ready to be turned on again. ;-) > Namesys still hasn't commented directly on the /meta proposal, have > they? That would avoid the flamewar altogether. Well, we could still have a flamewar about whether metafs is useful, what type of functionality it should provide, etc. I'm sure we could find something to argue about. ;-) >From a purely philosophical/aesthetic/theoretical point of view, I like file-as-dir much better than anything else, and would be ecstatic it if Namesys was able to make it work. From a practical/technical point of view, metafs may be an acceptable compromise. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-28 19:57 ` Hubert Chan 0 siblings, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-28 19:57 UTC (permalink / raw) To: David Masover Cc: Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Tue, 28 Jun 2005 14:32:56 -0500, David Masover <ninja@slaphack.com> said: > Hubert Chan wrote: >> On Tue, 28 Jun 2005 02:01:12 -0400, Kyle Moffett >> <mrmacman_g4@mac.com> said: >>> I don't deny them the right to add other interfaces later, but such >>> should be a secondary or tertiary patch, after the rest of the stuff >>> is in. In any case, if we were to provide an interface by which one >>> could $EDITOR the POSIX ACLs, it should be done in the VFS where all >>> filesystems can share it. >> I don't know if VFS is the right place for it, but I agree that it >> would be good to make it accessible to all filesystems. Sorry, "accessible to all filesystems" is incorrect terminology (it's backwards). What I meant was something more like "able to access extended data from all filesystems". But I think everyone understands what I meant. > You put it in /meta, which is available to all filesystems. >> Looking forward to the flamewar that happens when Namesys decides >> that file-as-dir is ready to be turned on again. ;-) > Namesys still hasn't commented directly on the /meta proposal, have > they? That would avoid the flamewar altogether. Well, we could still have a flamewar about whether metafs is useful, what type of functionality it should provide, etc. I'm sure we could find something to argue about. ;-) From a purely philosophical/aesthetic/theoretical point of view, I like file-as-dir much better than anything else, and would be ecstatic it if Namesys was able to make it work. From a practical/technical point of view, metafs may be an acceptable compromise. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 19:32 ` David Masover 2005-06-28 19:57 ` Hubert Chan @ 2005-06-28 20:03 ` Kyle Moffett 1 sibling, 0 replies; 620+ messages in thread From: Kyle Moffett @ 2005-06-28 20:03 UTC (permalink / raw) To: David Masover Cc: Hubert Chan, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Jun 28, 2005, at 15:32:56, David Masover wrote: > Hubert Chan wrote: >> On Tue, 28 Jun 2005 02:01:12 -0400, Kyle Moffett >> <mrmacman_g4@mac.com> said: >>> I don't disagree with the thumbnail/icon/description, but things >>> like >>> POSIX acls and extended attributes have _existing_ interfaces which >>> should be used. > > Any existing interface should be supported, but Reiser4 seems to > want to > replace all existing interfaces except a direct open() and a proposed > new system call which opens lots of small files at once, efficiently. > > The two approaches are not mutually exclusive, though. I don't think most kernel developers are likely to favor an approach which smashes a bunch of unrelated data together. Linux syscall overhead is incredibly low, so I don't see it being much of a problem. On the other hand, a simple syscall which smashes together a bunch of related data would not be that big of an issue either. Let's wait for Namesys to tinker and come back with a new patch before commenting further. Cheers, Kyle Moffett -- Somone asked me why I work on this free (http://www.fsf.org/philosophy/) software stuff and not get a real job. Charles Shultz had the best answer: "Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life." -- Charles Shultz ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 17:51 ` Hubert Chan 2005-06-28 19:32 ` David Masover @ 2005-06-28 19:59 ` Kyle Moffett 2005-06-29 1:40 ` Hubert Chan 1 sibling, 1 reply; 620+ messages in thread From: Kyle Moffett @ 2005-06-28 19:59 UTC (permalink / raw) To: Hubert Chan Cc: David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Jun 28, 2005, at 13:51:04, Hubert Chan wrote: > On Tue, 28 Jun 2005 02:01:12 -0400, Kyle Moffett > <mrmacman_g4@mac.com> said: >> I don't disagree with the thumbnail/icon/description, but things like >> POSIX acls and extended attributes have _existing_ interfaces which >> should be used. > > OK. I agree with that. Of course, Reiser4 can always present both > interfaces, just like it presented two interfaces to the stat data -- > the regular interface and the metas (now '...') interface -- before > file-as-dir got disabled by default. Yeah, but let's get the normal interface working first and discuss what form the alternate one should take >> I don't deny them the right to add other interfaces later, but such >> should be a secondary or tertiary patch, after the rest of the stuff >> is in. In any case, if we were to provide an interface by which one >> could $EDITOR the POSIX ACLs, it should be done in the VFS where all >> filesystems can share it. > > I don't know if VFS is the right place for it, but I agree that it > would > be good to make it accessible to all filesystems. That's somewhat of a contradiction in terms. The whole point of the VFS is to hold all of the things that multiple filesystems want to share :-D. >> The key difference here is that Mac OS X does all of the bundle mess >> in userspace where it belongs. :-D (I know, I use it daily) > > Yes. It's handled by NSWorkspace which is approximately equivalent to > this sort of thing being handled by GnomeVFS or the KDE > equivalent. Of > course the problem with handling it in userspace is that behaviour > isn't > uniform -- applications that don't use NSWorkspace (e.g. some > command-line utils, programs ported over from UNIX, etc.) won't > have the > same behaviour. Whether or not that is an actual problem seems to be > debatable. (I don't use MacOS X, but I've done some programming in > GNUstep.) Personally, I think that the multiple views is a good thing. I like being able to "cd /Applications/Games/UnrealTournament2004.app/System" and mess with my game files, while double-clicking it in the Finder just starts it so I can get on with owning my friends :-D. > Another problem is that it only works with bundle files. e.g. I can't > add an icon to a regular txt file. Tiger now supports xattrs, > which you > could use for that functionality, but then we run into the problem > again > of not being able to edit it with regular applications. Maybe we just need better regular applications? I think that for the icon case, for the Samba/streams case, and for many others I'm probably forgetting, we should try to come up with a new "data-stream" VFS API, so that the icon data and other larger quantities can be stored in a filesystem without much effort. Such a layer could even be bridged onto existing filesystems via a VFS-wrapper bind-mount: # mount -t reiser4 /dev/hda1 /mnt1 # mount -t ext3 /dev/hda2 /mnt2 # cat $(metapath /mnt1/foo)/streams/description Some random file # cat $(metapath /mnt2/foo)/streams/description cat: Unsupported operation # mount -t none -o bind,streamify /mnt2 /mnt3 # cat $(metapath /mnt3/foo)/streams/description Another random file Such a wrapper interface might use the directory '...' to store files on the underlying filesystem, but I don't think that the meta interface itself should use those directories. Cheers, Kyle Moffett -----BEGIN GEEK CODE BLOCK----- Version: 3.12 GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$ L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+ PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r !y?(-) ------END GEEK CODE BLOCK------ ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 19:59 ` Kyle Moffett @ 2005-06-29 1:40 ` Hubert Chan 2005-06-29 5:09 ` Horst von Brand 2005-07-01 9:08 ` David Masover 0 siblings, 2 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-29 1:40 UTC (permalink / raw) To: Kyle Moffett Cc: David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Tue, 28 Jun 2005 15:59:03 -0400, Kyle Moffett <mrmacman_g4@mac.com> said: > On Jun 28, 2005, at 13:51:04, Hubert Chan wrote: >> I don't know if VFS is the right place for it, but I agree that it >> would be good to make it accessible to all filesystems. > That's somewhat of a contradiction in terms. The whole point of the > VFS is to hold all of the things that multiple filesystems want to > share :-D. VFS provides a common interface to the filesystem. I don't think metafs needs any VFS changes. It may be able to get by without making changes to the VFS, and if so, it shouldn't touch the VFS. It should just be its own separate filesystem. I imagine most of it could be implemented by a FUSE filesystem. > Personally, I think that the multiple views is a good thing. I like > being able to "cd /Applications/Games/UnrealTournament2004.app/System" > and mess with my game files, while double-clicking it in the Finder > just starts it so I can get on with owning my friends :-D. Of course. With file-as-dir, you can "cd /usr/games/tetris/..." and mess with the game files, or you can run "/usr/games/tetris" and get on with ... stacking blocks. The advantage of doing it in the kernel is that you don't need extra support from the applications (or GnomeVFS or KDE). So typing "/usr/games/tetris" from the command prompt does the same thing as double-clicking it in the file manager. Of course, this change does require file managers to understand default actions when it's ambiguous what to do on a double-click -- but MacOS X has that too, in the form of the "Open as Folder" option (or whatever it's called). >> Another problem is that it only works with bundle files. e.g. I >> can't add an icon to a regular txt file. Tiger now supports xattrs, >> which you could use for that functionality, but then we run into the >> problem again of not being able to edit it with regular applications. > Maybe we just need better regular applications? You mean patch them all so that they understand and can edit xattr/substreams/etc.? The file-as-dir interface is meant to avoid having to do that. metafs also avoids having to patch all the applications by exposing them as regular files. The example you give below isn't a new VFS API. It's metafs exposing xattrs/substreams/etc. through the regular file interface. (Perhaps we're just using different terminology.) > I think that for the icon case, for the Samba/streams case, and for > many others I'm probably forgetting, we should try to come up with a > new "data-stream" VFS API, so that the icon data and other larger > quantities can be stored in a filesystem without much effort. Such a > layer could even be bridged onto existing filesystems via a > VFS-wrapper bind-mount: > # mount -t reiser4 /dev/hda1 /mnt1 > # mount -t ext3 /dev/hda2 /mnt2 > # cat $(metapath /mnt1/foo)/streams/description > Some random file > # cat $(metapath /mnt2/foo)/streams/description > cat: Unsupported operation > # mount -t none -o bind,streamify /mnt2 /mnt3 > # cat $(metapath /mnt3/foo)/streams/description > Another random file > Such a wrapper interface might use the directory '...' to store files > on the underlying filesystem, but I don't think that the meta > interface itself should use those directories. Agreed. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 1:40 ` Hubert Chan @ 2005-06-29 5:09 ` Horst von Brand 2005-06-29 13:50 ` Douglas McNaught 2005-06-29 20:40 ` Hubert Chan 2005-07-01 9:08 ` David Masover 1 sibling, 2 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-29 5:09 UTC (permalink / raw) To: Hubert Chan Cc: Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hubert Chan <hubert@uhoreg.ca> wrote: [...] > Of course. With file-as-dir, you can "cd /usr/games/tetris/..." and > mess with the game files, or you can run "/usr/games/tetris" and get on > with ... stacking blocks. And doing "tar cf /dev/tape /usr/games/tetris" gives you a nice tangle of undecipherable junk. > The advantage of doing it in the kernel is > that you don't need extra support from the applications (or GnomeVFS or > KDE). I don't see any advantage here... it has to be implemented, and doing it in-kernel for one filesystem is /much/ harder than doing it in userland, where it'll probably work whatever the underlying filesystem is. Besides, why should the Tetris scores (or icon, or whatever) reside inside the executable, when said executable is perhaps shared by a bunch of machines on the network, each of which would like to keep its own? Or perhaps the sysadmin is terminally paranoid, and mounts /usr read-only (Hey, the FHS people defined what goes in there with /exactly/ that in mind too!). > So typing "/usr/games/tetris" from the command prompt does the > same thing as double-clicking it in the file manager. Right. And what should happen if I run the (logically equivalent) directory /home? Or /etc? What if I want to shove a directory into the tetris executable? Symlinks? Hard links to files inside? Outside? Symlinks from the outside in? Hardlinks? And if you now move that to another filesystem, or ship it to another machine? No, the answer "All that will be forbidden" is /not/ allowed, you want this stuff working "normally". "Unpack" the contents into real files and directories how? "Pack" a directory into one of those structured objects? What would be the /practical/ difference between the packed and unpacked forms? (If none, why do it in the first place? If the packed version has significant restrictions, why use it? If the packed version does have significant extra capabilities, why not just give those to regular objects?) Yet again, what somebody (I forget who) called the "Zero - one - infinity principle" (I think it was in the area of programming languages; which arguably are very complex user interfaces, just like what an operating system provides): It only makes sense giving zero, exactly one, or how many you wish for of some item/nesting. Here: Either files got /no/ strange things inside, or /exactly one/ (and then it becomes questionable, as there is no space for regular data anymore...), or whatever complex (sub)structure you want. But in the last case, there is /no/ difference between a file and a directory... and the whole setup is just mess for mess sake, nothing else. > Of course, this > change does require file managers to understand default actions when > it's ambiguous what to do on a double-click -- but MacOS X has that too, > in the form of the "Open as Folder" option (or whatever it's called). Right. For the sake of a filesystem among many on a minority operating system /all/ GUI programs have to be rewritten. And all command-line stuff. Just because. Please /do/ think the above through, giving reasonable answers for /all/ of the operations mentioned. Tell what the advantage of all this would be on a multiuser operating system, when files are shared via the net, when files are being handled from read-only media (or filesystems). When different users want different metadata (I'm interested in the names of members of the band playing a song, you might want a high-resolution image of the album cover, the next guy wants the song's lyrics translated into swahili, all for the MP3s on the shared server or on the CD each of us bought separately). Consider the scenario where all this works only on /a very few/ machines (no Sun or *BSD box will have this for a very long time, and to get a significant fraction of just Linux systems will take some time), for only a limited selection of tools (existing tools will have to be rewritten or at least adapted, that doesn't happen overnight). Factor in space and processing requirements for several, even hundreds, of concurrent users (or versions of metadata at least). How do you account for that? The 1 byte file with GiB of metadata where everybody and their pet iguana store their junk is handled how by quota systems? What should happen if I email such a file with several versions of metadata to someone? Do they get just my metadata, or everybody's? If I copy it to, say, a CD for safekeeping for myself? For system backup purposes? What if I copy such a file (with only my, or everybody's) metadata back from a backup? Or try to merge it with the version I took with my notebook on a trip, changing my parts while the otghers change theirs? I'd assume all of those will requiere special tools, or at least flags selecting the right operation or some other unspecified magic, and the "simple to use" just went out for lunch. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 5:09 ` Horst von Brand @ 2005-06-29 13:50 ` Douglas McNaught 2005-06-29 13:58 ` mjt 2005-06-29 20:40 ` Hubert Chan 1 sibling, 1 reply; 620+ messages in thread From: Douglas McNaught @ 2005-06-29 13:50 UTC (permalink / raw) To: Horst von Brand Cc: Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Horst von Brand <vonbrand@inf.utfsm.cl> writes: > Hubert Chan <hubert@uhoreg.ca> wrote: > >> Of course, this >> change does require file managers to understand default actions when >> it's ambiguous what to do on a double-click -- but MacOS X has that too, >> in the form of the "Open as Folder" option (or whatever it's called). > > Right. For the sake of a filesystem among many on a minority operating > system /all/ GUI programs have to be rewritten. And all command-line > stuff. Just because. I'll just note that the "applications bundled as directories" stuff on MacOS/NextStep is done completely in userspace--as far as the kernel is concerned, "Mail.app" is a regular directory. The file manager handles recognition and invocation of application bundles (and there is an 'open' shell command that does the same thing). -Doug ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 13:50 ` Douglas McNaught @ 2005-06-29 13:58 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-29 13:58 UTC (permalink / raw) To: Douglas McNaught Cc: Horst von Brand, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 900 bytes --] On Wed, Jun 29, 2005 at 09:50:27AM -0400, Douglas McNaught wrote: > >I'll just note that the "applications bundled as directories" stuff on >MacOS/NextStep is done completely in userspace--as far as the kernel >is concerned, "Mail.app" is a regular directory. The file manager >handles recognition and invocation of application bundles (and there >is an 'open' shell command that does the same thing). Note that MacOS has the monopoly on what they ship, Linux has a motherload of file managers and window systems and all. What pisses me off is the fact that Gnome and friends implement their own incompatible-with-others VFS's and automounters and stuff. Surely supporting this in the kernel and extending the LSB to require this is the best step to take without infringing anyone's freedom as such. *still pissed off about having to hassle an automatic mount* -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-29 13:58 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-29 13:58 UTC (permalink / raw) To: Douglas McNaught Cc: Horst von Brand, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 900 bytes --] On Wed, Jun 29, 2005 at 09:50:27AM -0400, Douglas McNaught wrote: > >I'll just note that the "applications bundled as directories" stuff on >MacOS/NextStep is done completely in userspace--as far as the kernel >is concerned, "Mail.app" is a regular directory. The file manager >handles recognition and invocation of application bundles (and there >is an 'open' shell command that does the same thing). Note that MacOS has the monopoly on what they ship, Linux has a motherload of file managers and window systems and all. What pisses me off is the fact that Gnome and friends implement their own incompatible-with-others VFS's and automounters and stuff. Surely supporting this in the kernel and extending the LSB to require this is the best step to take without infringing anyone's freedom as such. *still pissed off about having to hassle an automatic mount* -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 13:58 ` mjt @ 2005-06-29 17:19 ` Horst von Brand -1 siblings, 0 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-29 17:19 UTC (permalink / raw) To: Markus T rnqvist Cc: Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain, Size: 1532 bytes --] Markus Törnqvist <mjt@nysv.org> wrote: [...] > Note that MacOS has the monopoly on what they ship, Linux has a > motherload of file managers and window systems and all. Yep. Part of what is nice about it, too ;-) > What pisses me off is the fact that Gnome and friends implement > their own incompatible-with-others VFS's and automounters and > stuff. Then get them to agree on a common framework! They are trying hard to get other parts of the GUI work well together, so this isn't far off in wishfull thinking land. > Surely supporting this in the kernel and extending the LSB > to require this is the best step to take without infringing > anyone's freedom as such. Right. So then we have Gnome's way of doing things (Gnome isn't just for Linux!), KDE's way, XFCE's way, ... (ditto). Plus the kernel way. Flambee with a monthly thread on all reachable fora about "Why on &%@# does the %&~#@ GUI not use the *#%&@ kernel's way?!". This is /not/ the way of fxing that particular problem. Shoving random stuff into the kernel /can't/ force its use. At least not until Linux is the only Unixy system around, and that is still some way off. And when that has happened, the kernel's way must be /clearly/ better for /all/ users, or it won't matter. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-29 17:19 ` Horst von Brand 0 siblings, 0 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-29 17:19 UTC (permalink / raw) To: Markus T rnqvist Cc: Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Markus Törnqvist <mjt@nysv.org> wrote: [...] > Note that MacOS has the monopoly on what they ship, Linux has a > motherload of file managers and window systems and all. Yep. Part of what is nice about it, too ;-) > What pisses me off is the fact that Gnome and friends implement > their own incompatible-with-others VFS's and automounters and > stuff. Then get them to agree on a common framework! They are trying hard to get other parts of the GUI work well together, so this isn't far off in wishfull thinking land. > Surely supporting this in the kernel and extending the LSB > to require this is the best step to take without infringing > anyone's freedom as such. Right. So then we have Gnome's way of doing things (Gnome isn't just for Linux!), KDE's way, XFCE's way, ... (ditto). Plus the kernel way. Flambee with a monthly thread on all reachable fora about "Why on &%@# does the %&~#@ GUI not use the *#%&@ kernel's way?!". This is /not/ the way of fxing that particular problem. Shoving random stuff into the kernel /can't/ force its use. At least not until Linux is the only Unixy system around, and that is still some way off. And when that has happened, the kernel's way must be /clearly/ better for /all/ users, or it won't matter. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 17:19 ` Horst von Brand @ 2005-06-29 20:57 ` Hubert Chan -1 siblings, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-29 20:57 UTC (permalink / raw) To: Horst von Brand Cc: Douglas McNaught, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Wed, 29 Jun 2005 13:19:12 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: > Markus Törnqvist <mjt@nysv.org> wrote: [...] >> Note that MacOS has the monopoly on what they ship, Linux has a >> motherload of file managers and window systems and all. > Yep. Part of what is nice about it, too ;-) I guess that's why Apple doesn't want to make its X11 support too good. Otherwise the Mac world would get flooded with X11 programs that don't work the same way as the rest of the Mac programs. >> What pisses me off is the fact that Gnome and friends implement their >> own incompatible-with-others VFS's and automounters and stuff. > Then get them to agree on a common framework! They are trying hard to > get other parts of the GUI work well together, so this isn't far off > in wishfull thinking land. I honestly hope you're right. The more cooperation we have between GNOME and KDE and everyone else, the better. >> Surely supporting this in the kernel and extending the LSB to require >> this is the best step to take without infringing anyone's freedom as >> such. > Right. So then we have Gnome's way of doing things (Gnome isn't just > for Linux!), KDE's way, XFCE's way, ... (ditto). Plus the kernel > way. Flambee with a monthly thread on all reachable fora about "Why on > &%@# does the %&~#@ GUI not use the *#%&@ kernel's way?!". GNOME has HAL, their hardware abstraction layer, which uses (I assume) the kernel's facility to interact with the hardware. If someone ports GNOME to Windows, then HAL would use the Windows way to interact with the hardware. I can't see why it can't do something similar with the filesystem. Because who knows, maybe Gnumeric will need to access substream data under Windows to open an Excel file properly. Or Nautilus may have to read ... OS/2 extended attributes :-/ -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-29 20:57 ` Hubert Chan 0 siblings, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-29 20:57 UTC (permalink / raw) To: Horst von Brand Cc: Douglas McNaught, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Wed, 29 Jun 2005 13:19:12 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: > Markus Törnqvist <mjt@nysv.org> wrote: [...] >> Note that MacOS has the monopoly on what they ship, Linux has a >> motherload of file managers and window systems and all. > Yep. Part of what is nice about it, too ;-) I guess that's why Apple doesn't want to make its X11 support too good. Otherwise the Mac world would get flooded with X11 programs that don't work the same way as the rest of the Mac programs. >> What pisses me off is the fact that Gnome and friends implement their >> own incompatible-with-others VFS's and automounters and stuff. > Then get them to agree on a common framework! They are trying hard to > get other parts of the GUI work well together, so this isn't far off > in wishfull thinking land. I honestly hope you're right. The more cooperation we have between GNOME and KDE and everyone else, the better. >> Surely supporting this in the kernel and extending the LSB to require >> this is the best step to take without infringing anyone's freedom as >> such. > Right. So then we have Gnome's way of doing things (Gnome isn't just > for Linux!), KDE's way, XFCE's way, ... (ditto). Plus the kernel > way. Flambee with a monthly thread on all reachable fora about "Why on > &%@# does the %&~#@ GUI not use the *#%&@ kernel's way?!". GNOME has HAL, their hardware abstraction layer, which uses (I assume) the kernel's facility to interact with the hardware. If someone ports GNOME to Windows, then HAL would use the Windows way to interact with the hardware. I can't see why it can't do something similar with the filesystem. Because who knows, maybe Gnumeric will need to access substream data under Windows to open an Excel file properly. Or Nautilus may have to read ... OS/2 extended attributes :-/ -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 17:19 ` Horst von Brand @ 2005-06-30 9:59 ` mjt -1 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-30 9:59 UTC (permalink / raw) To: Horst von Brand Cc: Douglas McNaught, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 3324 bytes --] On Wed, Jun 29, 2005 at 01:19:12PM -0400, Horst von Brand wrote: >> What pisses me off is the fact that Gnome and friends implement >> their own incompatible-with-others VFS's and automounters and >> stuff. >Then get them to agree on a common framework! They are trying hard to get >other parts of the GUI work well together, so this isn't far off in >wishfull thinking land. Looking at the situ with devfs vs udev, which should have been clear ages ago, I don't think this will easily happen. What would a gnome developer say if I told him to bugger off, for the storm front of ivman is approaching? Having the mount system in userspace makes sense, it's implemented and done. Having metadata in some database chunk in userland makes less sense. For something that handles files and directories already, it must be easier to patch them to look at the assorted meta stuff inside the data object/file-as-dir. >> Surely supporting this in the kernel and extending the LSB >> to require this is the best step to take without infringing >> anyone's freedom as such. > >Right. So then we have Gnome's way of doing things (Gnome isn't just for >Linux!), KDE's way, XFCE's way, ... (ditto). Plus the kernel way. Flambee >with a monthly thread on all reachable fora about "Why on &%@# does the >%&~#@ GUI not use the *#%&@ kernel's way?!". Sure it's not "Dictum, factum" but having the kernel team bless an extended VFS which allows for metadata in any given FS is a significantly bigger thing than a pissing contest between Gnome, KDE and XFCE. Gnome not being only for Linux, well, make the official stance that the userspace VFS part is deprecated and to be used only on non-conforming systems. I haven't really used Gnome in long times, but if I have a picture and Nautilus shows me a thumbnail of it, doesn't the thumbnail live in a metadata VFS in userspace? That's a lot less usable than an in-kernel solution, where a serialized file can be sent to a conforming system or just the most important, or wanted, part of the file to a non-conforming one. That's the one thing all distros and managers and all have in common, the kernel. >This is /not/ the way of fxing that particular problem. Shoving random >stuff into the kernel /can't/ force its use. At least not until Linux is >the only Unixy system around, and that is still some way off. And when that >has happened, the kernel's way must be /clearly/ better for /all/ users, or >it won't matter. No use in making bigger deals of other Unixes on which Gnome and friends may run; my opinion and is that Linux should extend things further than the tradition so far has been. Maybe being better than the competitors may include breaking some rules, changing the playing field, making it more flexible. And the file system, and the VFS, is one of those things that are nice to have in the kernel. On a digressing side note, I found this classic: http://kernel.org/pub/linux/utils/kernel/hotplug/udev_vs_devfs "If the kernel tomorrow switches to randomly assign major and minor numbers to different devices, it would work just fine (this is exactly what I am proposing to do in 2.7...)" Apparently there isn't a 2.7 and no solid-enough testing ground for this either.. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-30 9:59 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-30 9:59 UTC (permalink / raw) To: Horst von Brand Cc: Douglas McNaught, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 3324 bytes --] On Wed, Jun 29, 2005 at 01:19:12PM -0400, Horst von Brand wrote: >> What pisses me off is the fact that Gnome and friends implement >> their own incompatible-with-others VFS's and automounters and >> stuff. >Then get them to agree on a common framework! They are trying hard to get >other parts of the GUI work well together, so this isn't far off in >wishfull thinking land. Looking at the situ with devfs vs udev, which should have been clear ages ago, I don't think this will easily happen. What would a gnome developer say if I told him to bugger off, for the storm front of ivman is approaching? Having the mount system in userspace makes sense, it's implemented and done. Having metadata in some database chunk in userland makes less sense. For something that handles files and directories already, it must be easier to patch them to look at the assorted meta stuff inside the data object/file-as-dir. >> Surely supporting this in the kernel and extending the LSB >> to require this is the best step to take without infringing >> anyone's freedom as such. > >Right. So then we have Gnome's way of doing things (Gnome isn't just for >Linux!), KDE's way, XFCE's way, ... (ditto). Plus the kernel way. Flambee >with a monthly thread on all reachable fora about "Why on &%@# does the >%&~#@ GUI not use the *#%&@ kernel's way?!". Sure it's not "Dictum, factum" but having the kernel team bless an extended VFS which allows for metadata in any given FS is a significantly bigger thing than a pissing contest between Gnome, KDE and XFCE. Gnome not being only for Linux, well, make the official stance that the userspace VFS part is deprecated and to be used only on non-conforming systems. I haven't really used Gnome in long times, but if I have a picture and Nautilus shows me a thumbnail of it, doesn't the thumbnail live in a metadata VFS in userspace? That's a lot less usable than an in-kernel solution, where a serialized file can be sent to a conforming system or just the most important, or wanted, part of the file to a non-conforming one. That's the one thing all distros and managers and all have in common, the kernel. >This is /not/ the way of fxing that particular problem. Shoving random >stuff into the kernel /can't/ force its use. At least not until Linux is >the only Unixy system around, and that is still some way off. And when that >has happened, the kernel's way must be /clearly/ better for /all/ users, or >it won't matter. No use in making bigger deals of other Unixes on which Gnome and friends may run; my opinion and is that Linux should extend things further than the tradition so far has been. Maybe being better than the competitors may include breaking some rules, changing the playing field, making it more flexible. And the file system, and the VFS, is one of those things that are nice to have in the kernel. On a digressing side note, I found this classic: http://kernel.org/pub/linux/utils/kernel/hotplug/udev_vs_devfs "If the kernel tomorrow switches to randomly assign major and minor numbers to different devices, it would work just fine (this is exactly what I am proposing to do in 2.7...)" Apparently there isn't a 2.7 and no solid-enough testing ground for this either.. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
[parent not found: <17091.60930.633968.822210@gargle.gargle.HOWL>]
* Re: reiser4 plugins [not found] ` <17091.60930.633968.822210@gargle.gargle.HOWL> @ 2005-06-30 14:21 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-30 14:21 UTC (permalink / raw) To: Nikita Danilov Cc: Douglas McNaught, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1165 bytes --] On Thu, Jun 30, 2005 at 05:05:06PM +0400, Nikita Danilov wrote: >You will have to force user level to use common framework >anyway. Naturally. >Otherwise one application will use > >~/pictures/sunset.jpg/...meta/mime-type > >and another one > >~/pictures/sunset.jpg/...meta/XML-schema-FOOBAR/version-0.99/attributes/common/MIME-type/value > >etc. Putting mechanism into kernel doesn't make it somehow magically >interoperable. It is _policy_ that does. And policy belongs to the >kernel not. Indeed this may not be something where Infinite Diversity in Infinite Combinations is a good thing. The best way to make sure everything works ok, is to draw out the specs for a policy and ensure it, but maybe it can't be enforced in the kernel. Unless the policy defined which directories (disregard the irony, please :) in the meta structure are available and off limits for mkdir et al. Sure this may be circumvented still by setting up an xml schema somewhere and so on and so forth. So my best guess probably is to have LSB extended with a damned good policy about How Things Work(tm) with the new meta system. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-30 14:21 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-30 14:21 UTC (permalink / raw) To: Nikita Danilov Cc: Douglas McNaught, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1165 bytes --] On Thu, Jun 30, 2005 at 05:05:06PM +0400, Nikita Danilov wrote: >You will have to force user level to use common framework >anyway. Naturally. >Otherwise one application will use > >~/pictures/sunset.jpg/...meta/mime-type > >and another one > >~/pictures/sunset.jpg/...meta/XML-schema-FOOBAR/version-0.99/attributes/common/MIME-type/value > >etc. Putting mechanism into kernel doesn't make it somehow magically >interoperable. It is _policy_ that does. And policy belongs to the >kernel not. Indeed this may not be something where Infinite Diversity in Infinite Combinations is a good thing. The best way to make sure everything works ok, is to draw out the specs for a policy and ensure it, but maybe it can't be enforced in the kernel. Unless the policy defined which directories (disregard the irony, please :) in the meta structure are available and off limits for mkdir et al. Sure this may be circumvented still by setting up an xml schema somewhere and so on and so forth. So my best guess probably is to have LSB extended with a damned good policy about How Things Work(tm) with the new meta system. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
[parent not found: <17092.3415.28856.827179@gargle.gargle.HOWL>]
* Re: reiser4 plugins [not found] ` <17092.3415.28856.827179@gargle.gargle.HOWL> @ 2005-06-30 15:37 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-30 15:37 UTC (permalink / raw) To: Nikita Danilov Cc: Douglas McNaught, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1220 bytes --] On Thu, Jun 30, 2005 at 07:18:47PM +0400, Nikita Danilov wrote: >Sorry, I don't see your point. Again: if you think that user level >developers are unlikely to agree to the common framework, what >difference it makes whether this framework is defined at the kernel or >library boundary? Applications would have to be changed to conform to >the common API either way. I see it as a heavier incentive to do it by a framework that's in the kernel. >If you can force application developers to conform to the LSB why you >cannot do the same with the library level interface? If I want to access metadata with bash, do I patch bash to support both Gnome's and KDE's solutions? Was there one of XFCE too? And FooBarXyzzyWM that'll want to do it's own VFS next year? I'd also guess that the upstream guys would much rather have patches for their progs that conform to the kernel than some obscure neighbor userspace system. Sure looks like having this in the kernel makes it easiest; there's just one common denominator to patch for. This doesn't even invalidate the userland VFSs of the other guys, they're still needed for systems whose kernels don't have a metadata facility. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-30 15:37 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-30 15:37 UTC (permalink / raw) To: Nikita Danilov Cc: Douglas McNaught, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1220 bytes --] On Thu, Jun 30, 2005 at 07:18:47PM +0400, Nikita Danilov wrote: >Sorry, I don't see your point. Again: if you think that user level >developers are unlikely to agree to the common framework, what >difference it makes whether this framework is defined at the kernel or >library boundary? Applications would have to be changed to conform to >the common API either way. I see it as a heavier incentive to do it by a framework that's in the kernel. >If you can force application developers to conform to the LSB why you >cannot do the same with the library level interface? If I want to access metadata with bash, do I patch bash to support both Gnome's and KDE's solutions? Was there one of XFCE too? And FooBarXyzzyWM that'll want to do it's own VFS next year? I'd also guess that the upstream guys would much rather have patches for their progs that conform to the kernel than some obscure neighbor userspace system. Sure looks like having this in the kernel makes it easiest; there's just one common denominator to patch for. This doesn't even invalidate the userland VFSs of the other guys, they're still needed for systems whose kernels don't have a metadata facility. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 15:37 ` mjt @ 2005-06-30 19:45 ` Diego Calleja -1 siblings, 0 replies; 620+ messages in thread From: Diego Calleja @ 2005-06-30 19:45 UTC (permalink / raw) To: Markus Törnqvist Cc: nikita, doug, hubert, mrmacman_g4, ninja, Valdis.Kletnieks, ltd, gmaxwell, reiser, jgarzik, hch, akpm, linux-kernel, reiserfs-list El Thu, 30 Jun 2005 18:37:38 +0300, mjt@nysv.org (Markus Törnqvist) escribió: > If I want to access metadata with bash, do I patch bash to support > both Gnome's and KDE's solutions? Was there one of XFCE too? > And FooBarXyzzyWM that'll want to do it's own VFS next year? The freedesktop people is already designing a common userspace VFS, take a look at http://freedesktop.org/Software/dvfs . At least it looks like there'll be a common standard, even if the standard is crappy. Apparently one of the main reasons for DVFS to exist is "POSIX is not enought for us". This approach is broken by design, having a second-class VFS separated from the main VFS (what libc functions see, etc) is wrong, IMO. The Mockup project (they try to bring a BeOS-like API to linux, I think) is something similar to DVFS but using FUSE. Take a look at http://wiki.mockup.org (currently down, google cache at http://66.249.93.104/search?q=cache:OnvclDCpwOwJ:wiki.mockup.org/UserSpaceVFS+mockup+wiki+VFS&hl=es&lr=&client=firefox&strip=1 ) ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-30 19:45 ` Diego Calleja 0 siblings, 0 replies; 620+ messages in thread From: Diego Calleja @ 2005-06-30 19:45 UTC (permalink / raw) To: Markus Törnqvist Cc: nikita, doug, hubert, mrmacman_g4, ninja, Valdis.Kletnieks, ltd, gmaxwell, reiser, jgarzik, hch, akpm, linux-kernel, reiserfs-list El Thu, 30 Jun 2005 18:37:38 +0300, mjt@nysv.org (Markus Törnqvist) escribió: > If I want to access metadata with bash, do I patch bash to support > both Gnome's and KDE's solutions? Was there one of XFCE too? > And FooBarXyzzyWM that'll want to do it's own VFS next year? The freedesktop people is already designing a common userspace VFS, take a look at http://freedesktop.org/Software/dvfs . At least it looks like there'll be a common standard, even if the standard is crappy. Apparently one of the main reasons for DVFS to exist is "POSIX is not enought for us". This approach is broken by design, having a second-class VFS separated from the main VFS (what libc functions see, etc) is wrong, IMO. The Mockup project (they try to bring a BeOS-like API to linux, I think) is something similar to DVFS but using FUSE. Take a look at http://wiki.mockup.org (currently down, google cache at http://66.249.93.104/search?q=cache:OnvclDCpwOwJ:wiki.mockup.org/UserSpaceVFS+mockup+wiki+VFS&hl=es&lr=&client=firefox&strip=1 ) ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 15:37 ` mjt @ 2005-06-30 19:52 ` Horst von Brand -1 siblings, 0 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-30 19:52 UTC (permalink / raw) To: Markus Törnqvist Cc: Nikita Danilov, Douglas McNaught, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain, Size: 2511 bytes --] Markus Törnqvist <mjt@nysv.org> wrote: > On Thu, Jun 30, 2005 at 07:18:47PM +0400, Nikita Danilov wrote: > >Sorry, I don't see your point. Again: if you think that user level > >developers are unlikely to agree to the common framework, what > >difference it makes whether this framework is defined at the kernel or > >library boundary? Applications would have to be changed to conform to > >the common API either way. > I see it as a heavier incentive to do it by a framework that's in > the kernel. API is API, if in-kernel, in-X-lib, or in-userland-VFS-lib is completely irrelevant. > >If you can force application developers to conform to the LSB why you > >cannot do the same with the library level interface? > If I want to access metadata with bash, do I patch bash to support > both Gnome's and KDE's solutions? Was there one of XFCE too? > And FooBarXyzzyWM that'll want to do it's own VFS next year? It's your only option... or get them together and define a common framework. But then again, what would I want to do with metadata in bash? If needed, it is probably much easier to write tools to extract whatever is needed, no reason to futz around with the shell. That simple idea was the single most important advance Unix introduced: The shell is a /simple/ program, it doesn't do word processing and coffee brewing for you. That is handled by other programs, one for each task. > I'd also guess that the upstream guys would much rather have > patches for their progs that conform to the kernel than some > obscure neighbor userspace system. Or just keep only their own obscure userspace system, no need to have to mess with our own format and a kernel system. > Sure looks like having this in the kernel makes it easiest; there's > just one common denominator to patch for. Again: API is API. If in kernel or in a standard library makes no difference. Libary is /much/ easier to develop, and hugely more flexible. It is for a reason that printf(3) and qsort(3) are /not/ in-kernel. > This doesn't even invalidate the userland VFSs of the other guys, > they're still needed for systems whose kernels don't have a > metadata facility. So the metadata facility in kernel won't be used, for portability's sake. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-30 19:52 ` Horst von Brand 0 siblings, 0 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-30 19:52 UTC (permalink / raw) To: Markus Törnqvist Cc: Nikita Danilov, Douglas McNaught, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Markus Törnqvist <mjt@nysv.org> wrote: > On Thu, Jun 30, 2005 at 07:18:47PM +0400, Nikita Danilov wrote: > >Sorry, I don't see your point. Again: if you think that user level > >developers are unlikely to agree to the common framework, what > >difference it makes whether this framework is defined at the kernel or > >library boundary? Applications would have to be changed to conform to > >the common API either way. > I see it as a heavier incentive to do it by a framework that's in > the kernel. API is API, if in-kernel, in-X-lib, or in-userland-VFS-lib is completely irrelevant. > >If you can force application developers to conform to the LSB why you > >cannot do the same with the library level interface? > If I want to access metadata with bash, do I patch bash to support > both Gnome's and KDE's solutions? Was there one of XFCE too? > And FooBarXyzzyWM that'll want to do it's own VFS next year? It's your only option... or get them together and define a common framework. But then again, what would I want to do with metadata in bash? If needed, it is probably much easier to write tools to extract whatever is needed, no reason to futz around with the shell. That simple idea was the single most important advance Unix introduced: The shell is a /simple/ program, it doesn't do word processing and coffee brewing for you. That is handled by other programs, one for each task. > I'd also guess that the upstream guys would much rather have > patches for their progs that conform to the kernel than some > obscure neighbor userspace system. Or just keep only their own obscure userspace system, no need to have to mess with our own format and a kernel system. > Sure looks like having this in the kernel makes it easiest; there's > just one common denominator to patch for. Again: API is API. If in kernel or in a standard library makes no difference. Libary is /much/ easier to develop, and hugely more flexible. It is for a reason that printf(3) and qsort(3) are /not/ in-kernel. > This doesn't even invalidate the userland VFSs of the other guys, > they're still needed for systems whose kernels don't have a > metadata facility. So the metadata facility in kernel won't be used, for portability's sake. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 19:52 ` Horst von Brand (?) @ 2005-07-05 20:46 ` Hubert Chan 2005-07-10 0:00 ` Stefan Smietanowski -1 siblings, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-07-05 20:46 UTC (permalink / raw) To: Horst von Brand Cc: Nikita Danilov, Douglas McNaught, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Thu, 30 Jun 2005 15:52:25 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: >> This doesn't even invalidate the userland VFSs of the other guys, >> they're still needed for systems whose kernels don't have a metadata >> facility. > So the metadata facility in kernel won't be used, for portability's > sake. Oh gee. Every operating system does threads differently. Mozilla has an abstraction layer called nspr that allows them to handle threads portably. glib/gtk has their own threads abstraction. On Windows, nspr will use the Windows method for handling threads. On Linux, it will use the Linux way. On systems that don't support threads, it can usually emulate it using timers. It's the exact same thing with the userspace VFS. If GNOME needs to handle extended attributes, it can use one mechanism under one operating system, or emulate it using some ugly hack on operating systems that don't support extended attributes. Isn't that the whole point of having a VFS? -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 20:46 ` Hubert Chan @ 2005-07-10 0:00 ` Stefan Smietanowski 2005-07-11 12:28 ` Jaroslav Soltys 2005-07-11 23:17 ` Hubert Chan 0 siblings, 2 replies; 620+ messages in thread From: Stefan Smietanowski @ 2005-07-10 0:00 UTC (permalink / raw) To: Hubert Chan Cc: Horst von Brand, Nikita Danilov, Douglas McNaught, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hubert Chan wrote: > On Thu, 30 Jun 2005 15:52:25 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: > > >>>This doesn't even invalidate the userland VFSs of the other guys, >>>they're still needed for systems whose kernels don't have a metadata >>>facility. > > >>So the metadata facility in kernel won't be used, for portability's >>sake. > > > Oh gee. Every operating system does threads differently. Mozilla has > an abstraction layer called nspr that allows them to handle threads > portably. glib/gtk has their own threads abstraction. On Windows, nspr > will use the Windows method for handling threads. On Linux, it will use > the Linux way. On systems that don't support threads, it can usually > emulate it using timers. > > It's the exact same thing with the userspace VFS. If GNOME needs to > handle extended attributes, it can use one mechanism under one operating > system, or emulate it using some ugly hack on operating systems that > don't support extended attributes. > > Isn't that the whole point of having a VFS? > So basically if I write a program that works in both Gnome and KDE I should (according to your description) implement my own VFS that will use the Gnome or KDE VFS that will then use the OS VFS. Is it only me finding that a little silly? I mean, if I am to have the same functionality under neither Gnome nor VFS and they don't support something I need I _NEED_ a vfs so that my program is so totally independent on anything at all. My program calling My VFS which calls KDE/Gnome's VFS which calls the OS VFS will be slowe than just calling the VFS immidiately - I do hope you can see that. // Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (MingW32) iD8DBQFC0GUxBrn2kJu9P78RApViAJ4q6BrVh0H19S4pN+Zc0bqh7zk2sgCeLRVK 8b+qlg2BHjwxg8HnVANQ5XA= =2uNQ -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-10 0:00 ` Stefan Smietanowski @ 2005-07-11 12:28 ` Jaroslav Soltys 2005-07-11 23:17 ` Hubert Chan 1 sibling, 0 replies; 620+ messages in thread From: Jaroslav Soltys @ 2005-07-11 12:28 UTC (permalink / raw) To: linux-kernel > So basically if I write a program that works in both Gnome and KDE > I should (according to your description) implement my own VFS that > will use the Gnome or KDE VFS that will then use the OS VFS. > > Is it only me finding that a little silly? Maybe. Advantages of kde/gnome/other userland vfs ? Authentication when using SMB shares, transparent access to fish/webdav/ftp/... but only for applications that use these libraries. Maybe patched automount together with lufs plus some GUI for acquiring user credentials from kde/gnome user could do the same, but kde/gnome vfs is portable to other unices. Oh, by the way... mount something with smbmount and turn of that computer you mounted the share from. You must switch to root to be able to do umount -lf. But yes, I agree with you, it is silly and I would also like to have one good solution than two average ones. > I mean, if I am to have the same functionality under neither Gnome > nor VFS and they don't support something I need I _NEED_ a vfs so > that my program is so totally independent on anything at all. right, wouldn't it be nice to 'cd /mnt/webdav/' or '/mnt/ssh/infernal.machine/' in bash ? :) > My program calling My VFS which calls KDE/Gnome's VFS which calls the OS > VFS will be slowe than just calling the VFS immidiately - I do hope you > can see that. Some of kde/gnome vfs are not even filesystems at all, this is the main advantage of userland vfs. Imagine that it is possible for user to add new FS to kde/gnome, is it possible for him to add it to kernel without root access ? compile new module and load it ? (Un)fortunately not. jaro ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-10 0:00 ` Stefan Smietanowski 2005-07-11 12:28 ` Jaroslav Soltys @ 2005-07-11 23:17 ` Hubert Chan 1 sibling, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-07-11 23:17 UTC (permalink / raw) To: Stefan Smietanowski Cc: Horst von Brand, Nikita Danilov, Douglas McNaught, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Sun, 10 Jul 2005 02:00:49 +0200, Stefan Smietanowski <stesmi@stesmi.com> said: > So basically if I write a program that works in both Gnome and KDE I > should (according to your description) implement my own VFS that will > use the Gnome or KDE VFS that will then use the OS VFS. Either that, or use a whole lot of #ifdefs, or hope that GNOME and KDE agree on a common VFS, or don't use their VFS and just use the basic OS calls and lose the functionality (and portability) of the VFSes, or pick one VFS and hope that the other users can adapt. GNOME and KDE already have their own VFS. I am not asking for anything new. If you don't like the idea of a VFS at that level, take it up with the GNOME and KDE people. > Is it only me finding that a little silly? > I mean, if I am to have the same functionality under neither Gnome nor > VFS and they don't support something I need I _NEED_ a vfs so that my > program is so totally independent on anything at all. > My program calling My VFS which calls KDE/Gnome's VFS which calls the > OS VFS will be slowe than just calling the VFS immidiately - I do hope > you can see that. Of course. It's probably slower than arranging the bits on the hard drive directly, and hand-coding everything in assembly. But there's always a performance price to pay for maintaining the programmer's sanity. There's always a price to pay when writing cross-platform stuff. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 17:19 ` Horst von Brand ` (2 preceding siblings ...) (?) @ 2005-07-01 8:16 ` David Masover -1 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-07-01 8:16 UTC (permalink / raw) To: Horst von Brand Cc: Markus T rnqvist, Douglas McNaught, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Horst von Brand wrote: > Markus Törnqvist <mjt@nysv.org> wrote: > > [...] > > >>Note that MacOS has the monopoly on what they ship, Linux has a >>motherload of file managers and window systems and all. > > > Yep. Part of what is nice about it, too ;-) > > >>What pisses me off is the fact that Gnome and friends implement >>their own incompatible-with-others VFS's and automounters and >>stuff. > > > Then get them to agree on a common framework! They are trying hard to get > other parts of the GUI work well together, so this isn't far off in > wishfull thinking land. Unfortunately, this leaves bash out in the cold, as usual. Kernel-based automounter works with Bash. Why can't GNOME/KDE use the kernel's one as at least one backend, even if they support others? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 13:58 ` mjt @ 2005-06-29 19:05 ` Valdis.Kletnieks -1 siblings, 0 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-29 19:05 UTC (permalink / raw) To: Markus =?UNKNOWN?Q?T=F6rnqvist?= Cc: Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, David Masover, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 622 bytes --] On Wed, 29 Jun 2005 16:58:20 +0300, Markus =?UNKNOWN?Q?T=F6rnqvist?= said: > What pisses me off is the fact that Gnome and friends implement > their own incompatible-with-others VFS's and automounters and > stuff. The fact that things like Gnome, which are basically consumers of their own dogfood, have incompatible versions says very loudly that there's no consensus on the semantics.... > Surely supporting this in the kernel and extending the LSB > to require this is the best step to take without infringing > anyone's freedom as such. First we need to decide *if* it's to be supported, then *what* to support.... [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-29 19:05 ` Valdis.Kletnieks 0 siblings, 0 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-29 19:05 UTC (permalink / raw) To: Markus =?UNKNOWN?Q?T=F6rnqvist?= Cc: Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, David Masover, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 622 bytes --] On Wed, 29 Jun 2005 16:58:20 +0300, Markus =?UNKNOWN?Q?T=F6rnqvist?= said: > What pisses me off is the fact that Gnome and friends implement > their own incompatible-with-others VFS's and automounters and > stuff. The fact that things like Gnome, which are basically consumers of their own dogfood, have incompatible versions says very loudly that there's no consensus on the semantics.... > Surely supporting this in the kernel and extending the LSB > to require this is the best step to take without infringing > anyone's freedom as such. First we need to decide *if* it's to be supported, then *what* to support.... [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 13:58 ` mjt ` (2 preceding siblings ...) (?) @ 2005-06-29 20:56 ` David Weinehall 2005-06-29 23:05 ` Chet Hosey ` (2 more replies) -1 siblings, 3 replies; 620+ messages in thread From: David Weinehall @ 2005-06-29 20:56 UTC (permalink / raw) To: Markus Törnqvist Cc: Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Wed, Jun 29, 2005 at 04:58:20PM +0300, Markus Törnqvist wrote: > On Wed, Jun 29, 2005 at 09:50:27AM -0400, Douglas McNaught wrote: > > > >I'll just note that the "applications bundled as directories" stuff on > >MacOS/NextStep is done completely in userspace--as far as the kernel > >is concerned, "Mail.app" is a regular directory. The file manager > >handles recognition and invocation of application bundles (and there > >is an 'open' shell command that does the same thing). > > Note that MacOS has the monopoly on what they ship, Linux has a > motherload of file managers and window systems and all. > > What pisses me off is the fact that Gnome and friends implement > their own incompatible-with-others VFS's and automounters and > stuff. > > Surely supporting this in the kernel and extending the LSB > to require this is the best step to take without infringing > anyone's freedom as such. > > *still pissed off about having to hassle an automatic mount* GNOME and KDE run on operating systems that run other kernels than Linux, hence they have to implement their own userland VFS anyway. Adding this to the Linux kernel won't help them one bit, unless we can magically convince Sun to add it to Solaris, all different BSD:s to add it to their kernels, etc. Not going to happen. An effort to get GNOME and KDE to unify their VFS:s would be far more benificial, and would if successful probably lead to other desktop environments (if there are any other DE:s with their own VFS:s, dunno about that) following their lead. FreeDesktop is doing a lot of work to make GNOME, KDE, and other DE:s interoperate as much as possible. Support their initiative instead of trying to get a monstrosity (albeit a very cool one, conceptually) into the kernel. Sure, it could be made to work, but not without dropping our Unixness. And if we do, we should start by looking at Plan 9 =) Regards: David -- /) David Weinehall <tao@acc.umu.se> /) Northern lights wander (\ // Maintainer of the v2.0 kernel // Dance across the winter sky // \) http://www.acc.umu.se/~tao/ (/ Full colour fire (/ ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 20:56 ` David Weinehall @ 2005-06-29 23:05 ` Chet Hosey 2005-06-30 10:01 ` mjt 2005-07-01 8:08 ` David Masover 2 siblings, 0 replies; 620+ messages in thread From: Chet Hosey @ 2005-06-29 23:05 UTC (permalink / raw) To: David Weinehall Cc: Markus Törnqvist, Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Weinehall wrote: >GNOME and KDE run on operating systems that run other kernels than >Linux, hence they have to implement their own userland VFS anyway. >Adding this to the Linux kernel won't help them one bit, unless >we can magically convince Sun to add it to Solaris, all different >BSD:s to add it to their kernels, etc. Not going to happen. >An effort to get GNOME and KDE to unify their VFS:s would be >far more benificial, and would if successful probably lead to >other desktop environments (if there are any other DE:s with their >own VFS:s, dunno about that) following their lead. > >FreeDesktop is doing a lot of work to make GNOME, KDE, and other >DE:s interoperate as much as possible. Support their initiative >instead of trying to get a monstrosity (albeit a very cool one, >conceptually) into the kernel. Sure, it could be made to work, >but not without dropping our Unixness. And if we do, we should >start by looking at Plan 9 =) > > >Regards: David > > The point of such ventures is that by placing features at a lower level you get to keep the advantages of UNIX in the first place -- namely, that many small tools can do neat things with most objects. By placing everything in a largish userspace library instead of at a system level (kernel, libc, etc.) you're essentially saying that, for instance, vi would have to be rewritten in order to interact with objects presented by the VFS. So would bash, fmt, sort, less, aspell, or anything else that can open a file. You'd end up with a situation in which you see objects via the VFS browser (file manager) that no longer exist when you want to drop to a shell to use common UNIX utilities and find that the object doesn't actually exist to the OS itself. This sounds like Joel Spolsky's law of leaky abstractions, and the fact that most operating systems lack a useful facility (which is why GNOME and KDE roll their own VFS) sounds like a poor excuse for keeping useful features out of the kernel. I'm *not* arguing for putting anything specific into the kernel. I *am* arguing that an inconsistent presentation in which some apps see VFS objects and others don't makes for a less-than-ideal UI. -- chet ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 20:56 ` David Weinehall @ 2005-06-30 10:01 ` mjt 2005-06-30 10:01 ` mjt 2005-07-01 8:08 ` David Masover 2 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-30 10:01 UTC (permalink / raw) To: Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 607 bytes --] On Wed, Jun 29, 2005 at 10:56:36PM +0200, David Weinehall wrote: > >FreeDesktop is doing a lot of work to make GNOME, KDE, and other >DE:s interoperate as much as possible. Support their initiative What about WindowMaker with bash? >instead of trying to get a monstrosity (albeit a very cool one, >conceptually) into the kernel. Sure, it could be made to work, >but not without dropping our Unixness. And if we do, we should >start by looking at Plan 9 =) What's wrong with "dropping our Unixness" if it means taking an extra step toward Plan 9? Why is this a bad idea? -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-30 10:01 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-30 10:01 UTC (permalink / raw) To: Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 607 bytes --] On Wed, Jun 29, 2005 at 10:56:36PM +0200, David Weinehall wrote: > >FreeDesktop is doing a lot of work to make GNOME, KDE, and other >DE:s interoperate as much as possible. Support their initiative What about WindowMaker with bash? >instead of trying to get a monstrosity (albeit a very cool one, >conceptually) into the kernel. Sure, it could be made to work, >but not without dropping our Unixness. And if we do, we should >start by looking at Plan 9 =) What's wrong with "dropping our Unixness" if it means taking an extra step toward Plan 9? Why is this a bad idea? -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Linux and Plan-9ness 2005-06-30 10:01 ` mjt @ 2005-06-30 12:45 ` Al Boldi -1 siblings, 0 replies; 620+ messages in thread From: Al Boldi @ 2005-06-30 12:45 UTC (permalink / raw) To: 'Markus Törnqvist' Cc: 'Douglas McNaught', 'Horst von Brand', 'Hubert Chan', 'Kyle Moffett', 'David Masover', Valdis.Kletnieks, 'Lincoln Dale', 'Gregory Maxwell', 'Hans Reiser', 'Jeff Garzik', 'Christoph Hellwig', 'Andrew Morton', linux-kernel, 'ReiserFS List' Markus Törnqvist wrote: { On Wed, Jun 29, 2005 at 10:56:36PM +0200, David Weinehall wrote: >instead of trying to get a monstrosity (albeit a very cool one, >conceptually) into the kernel. Sure, it could be made to work, but not >without dropping our Unixness. And if we do, we should start by >looking at Plan 9 =) What's wrong with "dropping our Unixness" if it means taking an extra step toward Plan 9? Why is this a bad idea? } Please explain! ^ permalink raw reply [flat|nested] 620+ messages in thread
* Linux and Plan-9ness @ 2005-06-30 12:45 ` Al Boldi 0 siblings, 0 replies; 620+ messages in thread From: Al Boldi @ 2005-06-30 12:45 UTC (permalink / raw) To: 'Markus Törnqvist' Cc: 'Douglas McNaught', 'Horst von Brand', 'Hubert Chan', 'Kyle Moffett', 'David Masover', Valdis.Kletnieks, 'Lincoln Dale', 'Gregory Maxwell', 'Hans Reiser', 'Jeff Garzik', 'Christoph Hellwig', 'Andrew Morton', linux-kernel, 'ReiserFS List' Markus Törnqvist wrote: { On Wed, Jun 29, 2005 at 10:56:36PM +0200, David Weinehall wrote: >instead of trying to get a monstrosity (albeit a very cool one, >conceptually) into the kernel. Sure, it could be made to work, but not >without dropping our Unixness. And if we do, we should start by >looking at Plan 9 =) What's wrong with "dropping our Unixness" if it means taking an extra step toward Plan 9? Why is this a bad idea? } Please explain! ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: Linux and Plan-9ness 2005-06-30 12:45 ` Al Boldi @ 2005-06-30 14:08 ` mjt -1 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-30 14:08 UTC (permalink / raw) To: Al Boldi Cc: 'Douglas McNaught', 'Horst von Brand', 'Hubert Chan', 'Kyle Moffett', 'David Masover', Valdis.Kletnieks, 'Lincoln Dale', 'Gregory Maxwell', 'Hans Reiser', 'Jeff Garzik', 'Christoph Hellwig', 'Andrew Morton', linux-kernel, 'ReiserFS List' [-- Attachment #1: Type: text/plain, Size: 605 bytes --] On Thu, Jun 30, 2005 at 03:45:48PM +0300, Al Boldi wrote: >Markus Törnqvist wrote: { > >What's wrong with "dropping our Unixness" if it means taking an extra step >toward Plan 9? > >Why is this a bad idea? >} > >Please explain! You mean you want me to explain this or someone else to explain why this is a bad idea?-) My take on this is that Linux should be allowed to move toward the directions outlined here on the list a million times, and "dropping our Unixness" may be a bit harsh, it should probably be more along the lines of "not being so uptightly unixy" :) -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: Linux and Plan-9ness @ 2005-06-30 14:08 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-30 14:08 UTC (permalink / raw) To: Al Boldi Cc: 'Douglas McNaught', 'Horst von Brand', 'Hubert Chan', 'Kyle Moffett', 'David Masover', Valdis.Kletnieks, 'Lincoln Dale', 'Gregory Maxwell', 'Hans Reiser', 'Jeff Garzik', 'Christoph Hellwig', 'Andrew Morton', linux-kernel, 'ReiserFS List' [-- Attachment #1: Type: text/plain, Size: 605 bytes --] On Thu, Jun 30, 2005 at 03:45:48PM +0300, Al Boldi wrote: >Markus Törnqvist wrote: { > >What's wrong with "dropping our Unixness" if it means taking an extra step >toward Plan 9? > >Why is this a bad idea? >} > >Please explain! You mean you want me to explain this or someone else to explain why this is a bad idea?-) My take on this is that Linux should be allowed to move toward the directions outlined here on the list a million times, and "dropping our Unixness" may be a bit harsh, it should probably be more along the lines of "not being so uptightly unixy" :) -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: Linux and Plan-9ness 2005-06-30 12:45 ` Al Boldi (?) (?) @ 2005-07-04 17:18 ` studdugie -1 siblings, 0 replies; 620+ messages in thread From: studdugie @ 2005-07-04 17:18 UTC (permalink / raw) To: linux-kernel, ReiserFS List If you dropped the "Unixness" you would have to rename. Maybe Linine, or Linsoft, or .... Think of all the chaos that would cause. You don't want chaos do you? I mean really, do u want to force all the \w*(?ilinux)\w*(\.(com|org|net))? sites and publications to rename too? Unconscionable! On 6/30/05, Al Boldi <a1426z@gawab.com> wrote: > Markus Törnqvist wrote: { > On Wed, Jun 29, 2005 at 10:56:36PM +0200, David Weinehall wrote: > >instead of trying to get a monstrosity (albeit a very cool one, > >conceptually) into the kernel. Sure, it could be made to work, but not > >without dropping our Unixness. And if we do, we should start by > >looking at Plan 9 =) > > What's wrong with "dropping our Unixness" if it means taking an extra step > toward Plan 9? > > Why is this a bad idea? > } > > Please explain! > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 10:01 ` mjt (?) (?) @ 2005-06-30 19:57 ` Eric Van Hensbergen -1 siblings, 0 replies; 620+ messages in thread From: Eric Van Hensbergen @ 2005-06-30 19:57 UTC (permalink / raw) To: Markus Törnqvist Cc: Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On 6/30/05, Markus Törnqvist <mjt@nysv.org> wrote: > > >instead of trying to get a monstrosity (albeit a very cool one, > >conceptually) into the kernel. Sure, it could be made to work, > >but not without dropping our Unixness. And if we do, we should > >start by looking at Plan 9 =) > > What's wrong with "dropping our Unixness" if it means taking > an extra step toward Plan 9? > > Why is this a bad idea? > It's not. For those who don't already know about it: check out the v9fs project (http://v9fs.sf.net) - we're taking steps of moving the Linux kernel towards Plan 9 while trying to preserve Unix semantics where they make sense. -eric ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 20:56 ` David Weinehall 2005-06-29 23:05 ` Chet Hosey 2005-06-30 10:01 ` mjt @ 2005-07-01 8:08 ` David Masover 2005-07-01 15:54 ` David Weinehall 2 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-01 8:08 UTC (permalink / raw) To: David Weinehall Cc: Markus Törnqvist, Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Weinehall wrote: > On Wed, Jun 29, 2005 at 04:58:20PM +0300, Markus Törnqvist wrote: > >>On Wed, Jun 29, 2005 at 09:50:27AM -0400, Douglas McNaught wrote: >> >>>I'll just note that the "applications bundled as directories" stuff on >>>MacOS/NextStep is done completely in userspace--as far as the kernel >>>is concerned, "Mail.app" is a regular directory. The file manager >>>handles recognition and invocation of application bundles (and there >>>is an 'open' shell command that does the same thing). >> >>Note that MacOS has the monopoly on what they ship, Linux has a >>motherload of file managers and window systems and all. >> >>What pisses me off is the fact that Gnome and friends implement >>their own incompatible-with-others VFS's and automounters and >>stuff. >> >>Surely supporting this in the kernel and extending the LSB >>to require this is the best step to take without infringing >>anyone's freedom as such. >> >>*still pissed off about having to hassle an automatic mount* > > > GNOME and KDE run on operating systems that run other kernels than > Linux, hence they have to implement their own userland VFS anyway. > Adding this to the Linux kernel won't help them one bit, unless > we can magically convince Sun to add it to Solaris, all different > BSD:s to add it to their kernels, etc. Not going to happen. > An effort to get GNOME and KDE to unify their VFS:s would be > far more benificial, Than what? Creating a unified VFS which I can access from Bash, and which obsoletes both GNOME and KDE's VFSes except in their presentation? > FreeDesktop is doing a lot of work to make GNOME, KDE, and other > DE:s interoperate as much as possible. Support their initiative > instead of trying to get a monstrosity (albeit a very cool one, > conceptually) into the kernel. Sure, it could be made to work, > but not without dropping our Unixness. (I'm talking about the metafs (/meta) idea, which isn't nearly as much a monstrocity, and doesn't kill our unixness, it enhances it.) ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 8:08 ` David Masover @ 2005-07-01 15:54 ` David Weinehall 2005-07-01 19:55 ` David Masover 2005-07-07 8:27 ` mjt 0 siblings, 2 replies; 620+ messages in thread From: David Weinehall @ 2005-07-01 15:54 UTC (permalink / raw) To: David Masover Cc: Markus Törnqvist, Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Fri, Jul 01, 2005 at 03:08:58AM -0500, David Masover wrote: > David Weinehall wrote: > > >GNOME and KDE run on operating systems that run other kernels than > >Linux, hence they have to implement their own userland VFS anyway. > >Adding this to the Linux kernel won't help them one bit, unless > >we can magically convince Sun to add it to Solaris, all different > >BSD:s to add it to their kernels, etc. Not going to happen. > >An effort to get GNOME and KDE to unify their VFS:s would be > >far more benificial, > > Than what? Creating a unified VFS which I can access from Bash, and > which obsoletes both GNOME and KDE's VFSes except in their presentation? On one of the platforms that they support, yes. But only for kernels newer than 2.6.yy... So they'd still have to have their own VFS for 2.4.xx, 2.6.xx (xx < yy), FreeBSD, OpenBSD, Solaris, etc... > >FreeDesktop is doing a lot of work to make GNOME, KDE, and other > >DE:s interoperate as much as possible. Support their initiative > >instead of trying to get a monstrosity (albeit a very cool one, > >conceptually) into the kernel. Sure, it could be made to work, > >but not without dropping our Unixness. > > (I'm talking about the metafs (/meta) idea, which isn't nearly as much a > monstrocity, and doesn't kill our unixness, it enhances it.) Which would neither need VFS changes nor be dependent on Reiser4 in any way, so I don't see why this thread lives on. Just get down to business and implement this metafs =) Regards: David Weinehall -- /) David Weinehall <tao@acc.umu.se> /) Northern lights wander (\ // Maintainer of the v2.0 kernel // Dance across the winter sky // \) http://www.acc.umu.se/~tao/ (/ Full colour fire (/ ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 15:54 ` David Weinehall @ 2005-07-01 19:55 ` David Masover 2005-07-02 1:48 ` Horst von Brand 2005-07-07 8:27 ` mjt 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-01 19:55 UTC (permalink / raw) To: David Weinehall Cc: Markus Törnqvist, Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Weinehall wrote: > On Fri, Jul 01, 2005 at 03:08:58AM -0500, David Masover wrote: > >>David Weinehall wrote: >> >> >>>GNOME and KDE run on operating systems that run other kernels than >>>Linux, hence they have to implement their own userland VFS anyway. >>>Adding this to the Linux kernel won't help them one bit, unless >>>we can magically convince Sun to add it to Solaris, all different >>>BSD:s to add it to their kernels, etc. Not going to happen. >>>An effort to get GNOME and KDE to unify their VFS:s would be >>>far more benificial, >> >>Than what? Creating a unified VFS which I can access from Bash, and >>which obsoletes both GNOME and KDE's VFSes except in their presentation? > > > On one of the platforms that they support, yes. But only for kernels > newer than 2.6.yy... So they'd still have to have their own VFS for > 2.4.xx, 2.6.xx (xx < yy), FreeBSD, OpenBSD, Solaris, etc... Right. But, /proc started somewhere, didn't it? I have the feeling that other systems will duplicate it if it's good. Even if they don't, it would be more beneficial to me and probably most Linux users to have metafs supported in both GNOME and KDE, even if they still need an emulation layer to support other systems. I agree that the emulation layer should be common. >>>FreeDesktop is doing a lot of work to make GNOME, KDE, and other >>>DE:s interoperate as much as possible. Support their initiative >>>instead of trying to get a monstrosity (albeit a very cool one, >>>conceptually) into the kernel. Sure, it could be made to work, >>>but not without dropping our Unixness. >> >>(I'm talking about the metafs (/meta) idea, which isn't nearly as much a >>monstrocity, and doesn't kill our unixness, it enhances it.) > > > Which would neither need VFS changes nor be dependent on Reiser4 in > any way, so I don't see why this thread lives on. Just get down to > business and implement this metafs =) I am trying, at least getting the conceptual glitches ironed out. But, it does need GNOME/KDE VFS changes in order to be sane -- otherwise, GNOME/KDE will duplicate it, creating even more of a mess than we already have. I'm not sure why the thread still lives, but now that I've typed this, I may as well send it... ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 19:55 ` David Masover @ 2005-07-02 1:48 ` Horst von Brand 2005-07-04 3:42 ` Hans Reiser 2005-07-04 18:47 ` David Masover 0 siblings, 2 replies; 620+ messages in thread From: Horst von Brand @ 2005-07-02 1:48 UTC (permalink / raw) To: David Masover Cc: David Weinehall, Markus Törnqvist, Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover <ninja@slaphack.com> wrote: > David Weinehall wrote: > > On Fri, Jul 01, 2005 at 03:08:58AM -0500, David Masover wrote: > >>David Weinehall wrote: > >>>GNOME and KDE run on operating systems that run other kernels than > >>>Linux, hence they have to implement their own userland VFS anyway. > >>>Adding this to the Linux kernel won't help them one bit, unless > >>>we can magically convince Sun to add it to Solaris, all different > >>>BSD:s to add it to their kernels, etc. Not going to happen. > >>>An effort to get GNOME and KDE to unify their VFS:s would be > >>>far more benificial, > >> Than what? Creating a unified VFS which I can access from Bash, > >> and which obsoletes both GNOME and KDE's VFSes except in their > >> presentation? > > On one of the platforms that they support, yes. But only for kernels > > newer than 2.6.yy... So they'd still have to have their own VFS for > > 2.4.xx, 2.6.xx (xx < yy), FreeBSD, OpenBSD, Solaris, etc... > Right. But, /proc started somewhere, didn't it? Sun. > I have the feeling that other systems will duplicate it if it's good. Linux copied here. > Even if they don't, it would be more beneficial to me How, exactly? Besides, /your/ convenience isn't the only thing that matters... > and probably > most Linux users "Most Linux users" don't use experimental filesystems at all... > to have metafs supported in both GNOME and KDE, even > if they still need an emulation layer to support other systems. So Gnome and KDE get larger (and thus slower) for everybody. Besides, Gnome and KDE will have to agree on the formats involved first, which is /exactly/ what is supposed to be impossible unless this stuff is implemented in the kernel... -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-02 1:48 ` Horst von Brand @ 2005-07-04 3:42 ` Hans Reiser 2005-07-04 7:17 ` malcolm 2005-07-04 9:16 ` Christoph Hellwig 2005-07-04 18:47 ` David Masover 1 sibling, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-07-04 3:42 UTC (permalink / raw) To: Horst von Brand Cc: David Masover, David Weinehall, Markus Törnqvist, Douglas McNaught, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Horst von Brand wrote: > >>Right. But, /proc started somewhere, didn't it? >> >> > >Sun. > > No, plan 9. It seems to me that what you are all arguing about is whether it is better to be at the front of the herd, or in the middle, or in the back. I don't think this one can be resolved, except that I would suggest that Linux, by virtue of the kernel configuration functionality, has space enough for persons of all three inclinations, if we can just leave each other be a bit. Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-04 3:42 ` Hans Reiser @ 2005-07-04 7:17 ` malcolm 2005-07-04 7:22 ` Hans Reiser 2005-07-04 9:16 ` Christoph Hellwig 1 sibling, 1 reply; 620+ messages in thread From: malcolm @ 2005-07-04 7:17 UTC (permalink / raw) To: reiserfs-list On Monday 04 July 2005 05:42, Hans Reiser wrote: > Horst von Brand wrote: > >>Right. But, /proc started somewhere, didn't it? > > > >Sun. > > No, plan 9. Actually, I heard a lecture on implementing /proc in BSD at least 5 years earlier than plan 9. > > It seems to me that what you are all arguing about is whether it is > better to be at the front of the herd, or in the middle, or in the > back. I don't think this one can be resolved, except that I would > suggest that Linux, by virtue of the kernel configuration functionality, > has space enough for persons of all three inclinations, if we can just > leave each other be a bit. > > Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-04 7:17 ` malcolm @ 2005-07-04 7:22 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-07-04 7:22 UTC (permalink / raw) To: malcolm; +Cc: reiserfs-list malcolm wrote: >On Monday 04 July 2005 05:42, Hans Reiser wrote: > > >>Horst von Brand wrote: >> >> >>>>Right. But, /proc started somewhere, didn't it? >>>> >>>> >>>Sun. >>> >>> >>No, plan 9. >> >> > >Actually, I heard a lecture on implementing /proc in BSD at least 5 years >earlier than plan 9. > > http://library.n0i.net/linux-unix/art-unix-programming/plan9.html says "Some Plan 9 ideas have been absorbed into modern Unixes, particularly the more innovative open-source versions. FreeBSD has a /proc file system modeled exactly on that of Plan 9 that can be used to query or control running processes." Can you cite the other viewpoint? >>It seems to me that what you are all arguing about is whether it is >>better to be at the front of the herd, or in the middle, or in the >>back. I don't think this one can be resolved, except that I would >>suggest that Linux, by virtue of the kernel configuration functionality, >>has space enough for persons of all three inclinations, if we can just >>leave each other be a bit. >> >>Hans >> >> > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-04 3:42 ` Hans Reiser 2005-07-04 7:17 ` malcolm @ 2005-07-04 9:16 ` Christoph Hellwig 1 sibling, 0 replies; 620+ messages in thread From: Christoph Hellwig @ 2005-07-04 9:16 UTC (permalink / raw) To: Hans Reiser Cc: Horst von Brand, David Masover, David Weinehall, Markus T?rnqvist, Douglas McNaught, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Sun, Jul 03, 2005 at 08:42:24PM -0700, Hans Reiser wrote: > >>Right. But, /proc started somewhere, didn't it? > >> > >> > > > >Sun. > > > > > No, plan 9. Almost on the right track, it was v8, two steps before plan9. But that's just the process-part of procfs, not the big mess we have now - that part is pretty much linux-only. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-02 1:48 ` Horst von Brand 2005-07-04 3:42 ` Hans Reiser @ 2005-07-04 18:47 ` David Masover 2005-07-04 20:32 ` Horst von Brand 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-04 18:47 UTC (permalink / raw) To: Horst von Brand Cc: David Weinehall, Markus Törnqvist, Douglas McNaught, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Horst von Brand wrote: > David Masover <ninja@slaphack.com> wrote: > >>David Weinehall wrote: >> >>>On Fri, Jul 01, 2005 at 03:08:58AM -0500, David Masover wrote: >>> >>>>David Weinehall wrote: > > >>>>>GNOME and KDE run on operating systems that run other kernels than >>>>>Linux, hence they have to implement their own userland VFS anyway. >>>>>Adding this to the Linux kernel won't help them one bit, unless >>>>>we can magically convince Sun to add it to Solaris, all different >>>>>BSD:s to add it to their kernels, etc. Not going to happen. >>>>>An effort to get GNOME and KDE to unify their VFS:s would be >>>>>far more benificial, > > >>>>Than what? Creating a unified VFS which I can access from Bash, >>>>and which obsoletes both GNOME and KDE's VFSes except in their >>>>presentation? > > >>>On one of the platforms that they support, yes. But only for kernels >>>newer than 2.6.yy... So they'd still have to have their own VFS for >>>2.4.xx, 2.6.xx (xx < yy), FreeBSD, OpenBSD, Solaris, etc... > > >>Right. But, /proc started somewhere, didn't it? > > > Sun. > > >>I have the feeling that other systems will duplicate it if it's good. > > > Linux copied here. So what? >>Even if they don't, it would be more beneficial to me > > > How, exactly? Go back and read. > Besides, /your/ convenience isn't the only thing that matters... Nor yours. Just because you can't get your mind around a concept doesn't mean that it's a bad concept, and that no one else can use it. >> and probably >>most Linux users > > > "Most Linux users" don't use experimental filesystems at all... Actually, they do -- ext3 was experimental once. ReiserFS was very experimental once. Please stop bashing it just because it's new/experimental. >> to have metafs supported in both GNOME and KDE, even >>if they still need an emulation layer to support other systems. > > > So Gnome and KDE get larger (and thus slower) for everybody. Smaller (and thus faster) on supported systems, otherwise exactly the same, but maybe a little more modular, which is good. > Besides, Gnome > and KDE will have to agree on the formats involved first, which is /exactly/ > what is supposed to be impossible unless this stuff is implemented in the > kernel... I never said that, but for one thing, whether they do or not, it's nice if my shell and my editor and all the other things that I use don't have to agree on anything to manipulate the formats involved. This is not just about GNOME/KDE. It is about GNOME/KDE not developing an additional layer, that you wouldn't like anyway, that cannot be accessed from anything except GNOME/KDE. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-04 18:47 ` David Masover @ 2005-07-04 20:32 ` Horst von Brand 2005-07-05 22:31 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Horst von Brand @ 2005-07-04 20:32 UTC (permalink / raw) To: David Masover Cc: Horst von Brand, David Weinehall, Markus Törnqvist, Douglas McNaught, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover <ninja@slaphack.com> wrote: > Horst von Brand wrote: > > David Masover <ninja@slaphack.com> wrote: > >>David Weinehall wrote: > >>>On Fri, Jul 01, 2005 at 03:08:58AM -0500, David Masover wrote: > >>>>David Weinehall wrote: [...] > >>Even if they don't, it would be more beneficial to me > > How, exactly? > Go back and read. I have. And have seen /no/ benefit to you. Except, of course, the benefit accrued from some magic in Linus' kernel, by which all format differences go in a puff of smoke if they are implemented inside it, and furthermore all userland gets rebuilt to use the kernel's way overnight. Sorry, I'm quite sceptical on this count. Sure, for complete outsiders it may look that way: Something gets implemented in the kernel, soon many applications are using it, everything is peachy. Except that before there have been /long/ (even bitter) discussions on if/how to do it, what the kernel should do (if anything) and what the application's responsibility should be. Many times it has ended with Linus slamming on the table and saying "no way", or "we do it /this/ way". Then, /when/ this is clear, it gets into the kernel and applications. And you only have ever seen the last phase... which is smooth and fast. Here, the "how should it be done, if at all" discussion has barely started. And AFAICS there are plenty of ways of getting 95% of the advantages /without/ touching the kernel, so that should be the path to follow. Nobody has shown any real evidence at all for this not being so. > > Besides, /your/ convenience isn't the only thing that matters... > Nor yours. Sure enough. > Just because you can't get your mind around a concept > doesn't mean that it's a bad concept, and that no one else can use it. I /do/ get my mind around the concept, it /is/ sometimes useful. It /can/ be done without the kernel, and so /should/ be done that way. That is all. > >> and probably > >>most Linux users > > "Most Linux users" don't use experimental filesystems at all... > Actually, they do -- ext3 was experimental once. Right. And ext2, and ext, and xiafs, and ... So? When they were experimental, only a handful of utter fools commited their data to them. > ReiserFS was very > experimental once. See above. > Please stop bashing it just because it's new/experimental. Sorry? > >> to have metafs supported in both GNOME and KDE, even > >>if they still need an emulation layer to support other systems. > > So Gnome and KDE get larger (and thus slower) for everybody. > Smaller (and thus faster) on supported systems, Sorry, but just because some $DISTRO gives ReiserFS4 as an /option/ doesn't mean they will have everything duplicated as "For ReiserFS4" and "For other filesystems". My assertion stands, until there are ReiserFS4-only distributions. > otherwise exactly the > same, but maybe a little more modular, which is good. Right, having to cope with different ways of representing metadata could induce better code organization. But to get there isn't painless either... > > Besides, Gnome > > and KDE will have to agree on the formats involved first, which is /exactly/ > > what is supposed to be impossible unless this stuff is implemented in the > > kernel... > I never said that, You (and others) have told us time and again that each of them have their own incompatible ways of handling metadata, and that only if this was handled in the kernel they would make use of a common way of managing it... > but for one thing, whether they do or not, it's > nice if my shell and my editor and all the other things that I use > don't have to agree on anything to manipulate the formats involved. Please, read what you just said. Everybody (kernel somewhat included) will have to agree on the format of the metadata. > This is not just about GNOME/KDE. It is about GNOME/KDE not developing > an additional layer, that you wouldn't like anyway, How do you know what I would or would not like? > that cannot be > accessed from anything except GNOME/KDE. So Gnome and KDE agree on some secret formats that only they can handle? Behind their user's backs? As open source? And happily shoot their feet by not giving users much wanted access to said data in decent ways? I see it more on the lines of libmetadata.so (or such), which is used by Gnome, KDE, and whatever else wishes to do so. Even your custom-tailored shell or cat(1). Or just some convention that metadata on ~/my/funky/file resides in ~/.metadata/.my/.funky/.file/metadata. Hey, you could (almost) do that today, by wrappers (perhaps even aliases, or shell functions) around the relevant commands! No kernel at all involved! Not even a miserable library!! -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-04 20:32 ` Horst von Brand @ 2005-07-05 22:31 ` David Masover 2005-07-05 22:43 ` Jeremy Maitin-Shepard 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-05 22:31 UTC (permalink / raw) To: Horst von Brand Cc: David Weinehall, Markus Törnqvist, Douglas McNaught, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Horst von Brand wrote: > David Masover <ninja@slaphack.com> wrote: > >>Horst von Brand wrote: >> >>>David Masover <ninja@slaphack.com> wrote: >>> >>>>David Weinehall wrote: >>>> >>>>>On Fri, Jul 01, 2005 at 03:08:58AM -0500, David Masover wrote: >>>>> >>>>>>David Weinehall wrote: > > > [...] > > >>>>Even if they don't, it would be more beneficial to me > > >>>How, exactly? > > >>Go back and read. > > > I have. And have seen /no/ benefit to you. Except, of course, the benefit > accrued from some magic in Linus' kernel, by which all format differences > go in a puff of smoke if they are implemented inside it, and furthermore > all userland gets rebuilt to use the kernel's way overnight. Let's say cryptocompress gets implemented. Not all of userland rewritten, not even any of userland rewritten, just a cryptocompress plugin for the kernel. And instead of having to learn a new tool, I can just browse around in /meta. Yes, all of userland gets rebuilt, and the world is better for it. But *incrementally*. Not overnight. > Here, the "how should it be done, if at all" discussion has barely started. > And AFAICS there are plenty of ways of getting 95% of the advantages > /without/ touching the kernel, Which advantages can we get without touching the kernel? And why does it make sense to do them outside the kernel? "Because we can" is a little HURD-ish... > so that should be the path to follow. Nobody > has shown any real evidence at all for this not being so. Go back and read again. >> Just because you can't get your mind around a concept >>doesn't mean that it's a bad concept, and that no one else can use it. > > > I /do/ get my mind around the concept, it /is/ sometimes useful. It /can/ > be done without the kernel, No, it can't. Unless you patch *all* of userland. > and so /should/ be done that way. That is all. I'll refer you to my HURD comment: Just about anything done currently in the kernel can be done in userland. Witness HURD. Doesn't mean it should. Witness HURD. >>>> and probably >>>>most Linux users > > >>>"Most Linux users" don't use experimental filesystems at all... > > >>Actually, they do -- ext3 was experimental once. > > > Right. And ext2, and ext, and xiafs, and ... So? When they were > experimental, only a handful of utter fools commited their data to them. But, they had to start somewhere. You seem hell-bent on keeping anything "unproven" out of the kernel and away from users. How do you suggest it "prove" itself, then? >>>> to have metafs supported in both GNOME and KDE, even >>>>if they still need an emulation layer to support other systems. >>> >>>So Gnome and KDE get larger (and thus slower) for everybody. > > >>Smaller (and thus faster) on supported systems, > > > Sorry, but just because some $DISTRO gives ReiserFS4 as an /option/ doesn't > mean they will have everything duplicated as "For ReiserFS4" and "For other > filesystems". My assertion stands, until there are ReiserFS4-only > distributions. Why not? The "For Reiser4" version is just a thin layer on top of metafs. And after all, metafs can be implemented for other filesystems. And, as someone else pointed out, dealing with different systems doing things in different ways is what portability is all about. >>>Besides, Gnome >>>and KDE will have to agree on the formats involved first, which is /exactly/ >>>what is supposed to be impossible unless this stuff is implemented in the >>>kernel... > > >>I never said that, > > > You (and others) Others, then. > have told us time and again that each of them have their > own incompatible ways of handling metadata, and that only if this was > handled in the kernel they would make use of a common way of managing it... All I ever said was that it would be easier for them to get along, and certainly *much* easier to use existing tools (the commandline) to poke inside the metadata, whether it's compatible or not. Ultimately, maybe it leads to cooperation, maybe not. But there are immediate benefits even if it doesn't lead to cooperation. >> but for one thing, whether they do or not, it's >>nice if my shell and my editor and all the other things that I use >>don't have to agree on anything to manipulate the formats involved. > > > Please, read what you just said. Everybody (kernel somewhat included) will > have to agree on the format of the metadata. > > >>This is not just about GNOME/KDE. It is about GNOME/KDE not developing >>an additional layer, that you wouldn't like anyway, > > > How do you know what I would or would not like? Maybe I was generalizing "you". Others don't like Reiser4 as it is because it's got "layering violations". GNOME/KDE developing their own VFS on top of the kernel's for no good reason is a layering violation. Am I misunderstanding? >> that cannot be >>accessed from anything except GNOME/KDE. > > > So Gnome and KDE agree on some secret formats that only they can handle? > Behind their user's backs? As open source? And happily shoot their feet by > not giving users much wanted access to said data in decent ways? Not "secret", but not easy to get at, unless everything links against libmetadata.so or reimplements it. > I see it more on the lines of libmetadata.so (or such), which is used by > Gnome, KDE, and whatever else wishes to do so. Even your custom-tailored > shell or cat(1). Or just some convention that metadata on ~/my/funky/file > resides in ~/.metadata/.my/.funky/.file/metadata. Hey, you could (almost) > do that today, by wrappers (perhaps even aliases, or shell functions) > around the relevant commands! No kernel at all involved! Not even a > miserable library!! Maybe. Then again, maybe not. GNOME, at least, likes XML-based systems. XML is much, much harder for me to deal with from the shell, and certainly doesn't lend itself to as easy scripting. Now, we could tell the GNOME people to make sure to use plain old text files / images everywhere, and make sure everything works for the shell -- and then, on filesystems where small files don't pack so well, you end up taking 10x more space for your metadata, and it's slower for GNOME to manage. Or, we implement /meta. Now I can get to individual bits of metadata from the shell, but GNOME can still use its XML format, or a db database, or some database-like system call like the proposed sys_reiser4 even. It doesn't matter so much anymore how they store metadata, but it becomes easier to access it without having to link against some magic library. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 22:31 ` David Masover @ 2005-07-05 22:43 ` Jeremy Maitin-Shepard 2005-07-05 22:52 ` David Masover 2005-07-06 8:56 ` Christoph Hellwig 0 siblings, 2 replies; 620+ messages in thread From: Jeremy Maitin-Shepard @ 2005-07-05 22:43 UTC (permalink / raw) To: ninja; +Cc: linux-kernel David Masover <ninja@slaphack.com> writes: [snip] >> I have. And have seen /no/ benefit to you. Except, of course, the benefit >> accrued from some magic in Linus' kernel, by which all format differences >> go in a puff of smoke if they are implemented inside it, and furthermore >> all userland gets rebuilt to use the kernel's way overnight. > Let's say cryptocompress gets implemented. Not all of userland > rewritten, not even any of userland rewritten, just a cryptocompress > plugin for the kernel. And instead of having to learn a new tool, I can > just browse around in /meta. What is the relationship between file-as-dir or special meta-data and transparent encryption+compression? I do not see why file-as-dir would require such a special interface. [snip] -- Jeremy Maitin-Shepard ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 22:43 ` Jeremy Maitin-Shepard @ 2005-07-05 22:52 ` David Masover 2005-07-05 23:12 ` Jeremy Maitin-Shepard 2005-07-06 8:56 ` Christoph Hellwig 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-05 22:52 UTC (permalink / raw) To: Jeremy Maitin-Shepard; +Cc: linux-kernel Jeremy Maitin-Shepard wrote: > David Masover <ninja@slaphack.com> writes: > > [snip] > > >>>I have. And have seen /no/ benefit to you. Except, of course, the benefit >>>accrued from some magic in Linus' kernel, by which all format differences >>>go in a puff of smoke if they are implemented inside it, and furthermore >>>all userland gets rebuilt to use the kernel's way overnight. > > >>Let's say cryptocompress gets implemented. Not all of userland >>rewritten, not even any of userland rewritten, just a cryptocompress >>plugin for the kernel. And instead of having to learn a new tool, I can >>just browse around in /meta. > > > What is the relationship between file-as-dir or special meta-data and > transparent encryption+compression? I do not see why file-as-dir would > require such a special interface. I'm ignoring file-as-dir until/unless someone figures out a sane way of doing it. Under Reiser4, think of all files/directories as objects. Meta-data, as accessed through file-as-dir or through /meta, is just a collection of methods to that file/directory. Some methods are just accessor (setter/getter) methods, like foo.mp3/artist -- they are effectively attributes, although there may be a little more logic going on there (foo.mp3/artist probably is connected to foo.mp3's id3 tag). Some methods actually do something, like secret.txt/compression. The reason I'm bringing that up is that some people seem determined to keep the very concept out of the kernel because it's new, strange, and doesn't seem to have any real applications. I want it in because it has real, fairly immediate applications, and much bigger payoffs down the road, and I don't think it's that strange -- it's a nice, clean design. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 22:52 ` David Masover @ 2005-07-05 23:12 ` Jeremy Maitin-Shepard 2005-07-06 0:31 ` Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: Jeremy Maitin-Shepard @ 2005-07-05 23:12 UTC (permalink / raw) To: David Masover; +Cc: linux-kernel David Masover <ninja@slaphack.com> writes: > Jeremy Maitin-Shepard wrote: >> David Masover <ninja@slaphack.com> writes: >> >> [snip] >> >> >>>> I have. And have seen /no/ benefit to you. Except, of course, the benefit >>>> accrued from some magic in Linus' kernel, by which all format differences >>>> go in a puff of smoke if they are implemented inside it, and furthermore >>>> all userland gets rebuilt to use the kernel's way overnight. >> >> >>> Let's say cryptocompress gets implemented. Not all of userland >>> rewritten, not even any of userland rewritten, just a cryptocompress >>> plugin for the kernel. And instead of having to learn a new tool, I can >>> just browse around in /meta. >> >> >> What is the relationship between file-as-dir or special meta-data and >> transparent encryption+compression? I do not see why file-as-dir would >> require such a special interface. I mistyped this --- I meant to ask "why transparent encryption+compression would require such a special interface." > I'm ignoring file-as-dir until/unless someone figures out a sane way of > doing it. [snip] > Some methods actually do something, like secret.txt/compression. [snip] Okay, so you are suggesting that file-as-dir would provide the user interface for enabling the encryption or compression. Alternatively, though, an ioctl could be used to control compression and encryption. -- Jeremy Maitin-Shepard ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 23:12 ` Jeremy Maitin-Shepard @ 2005-07-06 0:31 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-07-06 0:31 UTC (permalink / raw) To: Jeremy Maitin-Shepard; +Cc: David Masover, linux-kernel Jeremy Maitin-Shepard wrote: >Okay, so you are suggesting that file-as-dir would provide the user >interface for enabling the encryption or compression. Alternatively, >though, an ioctl could be used to control compression and encryption. > > > Why is it that /proc does not use an ioctl? Use of metafiles could allow eliminating ioctl(), which most folks I know hate as an interface. Wouldn't it be cleaner if we could find out what ioctl()s are supported by a given file using ls filename/..../ioctl? Excerpt from the ioctl man page, which lacks a list of what features are implemented or how to find out. CONFORMING TO No single standard. Arguments, returns, and semantics of ioctl(2) vary according to the device driver in question (the call is used as a catch-all for operations that don't cleanly fit the Unix stream I/O model). See ioctl_list(2) for a list of many of the known ioctl calls. The ioctl function call appeared in Version 7 AT&T Unix. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 22:43 ` Jeremy Maitin-Shepard 2005-07-05 22:52 ` David Masover @ 2005-07-06 8:56 ` Christoph Hellwig 1 sibling, 0 replies; 620+ messages in thread From: Christoph Hellwig @ 2005-07-06 8:56 UTC (permalink / raw) To: Jeremy Maitin-Shepard; +Cc: ninja, linux-kernel On Tue, Jul 05, 2005 at 06:43:31PM -0400, Jeremy Maitin-Shepard wrote: > > Let's say cryptocompress gets implemented. Not all of userland > > rewritten, not even any of userland rewritten, just a cryptocompress > > plugin for the kernel. And instead of having to learn a new tool, I can > > just browse around in /meta. > > What is the relationship between file-as-dir or special meta-data and > transparent encryption+compression? I do not see why file-as-dir would > require such a special interface. It's not related at all. transparent encryption+compression is a nice extension. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 15:54 ` David Weinehall @ 2005-07-07 8:27 ` mjt 2005-07-07 8:27 ` mjt 1 sibling, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-07-07 8:27 UTC (permalink / raw) To: David Masover, Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 649 bytes --] On Fri, Jul 01, 2005 at 05:54:46PM +0200, David Weinehall wrote: > >Which would neither need VFS changes nor be dependent on Reiser4 in >any way, so I don't see why this thread lives on. Just get down to >business and implement this metafs =) I've been gone for a while and suddenly drowning in mail... Anyway, I don't really like the metafs thing. To access the data, you still need to refactor userspace, so that's not a real advantage. Doing lookups from /meta all the time, instead of in the file-as-dir-whatever... And the best thing to do would be to bring these "Reiser4-specific" enhancements to every FS. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-07-07 8:27 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-07-07 8:27 UTC (permalink / raw) To: David Masover, Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 649 bytes --] On Fri, Jul 01, 2005 at 05:54:46PM +0200, David Weinehall wrote: > >Which would neither need VFS changes nor be dependent on Reiser4 in >any way, so I don't see why this thread lives on. Just get down to >business and implement this metafs =) I've been gone for a while and suddenly drowning in mail... Anyway, I don't really like the metafs thing. To access the data, you still need to refactor userspace, so that's not a real advantage. Doing lookups from /meta all the time, instead of in the file-as-dir-whatever... And the best thing to do would be to bring these "Reiser4-specific" enhancements to every FS. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-07 8:27 ` mjt (?) @ 2005-07-07 14:00 ` David Masover 2005-07-07 17:47 ` Miquel van Smoorenburg -1 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-07 14:00 UTC (permalink / raw) To: Markus Törnqvist Cc: Douglas McNaught, Horst von Brand, Hubert Chan, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Markus Törnqvist wrote: > On Fri, Jul 01, 2005 at 05:54:46PM +0200, David Weinehall wrote: > >>Which would neither need VFS changes nor be dependent on Reiser4 in >>any way, so I don't see why this thread lives on. Just get down to >>business and implement this metafs =) > > > I've been gone for a while and suddenly drowning in mail... > > Anyway, I don't really like the metafs thing. > > To access the data, you still need to refactor userspace, > so that's not a real advantage. Doing lookups from /meta > all the time, instead of in the file-as-dir-whatever... I don't really see the disadvantage. Also, metafs means much less of a fight to get people to adopt the whole meta concept, because it can be done in a POSIX-compliant way which doesn't break tar. File-as-dir is nice if you're using meta files, but it causes lots of unexpected weirdness. I don't think metafs costs us much in performance, and with one or two shell scripts, it wouldn't cost us that much efficiency on the commandline. But, I also like file-as-dir. I think it might be time for a vote. I vote metafs. Or, maybe Hans needs to tell us which way we're going... > And the best thing to do would be to bring these "Reiser4-specific" > enhancements to every FS. Which has nothing to do with whether it's done in "metafs" or not. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-07 14:00 ` David Masover @ 2005-07-07 17:47 ` Miquel van Smoorenburg 0 siblings, 0 replies; 620+ messages in thread From: Miquel van Smoorenburg @ 2005-07-07 17:47 UTC (permalink / raw) To: linux-kernel In article <42CD3580.4020008@slaphack.com>, David Masover <ninja@slaphack.com> wrote: >Markus Törnqvist wrote: >> Anyway, I don't really like the metafs thing. >> >> To access the data, you still need to refactor userspace, >> so that's not a real advantage. Doing lookups from /meta >> all the time, instead of in the file-as-dir-whatever... > >I don't really see the disadvantage. > >Also, metafs means much less of a fight to get people to adopt the whole >meta concept, because it can be done in a POSIX-compliant way which >doesn't break tar. > >File-as-dir is nice if you're using meta files, but it causes lots of >unexpected weirdness. I don't think metafs costs us much in >performance, and with one or two shell scripts, it wouldn't cost us that >much efficiency on the commandline. file-as-dir is an innovation. Metafs is an ugly compromise. Mike. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 5:09 ` Horst von Brand 2005-06-29 13:50 ` Douglas McNaught @ 2005-06-29 20:40 ` Hubert Chan 2005-06-29 21:34 ` Ross Biro 1 sibling, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-06-29 20:40 UTC (permalink / raw) To: Horst von Brand Cc: Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Wed, 29 Jun 2005 01:09:05 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: > Hubert Chan <hubert@uhoreg.ca> wrote: [...] >> Of course. With file-as-dir, you can "cd /usr/games/tetris/..." and >> mess with the game files, or you can run "/usr/games/tetris" and get >> on with ... stacking blocks. > And doing "tar cf /dev/tape /usr/games/tetris" gives you a nice tangle > of undecipherable junk. Uh, yeah. I thought we already discussed this in the last flamewar about merging Reiser4. You can patch tar/cp/backup programs if you want to go the file-as-dir route, or you can try to patch all the other programs if you want to go the xattr route and you really want to edit you extended attributes using $EDITOR (which I'll admit not everyone wants to do). >> The advantage of doing it in the kernel is that you don't need extra >> support from the applications (or GnomeVFS or KDE). > I don't see any advantage here... it has to be implemented, and doing > it in-kernel for one filesystem is /much/ harder than doing it in > userland, where it'll probably work whatever the underlying filesystem > is. Only filesystems that support the capability, or if you have an appropriate filename munging system (see below) or some other system to emulate the capability. > Besides, why should the Tetris scores (or icon, or whatever) reside > inside the executable, when said executable is perhaps shared by a > bunch of machines on the network, each of which would like to keep its > own? Or perhaps the sysadmin is terminally paranoid, and mounts /usr > read-only (Hey, the FHS people defined what goes in there with > /exactly/ that in mind too!). Scores, no. But Tetris still may have resource data that logically belongs to it. Like icons, internationalization data, UI data (say a Glade file), etc. It's all part of the same package; it would be nice (IMHO) if it was grouped together in the filesystem. I don't think MacOS X stores scores in the app file, but they have a lot of other stuff in there -- stuff that I mentioned in the previous paragraph. >> So typing "/usr/games/tetris" from the command prompt does the same >> thing as double-clicking it in the file manager. > Right. And what should happen if I run the (logically equivalent) > directory /home? Or /etc? If there's no binary data attached to /home or /etc (i.e. it only has directory contents), then nothing happens (other than an error message). That's like asking what happens if I do "cat nonexistent.file". If there's no data there to work with, then nothing happens. > What if I want to shove a directory into the tetris executable? > Symlinks? Hard links to files inside? Outside? Symlinks from the > outside in? Hardlinks? What problems do you see with those? > And if you now move that to another filesystem, or ship it to another > machine? What happens when you copy a file from NTFS that has multiple streams? Or a file that has extended attributes onto a filesystem that doesn't support xattr? > No, the answer "All that will be forbidden" is /not/ allowed, you want > this stuff working "normally". "Unpack" the contents into real files > and directories how? If you really want to, you can do some filename munging. The contents of the /usr/games/tetris/ directory could be put in /usr/games/...tetris, or something like that. > "Pack" a directory into one of those structured objects? What would be > the /practical/ difference between the packed and unpacked forms? It's really, really ugly. ;-) It's possible to just put all my files all in a single directory. But sorting things into different directories makes things look much nicer. > (If none, why do it in the first place? If the packed version has > significant restrictions, why use it? If the packed version does have > significant extra capabilities, why not just give those to regular > objects?) > Yet again, what somebody (I forget who) called the "Zero - one - > infinity principle" (I think it was in the area of programming > languages; which arguably are very complex user interfaces, just like > what an operating system provides): It only makes sense giving zero, > exactly one, or how many you wish for of some item/nesting. Here: > Either files got /no/ strange things inside, or /exactly one/ (and > then it becomes questionable, as there is no space for regular data > anymore...), or whatever complex (sub)structure you want. But in the > last case, there is /no/ difference between a file and a > directory... and the whole setup is just mess for mess sake, nothing > else. Files would have two "forks" (can't think of any better terminology right now). A data fork, and a directory fork. --- Begin useless analogy --- Sort of like a real folder in meatspace. I can talk about what I've written on the folder (because I can write stuff on real physical folders), or I can talk about what's inside the folder. Or if I have a document, I can talk about the document itself, or the random post-its, notes, and paperclips that I have attached to it. ---- End useless analogy ---- Right now, most Linux filesystems have xattrs and acls (stored as xattrs). Maybe in the future, the Samba people will get support for multiple streams. If I want to copy my NTFS file to ext3 and preserve the multiple streams, then ext3 will need to support multiple streams now. Now we have two strange things inside. Why not just unify the interface using something like file-as-dir, or metafs that was proposed in this thread, or openat, or something like that? >> Of course, this change does require file managers to understand >> default actions when it's ambiguous what to do on a double-click -- >> but MacOS X has that too, in the form of the "Open as Folder" option >> (or whatever it's called). > Right. For the sake of a filesystem among many on a minority operating > system /all/ GUI programs have to be rewritten. And all command-line > stuff. Just because. Not rewritten; patched. And not all programs. Actually, most programs would work just fine. The Namesys people can tell you how many programs didn't work properly when they tried out the metas directory, and how much patching was required to get them to work properly again. (Not counting tar/cp/backup programs. Everyone already knows about the problems there.) > Please /do/ think the above through, giving reasonable answers for > /all/ of the operations mentioned. Tell what the advantage of all this > would be on a multiuser operating system, when files are shared via > the net, when files are being handled from read-only media (or > filesystems). When different users want different metadata (I'm > interested in the names of members of the band playing a song, you > might want a high-resolution image of the album cover, the next guy > wants the song's lyrics translated into swahili, all for the MP3s on > the shared server or on the CD each of us bought separately). Consider > the scenario where all this works only on /a very few/ machines (no > Sun or *BSD box will have this for a very long time, and to get a > significant fraction of just Linux systems will take some time), for > only a limited selection of tools (existing tools will have to be > rewritten or at least adapted, that doesn't happen overnight). xattr is being added to most filesystems in Linux. It doesn't work everywhere, in all operating systems. Programs will have to be patched to work with it. Does that mean that xattr is a bad idea? > Factor in space and processing requirements for several, even > hundreds, of concurrent users (or versions of metadata at least). How > do you account for that? The 1 byte file with GiB of metadata where > everybody and their pet iguana store their junk is handled how by > quota systems? How do you handle it now when everyone wants different information on a song? I don't really see what the problem is here. Note: I'm not saying that keeping everyone's pet metadata with shared files is a good idea. This is your example. The local library doesn't allow me to make random annotations to their books, but I can do whatever I want with my own books. > What should happen if I email such a file with several versions of > metadata to someone? Do they get just my metadata, or everybody's? If I email a picture that I took to a friend, I don't send them the whole 6 megapixel picture in RAW format. I convert to JPEG and scale it down. If I email a music file that has the lyrics translated into swahili, I can strip that bit out. Nobody's forcing you to keep everything. > If I copy it to, say, a CD for safekeeping for myself? For system > backup purposes? What if I copy such a file (with only my, or > everybody's) metadata back from a backup? You do whatever you want to do with your data. If you want to keep the pet iguana's metadata, go ahead. > Or try to merge it with the version I took with my notebook on a trip, > changing my parts while the otghers change theirs? I'd assume all of > those will requiere special tools, or at least flags selecting the > right operation or some other unspecified magic, and the "simple to > use" just went out for lunch. So what do you do right now when you want to do something like that? File-as-dir (or the other proposals) won't prevent you from doing that. It just allows you more options. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 20:40 ` Hubert Chan @ 2005-06-29 21:34 ` Ross Biro 2005-06-29 23:29 ` Hubert Chan 2005-06-30 3:04 ` Hans Reiser 0 siblings, 2 replies; 620+ messages in thread From: Ross Biro @ 2005-06-29 21:34 UTC (permalink / raw) To: Hubert Chan Cc: Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On 6/29/05, Hubert Chan <hubert@uhoreg.ca> wrote: > On Wed, 29 Jun 2005 01:09:05 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: > > > Hubert Chan <hubert@uhoreg.ca> wrote: [...] > > And doing "tar cf /dev/tape /usr/games/tetris" gives you a nice tangle > > of undecipherable junk. > I'm confused. Can someone on one of these lists enlighten me? How is directories as files logically any different than putting all data into .data files and making all files directories (yes you would need some sort of special handling for files that were really called .data). Then it's just a matter of deciding what happens when you call open and stat on one of these files? For backwards compatibility, current existing system calls have to treat these things as directories. Perhaps an exception could be made for exec. But we could have a whole new set of system calls that treat things as magic, and if files as directories is as cool as many people think, apps will start using the new api. If not, they won't and the new api can be deprecated. Ross ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 21:34 ` Ross Biro @ 2005-06-29 23:29 ` Hubert Chan 2005-07-01 8:06 ` David Masover 2005-06-30 3:04 ` Hans Reiser 1 sibling, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-06-29 23:29 UTC (permalink / raw) To: Ross Biro Cc: Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Wed, 29 Jun 2005 17:34:41 -0400, Ross Biro <ross.biro@gmail.com> said: > I'm confused. Can someone on one of these lists enlighten me? > How is directories as files logically any different than putting all > data into .data files and making all files directories (yes you would > need some sort of special handling for files that were really called > .data). Then it's just a matter of deciding what happens when you > call open and stat on one of these files? Logically, I don't think there is a difference. A filesystem that doesn't support file-as-dir could implement the same functionality that way. [1] In fact, that's essentially what MacOS X/NeXTSTEP does with its bundle format -- it's just a regular directory with regular files inside. The Reiser4 implementation would probably look very similar to that structurally, except that since it understands file-as-dir, it doesn't need to use the .data name, and it should still know where to find it. (Of course, I'm not a Reiser4 developer, and I don't speak for them.) [1] (Well, except that file-as-dir as some people would like it to operate, can do a whole lot of other magic. (e.g. transparent compress, etc. In Reiser4, it's also supposed to allow an interface to filesystem stuff so that you can manipulate parameters such as whether it should use the cryptocompress plugin for that file.) But I think this is a sort of separate issue.) > For backwards compatibility, current existing system calls have to > treat these things as directories. Perhaps an exception could be made > for exec. Which makes the system really quite ugly. I certainly wouldn't enjoy having to type "$EDITOR foo.txt/.data" instead of "$EDITOR foo.txt". If you want to introduce new system calls, you would have to patch all the programs pretty much overnight in order for the system to be useful. It also makes portability a big pain if you want to support systems that don't offer the same system calls. You also get a chicken-and-egg problem. Application developers don't patch their programs because nobody's using the system. Nobody's using the system because no applications support it. > But we could have a whole new set of system calls that treat things as > magic, and if files as directories is as cool as many people think, > apps will start using the new api. If not, they won't and the new api > can be deprecated. File-as-dir doesn't require new system calls (that I know of), which is the whole point of the idea. Existing programs can edit the strange new attributes without being modified. The main thing blocking file-as-dir is that there are some locking(IIRC?) issues. And, of course, some people wouldn't want it to be merged into the mainline kernel. (Of course, the latter doesn't prevent Namesys from maintaining their own patches for people to play around with.) I would like to see file-as-dir so that we can try it out to see if it's as useful as some of us think it is. It might be less useful than I expect, or it could exceed my expectations and someone might come up with some killer use for it, once we can start playing with it. And I think it would be nice if it was merged into mainline so that it gets more exposure, and we have more people trying it out. People like Horst (and probably others, who are less vocal), I think, don't think that it's even worth trying it out because they don't see any major advantages. Or at least they think that the potential negatives outweigh the potential positives. I respect that they have different opinions, but I of course disagree and attempt to convince them otherwise. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 23:29 ` Hubert Chan @ 2005-07-01 8:06 ` David Masover 2005-07-01 8:17 ` Hans Reiser 2005-07-05 21:01 ` Hubert Chan 0 siblings, 2 replies; 620+ messages in thread From: David Masover @ 2005-07-01 8:06 UTC (permalink / raw) To: Hubert Chan Cc: Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hubert Chan wrote: > On Wed, 29 Jun 2005 17:34:41 -0400, Ross Biro <ross.biro@gmail.com> said: > > >>I'm confused. Can someone on one of these lists enlighten me? > > >>How is directories as files logically any different than putting all >>data into .data files and making all files directories (yes you would >>need some sort of special handling for files that were really called >>.data). Then it's just a matter of deciding what happens when you >>call open and stat on one of these files? > > > Logically, I don't think there is a difference. A filesystem that > doesn't support file-as-dir could implement the same functionality that > way. [1] In fact, that's essentially what MacOS X/NeXTSTEP does with its > bundle format -- it's just a regular directory with regular files > inside. I, personally, would hate it if everything in my /bin suddenly became a directory, mainly because everything would stop working. Is that the kind of thing you're suggesting? I'm a little confused about the .data idea, I guess. >>But we could have a whole new set of system calls that treat things as >>magic, and if files as directories is as cool as many people think, >>apps will start using the new api. If not, they won't and the new api >>can be deprecated. > > > File-as-dir doesn't require new system calls (that I know of), which is > the whole point of the idea. Existing programs can edit the strange new > attributes without being modified. That is indeed the point, but scroll down. > The main thing blocking file-as-dir is that there are some > locking(IIRC?) issues. And, of course, some people wouldn't want it to > be merged into the mainline kernel. (Of course, the latter doesn't > prevent Namesys from maintaining their own patches for people to play > around with.) What's the locking issue? I think that was more about transactions... [...] > People like Horst (and probably others, who are less vocal), I think, > don't think that it's even worth trying it out because they don't see > any major advantages. Or at least they think that the potential > negatives outweigh the potential positives. I respect that they have > different opinions, but I of course disagree and attempt to convince > them otherwise. Did the /meta (metafs) idea get killed while I was out? Using that approach, your potential negatives are that apps which crawl the entire FS tree, starting at /, with hardcoded apps for /proc and /sys, are now broken -- but then, /sys already broke them once, so I don't particularly care if we break them again. Potential positives? I think even just because we like the idea is enough, because it doesn't break anything and doesn't really affect anyone who doesn't use it. Maybe there are coding standards, but I think others are working that out now. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 8:06 ` David Masover @ 2005-07-01 8:17 ` Hans Reiser 2005-07-01 8:27 ` David Masover 2005-07-05 21:01 ` Hubert Chan 1 sibling, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-07-01 8:17 UTC (permalink / raw) To: David Masover Cc: Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List It was always the expectation that users would want to have one mountpoint with the files having metafiles as files, and another with the same files but strictly posix, and then their apps can use whichever they have the power to understand. Hans David Masover wrote: > Hubert Chan wrote: > >> On Wed, 29 Jun 2005 17:34:41 -0400, Ross Biro <ross.biro@gmail.com> >> said: >> >> >>> I'm confused. Can someone on one of these lists enlighten me? >> >> >> >>> How is directories as files logically any different than putting all >>> data into .data files and making all files directories (yes you would >>> need some sort of special handling for files that were really called >>> .data). Then it's just a matter of deciding what happens when you >>> call open and stat on one of these files? >> >> >> >> Logically, I don't think there is a difference. A filesystem that >> doesn't support file-as-dir could implement the same functionality that >> way. [1] In fact, that's essentially what MacOS X/NeXTSTEP does with >> its >> bundle format -- it's just a regular directory with regular files >> inside. > > > I, personally, would hate it if everything in my /bin suddenly became > a directory, mainly because everything would stop working. Is that > the kind of thing you're suggesting? > > I'm a little confused about the .data idea, I guess. > >>> But we could have a whole new set of system calls that treat things as >>> magic, and if files as directories is as cool as many people think, >>> apps will start using the new api. If not, they won't and the new api >>> can be deprecated. >> >> >> >> File-as-dir doesn't require new system calls (that I know of), which is >> the whole point of the idea. Existing programs can edit the strange new >> attributes without being modified. > > > That is indeed the point, but scroll down. > >> The main thing blocking file-as-dir is that there are some >> locking(IIRC?) issues. And, of course, some people wouldn't want it to >> be merged into the mainline kernel. (Of course, the latter doesn't >> prevent Namesys from maintaining their own patches for people to play >> around with.) > > > What's the locking issue? I think that was more about transactions... > > [...] > >> People like Horst (and probably others, who are less vocal), I think, >> don't think that it's even worth trying it out because they don't see >> any major advantages. Or at least they think that the potential >> negatives outweigh the potential positives. I respect that they have >> different opinions, but I of course disagree and attempt to convince >> them otherwise. > > > Did the /meta (metafs) idea get killed while I was out? Using that > approach, your potential negatives are that apps which crawl the > entire FS tree, starting at /, with hardcoded apps for /proc and /sys, > are now broken -- but then, /sys already broke them once, so I don't > particularly care if we break them again. > > Potential positives? I think even just because we like the idea is > enough, because it doesn't break anything and doesn't really affect > anyone who doesn't use it. > > Maybe there are coding standards, but I think others are working that > out now. > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 8:17 ` Hans Reiser @ 2005-07-01 8:27 ` David Masover 2005-07-01 8:44 ` Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-01 8:27 UTC (permalink / raw) To: Hans Reiser Cc: Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hans Reiser wrote: > It was always the expectation that users would want to have one > mountpoint with the files having metafiles as files, and another with > the same files but strictly posix, and then their apps can use whichever > they have the power to understand. It was never in the early betas I tried :( I'm proposing (or re-proposing) that the /meta mountpoint be strictly for accessing meta-files, with no intention of eventually using this by default. Furthermore, /meta should follow POSIX anyway, mostly -- no file-as-dir there, either, although you still wouldn't want to use tar on it. With only a few patches, using /meta could be almost as convenient as using file/..metas/foo, and it completely kills the file-as-directory flamewar -- everybody's happy. Or so I thought. It seems that the people arguing for file-as-directory are ignoring /meta, and the people arguing against it are arguing against all meta-files, saying that the good things about meta-files don't justify the risks of file-as-dir. Only once did I see someone bring up /meta. I don't like the idea of having /meta have file-as-dir and everything as we originally wanted, because then we've duplicated the interface. To read the *contents* of /foo, I can either cat /foo or cat /meta/foo. I'd hate to have the POSIX mountpoint still lying around if no one's using it, even more than I hate the idea of "foo" being in two places for no good reason. Is there a technical (performance?) reason that my approach is wrong? (metas go in /meta, files go in /, and everything feels POSIX-y) > Hans > > David Masover wrote: > > >>Hubert Chan wrote: >> >> >>>On Wed, 29 Jun 2005 17:34:41 -0400, Ross Biro <ross.biro@gmail.com> >>>said: >>> >>> >>> >>>>I'm confused. Can someone on one of these lists enlighten me? >>> >>> >>> >>>>How is directories as files logically any different than putting all >>>>data into .data files and making all files directories (yes you would >>>>need some sort of special handling for files that were really called >>>>.data). Then it's just a matter of deciding what happens when you >>>>call open and stat on one of these files? >>> >>> >>> >>>Logically, I don't think there is a difference. A filesystem that >>>doesn't support file-as-dir could implement the same functionality that >>>way. [1] In fact, that's essentially what MacOS X/NeXTSTEP does with >>>its >>>bundle format -- it's just a regular directory with regular files >>>inside. >> >> >>I, personally, would hate it if everything in my /bin suddenly became >>a directory, mainly because everything would stop working. Is that >>the kind of thing you're suggesting? >> >>I'm a little confused about the .data idea, I guess. >> >> >>>>But we could have a whole new set of system calls that treat things as >>>>magic, and if files as directories is as cool as many people think, >>>>apps will start using the new api. If not, they won't and the new api >>>>can be deprecated. >>> >>> >>> >>>File-as-dir doesn't require new system calls (that I know of), which is >>>the whole point of the idea. Existing programs can edit the strange new >>>attributes without being modified. >> >> >>That is indeed the point, but scroll down. >> >> >>>The main thing blocking file-as-dir is that there are some >>>locking(IIRC?) issues. And, of course, some people wouldn't want it to >>>be merged into the mainline kernel. (Of course, the latter doesn't >>>prevent Namesys from maintaining their own patches for people to play >>>around with.) >> >> >>What's the locking issue? I think that was more about transactions... >> >>[...] >> >> >>>People like Horst (and probably others, who are less vocal), I think, >>>don't think that it's even worth trying it out because they don't see >>>any major advantages. Or at least they think that the potential >>>negatives outweigh the potential positives. I respect that they have >>>different opinions, but I of course disagree and attempt to convince >>>them otherwise. >> >> >>Did the /meta (metafs) idea get killed while I was out? Using that >>approach, your potential negatives are that apps which crawl the >>entire FS tree, starting at /, with hardcoded apps for /proc and /sys, >>are now broken -- but then, /sys already broke them once, so I don't >>particularly care if we break them again. >> >>Potential positives? I think even just because we like the idea is >>enough, because it doesn't break anything and doesn't really affect >>anyone who doesn't use it. >> >>Maybe there are coding standards, but I think others are working that >>out now. >> >> > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 8:27 ` David Masover @ 2005-07-01 8:44 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-07-01 8:44 UTC (permalink / raw) To: David Masover Cc: Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover wrote: > Hans Reiser wrote: > >> It was always the expectation that users would want to have one >> mountpoint with the files having metafiles as files, and another with >> the same files but strictly posix, and then their apps can use whichever >> they have the power to understand. > > > It was never in the early betas I tried :( Yes, well, there are a lot of things missing in functionality still from what was, and still is, in the plan. Inheritance, files not listed by readdir, etc., lots of things need coding. We have done the hard stuff first though, so these other things will mostly not be a lot of code.... Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 8:06 ` David Masover 2005-07-01 8:17 ` Hans Reiser @ 2005-07-05 21:01 ` Hubert Chan 2005-07-05 22:01 ` Hans Reiser 1 sibling, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-07-05 21:01 UTC (permalink / raw) To: David Masover Cc: Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Fri, 01 Jul 2005 03:06:19 -0500, David Masover <ninja@slaphack.com> said: > Hubert Chan wrote: >> The main thing blocking file-as-dir is that there are some >> locking(IIRC?) issues. And, of course, some people wouldn't want it >> to be merged into the mainline kernel. (Of course, the latter >> doesn't prevent Namesys from maintaining their own patches for people >> to play around with.) > What's the locking issue? I think that was more about transactions... It was whatever was Al Viro's (technical) complaint about file-as-dir. I don't remember exactly what it was. The technical people know what it is (and the Namesys guys are probably working on it), and the exact issue doesn't concern us non-technical people that much, so I don't feel like looking it up. But if you want to, just look for Al Viro's message in this thread. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 21:01 ` Hubert Chan @ 2005-07-05 22:01 ` Hans Reiser 2005-07-05 22:21 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-07-05 22:01 UTC (permalink / raw) To: Hubert Chan Cc: David Masover, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hubert Chan wrote: >On Fri, 01 Jul 2005 03:06:19 -0500, David Masover <ninja@slaphack.com> said: > > > >>Hubert Chan wrote: >> >> > > > >>>The main thing blocking file-as-dir is that there are some >>>locking(IIRC?) issues. And, of course, some people wouldn't want it >>>to be merged into the mainline kernel. (Of course, the latter >>>doesn't prevent Namesys from maintaining their own patches for people >>>to play around with.) >>> >>> > > > >>What's the locking issue? I think that was more about transactions... >> >> > >It was whatever was Al Viro's (technical) complaint about file-as-dir. >I don't remember exactly what it was. The technical people know what it >is (and the Namesys guys are probably working on it), and the exact >issue doesn't concern us non-technical people that much, so I don't feel >like looking it up. But if you want to, just look for Al Viro's message >in this thread. > > > Cycle detection when hard links to directories are allowed. There is a debate over whether cycle detection is feasible that can only be resolved by working code or a formal proof that it is not computationally feasible. Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 22:01 ` Hans Reiser @ 2005-07-05 22:21 ` David Masover 2005-07-05 22:51 ` Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-05 22:21 UTC (permalink / raw) To: Hans Reiser Cc: Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hans Reiser wrote: > Hubert Chan wrote: > > >>On Fri, 01 Jul 2005 03:06:19 -0500, David Masover <ninja@slaphack.com> said: >> >> >> >> >>>Hubert Chan wrote: >>> >>> >> >> >> >> >>>>The main thing blocking file-as-dir is that there are some >>>>locking(IIRC?) issues. And, of course, some people wouldn't want it >>>>to be merged into the mainline kernel. (Of course, the latter >>>>doesn't prevent Namesys from maintaining their own patches for people >>>>to play around with.) >>>> >>>> >> >> >> >> >>>What's the locking issue? I think that was more about transactions... >>> >>> >> >>It was whatever was Al Viro's (technical) complaint about file-as-dir. >>I don't remember exactly what it was. The technical people know what it >>is (and the Namesys guys are probably working on it), and the exact >>issue doesn't concern us non-technical people that much, so I don't feel >>like looking it up. But if you want to, just look for Al Viro's message >>in this thread. >> >> >> > > Cycle detection when hard links to directories are allowed. There is a > debate over whether cycle detection is feasible that can only be > resolved by working code or a formal proof that it is not > computationally feasible. Ah. But then, one solution was to avoid the issue at all, and have the directory inside a file act as a mountpoint. After all, mount --bind doesn't cause problems... Hey! This sounds like metafs (/meta) already! I wonder if we can do file-as-dir in /meta, and just not support user-created hardlinks there? (other than creating brand-new files, of course...) ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 22:21 ` David Masover @ 2005-07-05 22:51 ` Hans Reiser 2005-07-05 23:00 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-07-05 22:51 UTC (permalink / raw) To: David Masover Cc: Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover wrote: >Hans Reiser wrote: > > >>Hubert Chan wrote: >> >> >> >> >>>On Fri, 01 Jul 2005 03:06:19 -0500, David Masover <ninja@slaphack.com> said: >>> >>> >>> >>> >>> >>> >>>>Hubert Chan wrote: >>>> >>>> >>>> >>>> >>> >>> >>> >>> >>>>>The main thing blocking file-as-dir is that there are some >>>>>locking(IIRC?) issues. And, of course, some people wouldn't want it >>>>>to be merged into the mainline kernel. (Of course, the latter >>>>>doesn't prevent Namesys from maintaining their own patches for people >>>>>to play around with.) >>>>> >>>>> >>>>> >>>>> >>> >>> >>> >>> >>>>What's the locking issue? I think that was more about transactions... >>>> >>>> >>>> >>>> >>>It was whatever was Al Viro's (technical) complaint about file-as-dir. >>>I don't remember exactly what it was. The technical people know what it >>>is (and the Namesys guys are probably working on it), and the exact >>>issue doesn't concern us non-technical people that much, so I don't feel >>>like looking it up. But if you want to, just look for Al Viro's message >>>in this thread. >>> >>> >>> >>> >>> >>Cycle detection when hard links to directories are allowed. There is a >>debate over whether cycle detection is feasible that can only be >>resolved by working code or a formal proof that it is not >>computationally feasible. >> >> > >Ah. But then, one solution was to avoid the issue at all, and have the >directory inside a file act as a mountpoint. After all, mount --bind >doesn't cause problems... > > Can you explain this idea at greater length? >Hey! This sounds like metafs (/meta) already! I wonder if we can do >file-as-dir in /meta, and just not support user-created hardlinks there? > (other than creating brand-new files, of course...) > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 22:51 ` Hans Reiser @ 2005-07-05 23:00 ` David Masover 2005-07-05 23:47 ` Hans Reiser ` (2 more replies) 0 siblings, 3 replies; 620+ messages in thread From: David Masover @ 2005-07-05 23:00 UTC (permalink / raw) To: Hans Reiser Cc: Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hans Reiser wrote: > David Masover wrote: > > >>Hans Reiser wrote: >> >> >> >>>Hubert Chan wrote: >>> >>> >>> >>> >>> >>>>On Fri, 01 Jul 2005 03:06:19 -0500, David Masover <ninja@slaphack.com> said: >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>>>Hubert Chan wrote: >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> >>>> >>>>>>The main thing blocking file-as-dir is that there are some >>>>>>locking(IIRC?) issues. And, of course, some people wouldn't want it >>>>>>to be merged into the mainline kernel. (Of course, the latter >>>>>>doesn't prevent Namesys from maintaining their own patches for people >>>>>>to play around with.) >>>>>> >>>>>> >>>>>> >>>>>> >>>> >>>> >>>> >>>> >>>> >>>>>What's the locking issue? I think that was more about transactions... >>>>> >>>>> >>>>> >>>>> >>>> >>>>It was whatever was Al Viro's (technical) complaint about file-as-dir. >>>>I don't remember exactly what it was. The technical people know what it >>>>is (and the Namesys guys are probably working on it), and the exact >>>>issue doesn't concern us non-technical people that much, so I don't feel >>>>like looking it up. But if you want to, just look for Al Viro's message >>>>in this thread. >>>> >>>> >>>> >>>> >>>> >>> >>>Cycle detection when hard links to directories are allowed. There is a >>>debate over whether cycle detection is feasible that can only be >>>resolved by working code or a formal proof that it is not >>>computationally feasible. >>> >>> >> >>Ah. But then, one solution was to avoid the issue at all, and have the >>directory inside a file act as a mountpoint. After all, mount --bind >>doesn't cause problems... >> >> > > Can you explain this idea at greater length? Just don't allow user-created hardlinks inside either metafs (/meta) or the magical meta directory inside files. In order to do that, one way would be to have "file/..." appear as a mountpoint. I don't know if this is feasable, performance-wise, but I think it would work. >>Hey! This sounds like metafs (/meta) already! I wonder if we can do >>file-as-dir in /meta, and just not support user-created hardlinks there? >>(other than creating brand-new files, of course...) This is still my preferred solution, because it's not any harder or less efficient to develop new apps based on metafs than on file-as-dir, but it means that old apps will see something *entirely* POSIX-compliant, and the strangeness will be confined to /meta. Basically, you have to allow hardlinks in /meta, in case some plugin wants to do that, but if you have files that are also directories, you also have to make sure that users can't create hardlinks. I don't think this would be particularly hard, though. Pretend it's a read-only filesystem unless the user is doing something safe, like creating a brand-new file, deleting an old one, or modifying the contents of an existing one, all assuming that the plugin responsible for the file/directory you're doing this in allows it. But then, if we're doing metafs, I don't think you need file-as-directory, and certain existing tools that we'd like to be able to point at metafs might not like file-as-directory anyway. Now, can anyone think of a situation where we want user-created hardlinks inside metadata? More importantly, what do we do about it? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 23:00 ` David Masover @ 2005-07-05 23:47 ` Hans Reiser 2005-07-06 0:15 ` Hans Reiser 2005-07-05 23:56 ` Hans Reiser 2005-07-06 2:55 ` Horst von Brand 2 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-07-05 23:47 UTC (permalink / raw) To: David Masover Cc: Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller David Masover wrote: >Now, can anyone think of a situation where we want user-created >hardlinks inside metadata? More importantly, what do we do about it? > > I think the equivalent of symlinks would be good enough to get by on for now for most linking of metafiles. Maybe some years from now somebody can fault me for saying this and write a patch to fix it to be better, at which point I will be happy to concede the point. So the basic principal here is, one can have hardlinks to directories without cycles provided that one does not allow any child of the directory to have a hardlink. The question is, how cleanly can that relaxed restriction be coded? Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 23:47 ` Hans Reiser @ 2005-07-06 0:15 ` Hans Reiser 2005-07-06 14:01 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-07-06 0:15 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller If we also add to this the restriction that once you create the file part of a file-directory, you can never hardlink to a child of it, it should then all work, yes? So, /filename/..../owner should be able to avoid colliding with any common names by virtue of the '....', and not letting any filedir (file-directory) have children with links to them should also work. The one thing that seems inelegant is that when you create the file part of a filedir, you must check all its children for hardlinks and fail if they exist, and you must flag all its directory children so that the plugins for them will disallow hardlinks to their children from that point onward. Still, seems workable.... Thanks David, Hans Hans Reiser wrote: >David Masover wrote: > > > >>Now, can anyone think of a situation where we want user-created >>hardlinks inside metadata? More importantly, what do we do about it? >> >> >> >> >I think the equivalent of symlinks would be good enough to get by on for >now for most linking of metafiles. Maybe some years from now somebody >can fault me for saying this and write a patch to fix it to be better, >at which point I will be happy to concede the point. > >So the basic principal here is, one can have hardlinks to directories >without cycles provided that one does not allow any child of the >directory to have a hardlink. The question is, how cleanly can that >relaxed restriction be coded? > >Hans > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 0:15 ` Hans Reiser @ 2005-07-06 14:01 ` David Masover 2005-07-11 4:07 ` Stefan Smietanowski 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-06 14:01 UTC (permalink / raw) To: Hans Reiser Cc: Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller Hans Reiser wrote: > If we also add to this the restriction that once you create the file > part of a file-directory, you can never hardlink to a child of it, it > should then all work, yes? > > So, /filename/..../owner should be able to avoid colliding with any > common names by virtue of the '....', and not letting any filedir > (file-directory) have children with links to them should also work. The > one thing that seems inelegant is that when you create the file part of > a filedir, you must check all its children for hardlinks and fail if > they exist, and you must flag all its directory children so that the > plugins for them will disallow hardlinks to their children from that > point onward. Still, seems workable.... Ok, still haven't heard much discussion of metafs vs file-as-directory, but it seems like it'd be easier in metafs. Basically, we are entirely POSIX compliant outside of metafs, so that /filename is a file and may be hardlinked, and /directory/ is a directory and may not. No problems yet. Inside /meta/inode, we have no problems yet because any hardlinks created outside /meta won't even show up as hardlinks here, so we don't get hardlinked directories. Inside /meta/vfs, where we can find the metadata of '/filename' under '/meta/vfs/filename/...', we have what looks like a problem -- we may have hardlinks created outside metafs, which will show up as two directories inside metafs, so those directories are hardlinks. I don't think that's such a problem, since we won't allow users to change anything in /meta/vfs except when it's inside metadata, such as '/meta/vfs/some/where/...'. Thus, it's impossible to create a hardlink to a directory unless we do it *outside* metafs, as a hardlink to a file. Now, the question I asked is, what if we want hardlinks *inside* metafs, *inside* the metadata for a given file, and would we ever want such a thing? Because we can avoid the whole problem if we just disallow any sys_link calls inside metafs. Of course, sometimes we want to have a chunk of metadata that appears *exactly* as if it were a normal, POSIX-compliant directory tree, such as the contents of a tarball or a zipfile. In that case, we might want to have the "zip" plugin allow hardlinks inside "/meta/file.zip/.../contents" -- the only restriction is that any hardlink made to a file inside 'contents' must be made to another file inside 'contents'. > Hans Reiser wrote: > > >>David Masover wrote: >> >> >> >> >>>Now, can anyone think of a situation where we want user-created >>>hardlinks inside metadata? More importantly, what do we do about it? >>> >>> >>> >>> >> >>I think the equivalent of symlinks would be good enough to get by on for >>now for most linking of metafiles. Maybe some years from now somebody >>can fault me for saying this and write a patch to fix it to be better, >>at which point I will be happy to concede the point. >> >>So the basic principal here is, one can have hardlinks to directories >>without cycles provided that one does not allow any child of the >>directory to have a hardlink. The question is, how cleanly can that >>relaxed restriction be coded? >> >>Hans >> >> >> > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 14:01 ` David Masover @ 2005-07-11 4:07 ` Stefan Smietanowski 2005-07-11 20:50 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Stefan Smietanowski @ 2005-07-11 4:07 UTC (permalink / raw) To: David Masover Cc: Hans Reiser, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > Ok, still haven't heard much discussion of metafs vs file-as-directory, > but it seems like it'd be easier in metafs. Why not implement it inside the directory containg the file ? Ie the metadata for /home/stesmi/foo is in /home/stesmi/.meta/foo This should be suit both camps I'd think? I mean, editing something is easy and you don't have to "know" how to navigate /meta and you don't have the clash of files vs metadata (is /meta/vfs/home/stesmi/foo a file or an attribute called foo of the dir stesmi ?). /home/stesmi/foo <- dir /home/stesmi/.meta/foo <- "dir" containing all metadata /home/stesmi/.meta/foo/attrib <- some attribute called attrib /home/stesmi/foo/bar <- file /home/stesmi/foo/.meta/bar <- "dir" containg all metadata /home/stesmi/foo/.meta/bar/attrib <- some attribute called attrib The file is $dir/$file. The attrib dir is $dir/.meta/$file. The attribute is $dir/.meta/$file/$attribute. If $attrib is something user-editable it's easy to $EDITOR $dir/.meta/$file/$attrib If this has already been taken up, I must've missed it. // Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (MingW32) iD8DBQFC0fBsBrn2kJu9P78RAlt4AJ4qWik6hA4oXBNZMdZ1TkweYrJHmgCdFAY+ m+Qtg9uqBq9m0qKfRkK6iUI= =ghWb -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-11 4:07 ` Stefan Smietanowski @ 2005-07-11 20:50 ` David Masover 2005-07-11 20:54 ` Stefan Smietanowski ` (2 more replies) 0 siblings, 3 replies; 620+ messages in thread From: David Masover @ 2005-07-11 20:50 UTC (permalink / raw) To: Stefan Smietanowski Cc: Hans Reiser, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller Stefan Smietanowski wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > >>Ok, still haven't heard much discussion of metafs vs file-as-directory, >>but it seems like it'd be easier in metafs. > > > Why not implement it inside the directory containg the file ? > > Ie the metadata for /home/stesmi/foo is in /home/stesmi/.meta/foo > > This should be suit both camps I'd think? You still need to figure out the parent of "foo", which isn't necessarily easy, especially considering that even if we store a link to the parent, it will be inside the metadata. Then you have a chicken-and-egg situation. Both camps seem to want to allow easy access to the metadata of a file, whether we're given that file as an inode or as a pathname. That's why I suggested /meta/vfs and /meta/inode -- sometimes I want to look up a file by name, and sometimes by inode, but either way, I should be able to get its metadata. > I mean, editing something is easy and you don't have to "know" how > to navigate /meta But then you have to "know" how to navigate .meta, and more importantly, how to find .meta > and you don't have the clash of files vs metadata > (is /meta/vfs/home/stesmi/foo a file or an attribute called foo of > the dir stesmi ?). So, how do I get metadata of a directory? If the metadata for /home/me is in /home/.meta/me, and the metadata for /home is in /.meta/home, then where is the metadata for / ? > /home/stesmi/foo <- dir > /home/stesmi/.meta/foo <- "dir" containing all metadata > /home/stesmi/.meta/foo/attrib <- some attribute called attrib > /home/stesmi/foo/bar <- file > /home/stesmi/foo/.meta/bar <- "dir" containg all metadata > /home/stesmi/foo/.meta/bar/attrib <- some attribute called attrib [...] > If this has already been taken up, I must've missed it. It looks a lot like how I suggested we resolve the ambiguity within /meta/vfs, only I called it '...' instead of '.meta'. Either way, the major challenge to this method is not so much whether it'd work, but how to choose a suitable delimiter? The delimiter must be standard if we're going to have apps develop for it, and it must not be used in the name of any existing file or directory. I had a long essay here on how '.meta' breaks every recursive operation on directories (rm -rf, tar, mkisofs), but I deleted it when I remembered that I played around with the alpha/beta reiser4 which had a 'metas' dir that did the same thing, but was hidden from the normal directory listing. 'cd metas' worked, 'ls metas' worked, but 'ls' showed no directory called 'metas'. I don't *think* there are any apps that will break if you tell them to open a path that doesn't exist in a directory listing. That is, typing 'vim /home/metas/acl' should always let me edit the acl on /home, and vim should never complain about how /home/metas doesn't show up in /home. A program would have to be pretty retarded to fail on something like that. But, we have to support some pretty retarded programs. That is why I like /meta -- the default view is a completely POSIX-compliant tree that works with tar, and even the /meta view is POSIX-compliant, even if it'd be a bad idea to tar it. Then we don't have to worry at all about stupid programs. How much should we be worrying about this particular brand of stupidity? And what's a good delimiter? -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.323 / Virus Database: 267.8.12/46 - Release Date: 7/11/2005 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-11 20:50 ` David Masover @ 2005-07-11 20:54 ` Stefan Smietanowski 2005-07-11 22:58 ` Hans Reiser 2005-07-12 1:53 ` Horst von Brand 2005-07-12 7:19 ` Neil Brown 2 siblings, 1 reply; 620+ messages in thread From: Stefan Smietanowski @ 2005-07-11 20:54 UTC (permalink / raw) To: David Masover Cc: Hans Reiser, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi. >> Why not implement it inside the directory containg the file ? >> >> Ie the metadata for /home/stesmi/foo is in /home/stesmi/.meta/foo >> >> This should be suit both camps I'd think? > > You still need to figure out the parent of "foo", which isn't > necessarily easy, especially considering that even if we store a link to > the parent, it will be inside the metadata. Then you have a > chicken-and-egg situation. Either you're describing another problem than I'm thinking of or /home/stesmi/.meta/foo 's "file" will always be /home/stesmi/foo . $dir/.meta/$file and $dir/$file . Or do you mean something else when you say "parent" ? Do you mean that if you know $dir/.meta/$foo then you don't know the meta of the parent? Won't $dir/../.meta/$end-of-path/.meta solve that? /home/stesmi/foo <- file /home/stesmi/.meta/foo <- meta of foo /home/stesmi/ <- parentdir /home/.meta/stesmi <- meta of parentdir /home/stesmi/../.meta/stesmi <- also meta of parentdir Or is your problem that you don't know "stesmi" ? Ie you can't figure out what the current dir name is ? So you can't construct /home/stesmi/../stesmi because you don't know the name of the dir? In that case do : /home/stesmi/.meta/. <- meta of parent. Naturally same as /home/.meta/stesmi And if then ".." is mirrored as well is a secondary discussion. /home/stesmi/.meta/. <- meta of "stesmi" /home/stesmi/.meta/.. <- meta of "home" /home/stesmi/.meta/foo <- meta of file "foo" > Both camps seem to want to allow easy access to the metadata of a file, > whether we're given that file as an inode or as a pathname. That's why > I suggested /meta/vfs and /meta/inode -- sometimes I want to look up a > file by name, and sometimes by inode, but either way, I should be able > to get its metadata. > >> I mean, editing something is easy and you don't have to "know" how >> to navigate /meta > > But then you have to "know" how to navigate .meta, and more importantly, > how to find .meta The biggest problem in my eyes is exportability. >> and you don't have the clash of files vs metadata >> (is /meta/vfs/home/stesmi/foo a file or an attribute called foo of >> the dir stesmi ?). > > So, how do I get metadata of a directory? If the metadata for /home/me > is in /home/.meta/me, and the metadata for /home is in /.meta/home, then > where is the metadata for / ? /.meta/. (look above) The "." is also inside .meta >> /home/stesmi/foo <- dir >> /home/stesmi/.meta/foo <- "dir" containing all metadata >> /home/stesmi/.meta/foo/attrib <- some attribute called attrib >> /home/stesmi/foo/bar <- file >> /home/stesmi/foo/.meta/bar <- "dir" containg all metadata >> /home/stesmi/foo/.meta/bar/attrib <- some attribute called attrib > > > [...] > >> If this has already been taken up, I must've missed it. > > > It looks a lot like how I suggested we resolve the ambiguity within > /meta/vfs, only I called it '...' instead of '.meta'. Ah. When I read the talk about "..." I mistook it for something else. I'm sorry. > Either way, the major challenge to this method is not so much whether > it'd work, but how to choose a suitable delimiter? The delimiter must > be standard if we're going to have apps develop for it, and it must not > be used in the name of any existing file or directory. I think "..." and ".meta" both serve as a logical delimiter. However some programs implement their own "..." which would make it clash with them. Naturally if some program created a directory called .meta we're equally screwed. > I had a long essay here on how '.meta' breaks every recursive operation > on directories (rm -rf, tar, mkisofs), but I deleted it when I > remembered that I played around with the alpha/beta reiser4 which had a > 'metas' dir that did the same thing, but was hidden from the normal > directory listing. 'cd metas' worked, 'ls metas' worked, but 'ls' > showed no directory called 'metas'. > > I don't *think* there are any apps that will break if you tell them to > open a path that doesn't exist in a directory listing. That is, typing > 'vim /home/metas/acl' should always let me edit the acl on /home, and > vim should never complain about how /home/metas doesn't show up in > /home. A program would have to be pretty retarded to fail on something > like that. > > But, we have to support some pretty retarded programs. That is why I > like /meta -- the default view is a completely POSIX-compliant tree that > works with tar, and even the /meta view is POSIX-compliant, even if it'd > be a bad idea to tar it. Then we don't have to worry at all about > stupid programs. > > How much should we be worrying about this particular brand of stupidity? > And what's a good delimiter? Very good points. I don't like it being seperate from the dir though so I'm more inclined to support having the metas "close" to the file itself (/home/stesmi/.meta/) instead of seperate from (/meta). // Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (MingW32) iD8DBQFC0tygBrn2kJu9P78RAmBYAJ9YVeAacrREuBFrpGz9Ov5STo6dXwCgxrFi d2BN/T+//CfIZXPCUN9uhC4= =6SfT -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-11 20:54 ` Stefan Smietanowski @ 2005-07-11 22:58 ` Hans Reiser 2005-07-12 2:33 ` Horst von Brand 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-07-11 22:58 UTC (permalink / raw) To: Stefan Smietanowski Cc: David Masover, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller Stefan Smietanowski wrote: > > I think "..." and ".meta" both serve as a logical delimiter. However > some programs implement their own "..." which would make it clash with > them. Naturally if some program created a directory called .meta we're > equally screwed. I chose '....' (four dots) because it clashes with less, not three dots. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-11 22:58 ` Hans Reiser @ 2005-07-12 2:33 ` Horst von Brand 2005-07-12 5:10 ` Stefan Traby 2005-07-12 5:53 ` Hans Reiser 0 siblings, 2 replies; 620+ messages in thread From: Horst von Brand @ 2005-07-12 2:33 UTC (permalink / raw) To: Hans Reiser Cc: Stefan Smietanowski, David Masover, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller Hans Reiser <reiser@namesys.com> wrote: > Stefan Smietanowski wrote: > > I think "..." and ".meta" both serve as a logical delimiter. However > > some programs implement their own "..." which would make it clash with > > them. Naturally if some program created a directory called .meta we're > > equally screwed. > I chose '....' (four dots) because it clashes with less, not three dots. Is this some kind of "My dots are more than yours" contest?! /None/ of them is safe. ".meta", "...", "....", ".this.has.five.dots." are all perfectly legal file (or directory) names, POSIXly. If any one of them won't do, none will. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-12 2:33 ` Horst von Brand @ 2005-07-12 5:10 ` Stefan Traby 2005-07-12 5:53 ` Hans Reiser 1 sibling, 0 replies; 620+ messages in thread From: Stefan Traby @ 2005-07-12 5:10 UTC (permalink / raw) To: Horst von Brand Cc: Hans Reiser, Stefan Smietanowski, David Masover, Hubert Chan, Ross Biro, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller On Mon, Jul 11, 2005 at 10:33:24PM -0400, Horst von Brand wrote: > Hans Reiser <reiser@namesys.com> wrote: > > I chose '....' (four dots) because it clashes with less, not three dots. > > Is this some kind of "My dots are more than yours" contest?! > > /None/ of them is safe. ".meta", "...", "....", ".this.has.five.dots." are > all perfectly legal file (or directory) names, POSIXly. If any one of them > won't do, none will. Correct. I suggest "lost+found". It's also a legal name but was always somewhat special. [not sure if I should place a smiley here] -- ciao - Stefan ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-12 2:33 ` Horst von Brand 2005-07-12 5:10 ` Stefan Traby @ 2005-07-12 5:53 ` Hans Reiser 2005-07-12 23:22 ` David Masover 1 sibling, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-07-12 5:53 UTC (permalink / raw) To: Horst von Brand Cc: Stefan Smietanowski, David Masover, Hubert Chan, Ross Biro, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller Horst von Brand wrote: >Hans Reiser <reiser@namesys.com> wrote: > > >>Stefan Smietanowski wrote: >> >> >>>I think "..." and ".meta" both serve as a logical delimiter. However >>>some programs implement their own "..." which would make it clash with >>>them. Naturally if some program created a directory called .meta we're >>>equally screwed. >>> >>> > > > >>I chose '....' (four dots) because it clashes with less, not three dots. >> >> > >Is this some kind of "My dots are more than yours" contest?! > >/None/ of them is safe. ".meta", "...", "....", ".this.has.five.dots." are >all perfectly legal file (or directory) names, POSIXly. If any one of them >won't do, none will. > > There is a long history of encroaching namespaces by creating new keywords in CS, and it is a survivable problem. Clearcase (the version control filesystem) is an excellent example. They have special meanings for @ and some other common characters, and you have to learn how to escape those characters if you use them in Clearcase. It works, it makes users happy, in practice it is far less of a problem than one might think. I think there were two or three times I had to remember how to escape things in 2 years of using it..... Better to spend one's mind looking for bugs instead of this issue..... ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-12 5:53 ` Hans Reiser @ 2005-07-12 23:22 ` David Masover 2005-07-12 23:38 ` Hans Reiser 2005-07-13 2:09 ` Horst von Brand 0 siblings, 2 replies; 620+ messages in thread From: David Masover @ 2005-07-12 23:22 UTC (permalink / raw) To: Hans Reiser Cc: Horst von Brand, Stefan Smietanowski, Hubert Chan, Ross Biro, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hans Reiser wrote: > Horst von Brand wrote: > > >>Hans Reiser <reiser@namesys.com> wrote: >> >> >> >>>Stefan Smietanowski wrote: >>> >>> >>> >>>>I think "..." and ".meta" both serve as a logical delimiter. However >>>>some programs implement their own "..." which would make it clash with >>>>them. Naturally if some program created a directory called .meta we're >>>>equally screwed. >>>> >>>> >> >> >> >> >>>I chose '....' (four dots) because it clashes with less, not three dots. >>> >>> >> >>Is this some kind of "My dots are more than yours" contest?! >> >>/None/ of them is safe. ".meta", "...", "....", ".this.has.five.dots." are >>all perfectly legal file (or directory) names, POSIXly. If any one of them >>won't do, none will. And, conversely, if any one of them makes sense, they all do. Well, with the exception that some of them have never, ever been used before, and that's what we should be aiming for. No conflicts with existing programs, then new programs can be written to avoid conflicting with us. > There is a long history of encroaching namespaces by creating new > keywords in CS, and it is a survivable problem. Agreed, but not one to be taken too lightly. > Clearcase (the version control filesystem) is an excellent example. > They have special meanings for @ and some other common characters, and > you have to learn how to escape those characters if you use them in > Clearcase. I like using @, though. Without having to escape it. That's why we're trying to find something that people won't actually touch, especially since if we design it right, this will be the last delimiter introduced at the fs/vfs level. > It works, it makes users happy, in practice it is far less of a problem > than one might think. I think there were two or three times I had to > remember how to escape things in 2 years of using it..... > > Better to spend one's mind looking for bugs instead of this issue..... .....if bugs were seen as such a big deal. I think it's far easier to get into the kernel with something ludicrously buggy than something which actually changes fundamental behavior. That is, you can put in an FS which actually corrupts data (such as the old NTFS write support), so long as it doesn't break POSIX, or cause other weird restrictions like "No files named 'metas'" Now, if we can decide that we don't care about being in the vanilla kernel, then we can just call it ".metas" or "lost+found" or whatever and get to work on bug fixes and other much-needed features such as a repacker. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQtRQvXgHNmZLgCUhAQJXWBAAlZggJQGNi2lEZh0MAqnz+rNVT1JYcY/b adHgvVOZZNiJw2GmVjGLIiG5cqSqML1//f0+4XOxjYED2rbfOwBJiDqcq/0IsKPp 5JJflJV2uWkEZukJ+oA5mspfSeKof2N5egBoD1Ije079VBKXdjoN5Kprkbe4ZYZ2 +Yu+w3FpdcaGvSqVlTRHPWsS0je4z8ieh0u6ql+HNR4ze/hKwMw4nrb2sARW9NOQ EZ8Ot5NDhVaxz9Rj72rLVuQrUZEF8b0ihkpmzTauVV/nysEGi33SPqYTF7nYGBnM 5mVY63uXG4zxmThMDpn+J/iofA/hS1fd1bY9tCgF1ZAPu9HrCTnVzKaTYoOq+ciD sY0m7HEc49JfaiyZ/SJGH02WUH3JqLQAVGQftEkqAQLyYdVRbzdBHUVUR2i8hX2M ofFLM6DGJgFK784PfO0sjNboQT5ay4B8js8NltdfytsOVwDzjMST7dZAWcnMgTZU jrMCJuMYeYh8468fy1C9pCMUPahVXvmF4O5Xazt7m9HvV4lxRJIUDZsaR0Volg2K OAQIO3rtT9heK/+/PaUuXX8RYwLQUlhdUmdHfdtjRiLtKRcbw/fYBOqjkgXZfOfy RzyBgD463sVCVAE8qMbAVnnHNGKZ25tyRTYITiTZ2kfnIeURJawj2SZVmn5ezGQG xRnI3+Ir5e0= =moA6 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-12 23:22 ` David Masover @ 2005-07-12 23:38 ` Hans Reiser 2005-07-13 3:43 ` David Masover 2005-07-13 2:09 ` Horst von Brand 1 sibling, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-07-12 23:38 UTC (permalink / raw) To: David Masover Cc: Horst von Brand, Stefan Smietanowski, Hubert Chan, Ross Biro, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller David Masover wrote: > > That's why we're trying to find something that people won't actually > touch, especially since if we design it right, this will be the last > delimiter introduced at the fs/vfs level. Uh, no, there needs to be about a dozen or so more. But not this year. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-12 23:38 ` Hans Reiser @ 2005-07-13 3:43 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-07-13 3:43 UTC (permalink / raw) To: Hans Reiser Cc: Horst von Brand, Stefan Smietanowski, Hubert Chan, Ross Biro, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller Hans Reiser wrote: > David Masover wrote: > > >>That's why we're trying to find something that people won't actually >>touch, especially since if we design it right, this will be the last >>delimiter introduced at the fs/vfs level. > > > Uh, no, there needs to be about a dozen or so more. Where? From what I (vaguely) remember of the future-vision paper, having the meta delimiter lets us do everything else from inside the metas. We can certainly add delimiters to stuff in a meta-dir... -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.323 / Virus Database: 267.8.12/46 - Release Date: 7/11/2005 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-12 23:22 ` David Masover 2005-07-12 23:38 ` Hans Reiser @ 2005-07-13 2:09 ` Horst von Brand 1 sibling, 0 replies; 620+ messages in thread From: Horst von Brand @ 2005-07-13 2:09 UTC (permalink / raw) To: David Masover Cc: Hans Reiser, Horst von Brand, Stefan Smietanowski, Hubert Chan, Ross Biro, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller David Masover <ninja@slaphack.com> wrote: > Hans Reiser wrote: > > Horst von Brand wrote: > >>Hans Reiser <reiser@namesys.com> wrote: > >>>Stefan Smietanowski wrote: [...] > > Better to spend one's mind looking for bugs instead of this issue..... > > .....if bugs were seen as such a big deal. > I think it's far easier to get into the kernel with something > ludicrously buggy than something which actually changes fundamental > behavior. Wonder why.... [Fixing bugs in the $FOO driver or the $BAR filesystem is /easy/, fixing bugs in "fundamental behaviour changes" is /extremely hard/.] > That is, you can put in an FS which actually corrupts data > (such as the old NTFS write support), so long as it doesn't break POSIX, > or cause other weird restrictions like "No files named 'metas'" Because that kind of problems are isolated. If you introduce a change that affects /all/ filesystems, and that change later on has unfixable bugs, or fundamental design issues, it is /a lot/ of work. > Now, if we can decide that we don't care about being in the vanilla > kernel, then we can just call it ".metas" or "lost+found" or whatever > and get to work on bug fixes and other much-needed features such as a > repacker. Great! -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-11 20:50 ` David Masover 2005-07-11 20:54 ` Stefan Smietanowski @ 2005-07-12 1:53 ` Horst von Brand 2005-07-12 7:19 ` Neil Brown 2 siblings, 0 replies; 620+ messages in thread From: Horst von Brand @ 2005-07-12 1:53 UTC (permalink / raw) To: David Masover Cc: Stefan Smietanowski, Hans Reiser, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller David Masover <ninja@slaphack.com> wrote: [...] > Both camps seem to want to allow easy access to the metadata of a > file, whether we're given that file as an inode or as a pathname. > That's why I suggested /meta/vfs and /meta/inode -- sometimes I want > to look up a file by name, and sometimes by inode, but either way, I > should be able to get its metadata. You should /never/ give access just by inode, as that makes it very easy to bypass security given by directory permissions. > > I mean, editing something is easy and you don't have to "know" how > > to navigate /meta > But then you have to "know" how to navigate .meta, and more > importantly, how to find .meta And what the heck the format of the metadata is today/for this particular file. [...] > Either way, the major challenge to this method is not so much whether > it'd work, but how to choose a suitable delimiter? The delimiter must > be standard if we're going to have apps develop for it, and it must > not be used in the name of any existing file or directory. The only characters that aren't used in filenames today are '/' and '\0'. > I had a long essay here on how '.meta' breaks every recursive > operation on directories (rm -rf, tar, mkisofs), but I deleted it when > I remembered that I played around with the alpha/beta reiser4 which > had a 'metas' dir that did the same thing, but was hidden from the > normal directory listing. 'cd metas' worked, 'ls metas' worked, but > 'ls' showed no directory called 'metas'. And if I try to create a file or directory called metas? > I don't *think* there are any apps that will break if you tell them to > open a path that doesn't exist in a directory listing. That is, > typing 'vim /home/metas/acl' should always let me edit the acl on > /home, and vim should never complain about how /home/metas doesn't > show up in /home. A program would have to be pretty retarded to fail > on something like that. Think a GUI that reads the directory to find out what is available and present them as icons for mousing around. Then there is no way to futz around with your metadata. Think filename completion, bash style. No, not just bash uses this. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-11 20:50 ` David Masover 2005-07-11 20:54 ` Stefan Smietanowski 2005-07-12 1:53 ` Horst von Brand @ 2005-07-12 7:19 ` Neil Brown 2005-07-12 7:45 ` Hans Reiser 2005-07-12 23:13 ` David Masover 2 siblings, 2 replies; 620+ messages in thread From: Neil Brown @ 2005-07-12 7:19 UTC (permalink / raw) To: David Masover Cc: Stefan Smietanowski, Hans Reiser, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller On Monday July 11, ninja@slaphack.com wrote: > Stefan Smietanowski wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > > >>Ok, still haven't heard much discussion of metafs vs file-as-directory, > >>but it seems like it'd be easier in metafs. > > > > > > Why not implement it inside the directory containg the file ? > > > > Ie the metadata for /home/stesmi/foo is in /home/stesmi/.meta/foo > > > > This should be suit both camps I'd think? > > You still need to figure out the parent of "foo", which isn't > necessarily easy, especially considering that even if we store a link to > the parent, it will be inside the metadata. Then you have a > chicken-and-egg situation. > > Both camps seem to want to allow easy access to the metadata of a file, > whether we're given that file as an inode or as a pathname. That's why > I suggested /meta/vfs and /meta/inode -- sometimes I want to look up a > file by name, and sometimes by inode, but either way, I should be able > to get its metadata. Well, it's not really 'as an inode or as a pathname'. It is 'as an open file descriptor or as a path name' which is an important difference. Maybe it is worth repeating Al Viro's suggestion at this point. I don't have a reference but the idea was basically that if you open "/foo" and get filedescriptor N, then /proc/self/fds/N-meta is a directory which contains all the meta stuff for "/foo". Then it is trivial to get the 'meta' stuff given a filedescriptor and if you have a pathname, you can always get yourself a filedescriptor. Then to allow cat /home/friend/foo/.meta.../perms you write a little .so which redefines open, stat, and a few others to scan for ".meta..." in the pathname and to the necessary magic, and export LD_PRELOAD=/that/.so Finally you write a killer app which fundamentally relies on this functionality, get everyone addicted to it, and *then* (and only then) start trying to get this functionality into the kernel. i.e. prototype new semantics in userspace. NeilBrown ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-12 7:19 ` Neil Brown @ 2005-07-12 7:45 ` Hans Reiser 2005-07-13 0:05 ` Neil Brown 2005-07-12 23:13 ` David Masover 1 sibling, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-07-12 7:45 UTC (permalink / raw) To: Neil Brown Cc: David Masover, Stefan Smietanowski, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller Neil Brown wrote: > > >Maybe it is worth repeating Al Viro's suggestion at this point. I >don't have a reference but the idea was basically that if you open >"/foo" and get filedescriptor N, then > /proc/self/fds/N-meta >is a directory which contains all the meta stuff for "/foo". >Then it is trivial to get the 'meta' stuff given a filedescriptor and >if you have a pathname, you can always get yourself a filedescriptor. > > This sound like it might be cute, but filedescriptors are too heavy weight for stat data accesses in quantity. In general, the whole file handle paradigm is too heavy for lightweight files. Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-12 7:45 ` Hans Reiser @ 2005-07-13 0:05 ` Neil Brown 2005-07-13 1:20 ` Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: Neil Brown @ 2005-07-13 0:05 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Stefan Smietanowski, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller On Tuesday July 12, reiser@namesys.com wrote: > Neil Brown wrote: > > > > > > >Maybe it is worth repeating Al Viro's suggestion at this point. I > >don't have a reference but the idea was basically that if you open > >"/foo" and get filedescriptor N, then > > /proc/self/fds/N-meta > >is a directory which contains all the meta stuff for "/foo". > >Then it is trivial to get the 'meta' stuff given a filedescriptor and > >if you have a pathname, you can always get yourself a filedescriptor. > > > > > This sound like it might be cute, but filedescriptors are too heavy > weight for stat data accesses in quantity. > > In general, the whole file handle paradigm is too heavy for lightweight > files. That may well be true, but is completely orthogonal to filesystem name semantics. If you find file descriptors too slow, come up with an alternate (I suspect you have in the reiser4 syscall, but I haven't looked at that yet), implement it in the VFS, and show the world benchmarks of real-world applications that go faster with this new interface. I doubt that you would then have a great deal of trouble in getting the interface accepted (some trouble of course as you will need to convince a few people, but numbers speak quite loudly). I suspect that there might need to be a new internal interface into filesystems, and filesystems which don't provide that will not get the same speed benefit, but that is perfectly acceptable. NeilBrown ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-13 0:05 ` Neil Brown @ 2005-07-13 1:20 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-07-13 1:20 UTC (permalink / raw) To: Neil Brown Cc: David Masover, Stefan Smietanowski, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller, Vladimir Demidov Neil Brown wrote: >On Tuesday July 12, reiser@namesys.com wrote: > > >>Neil Brown wrote: >> >> >> >>>Maybe it is worth repeating Al Viro's suggestion at this point. I >>>don't have a reference but the idea was basically that if you open >>>"/foo" and get filedescriptor N, then >>> /proc/self/fds/N-meta >>>is a directory which contains all the meta stuff for "/foo". >>>Then it is trivial to get the 'meta' stuff given a filedescriptor and >>>if you have a pathname, you can always get yourself a filedescriptor. >>> >>> >>> >>> >>This sound like it might be cute, but filedescriptors are too heavy >>weight for stat data accesses in quantity. >> >>In general, the whole file handle paradigm is too heavy for lightweight >>files. >> >> > >That may well be true, but is completely orthogonal to filesystem name >semantics. > >If you find file descriptors too slow, come up with an alternate (I >suspect you have in the reiser4 syscall, but I haven't looked at >that yet), implement it in the VFS, and show the world benchmarks of >real-world applications that go faster with this new interface. > >I doubt that you would then have a great deal of trouble in getting >the interface accepted (some trouble of course as you will need to >convince a few people, but numbers speak quite loudly). > >I suspect that there might need to be a new internal interface into >filesystems, and filesystems which don't provide that will not get the >same speed benefit, but that is perfectly acceptable. > >NeilBrown > > > > We need time, and then we can demonstrate sys_reiser4, it is not ready for showing yet..... Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-12 7:19 ` Neil Brown 2005-07-12 7:45 ` Hans Reiser @ 2005-07-12 23:13 ` David Masover 2005-07-12 23:40 ` Neil Brown 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-12 23:13 UTC (permalink / raw) To: Neil Brown Cc: Stefan Smietanowski, Hans Reiser, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Neil Brown wrote: > On Monday July 11, ninja@slaphack.com wrote: > >>Stefan Smietanowski wrote: >> >>>-----BEGIN PGP SIGNED MESSAGE----- >>>Hash: SHA1 >>> >>> >>> >>>>Ok, still haven't heard much discussion of metafs vs file-as-directory, >>>>but it seems like it'd be easier in metafs. >>> >>> >>>Why not implement it inside the directory containg the file ? >>> >>>Ie the metadata for /home/stesmi/foo is in /home/stesmi/.meta/foo >>> >>>This should be suit both camps I'd think? >> >>You still need to figure out the parent of "foo", which isn't >>necessarily easy, especially considering that even if we store a link to >>the parent, it will be inside the metadata. Then you have a >>chicken-and-egg situation. >> >>Both camps seem to want to allow easy access to the metadata of a file, >>whether we're given that file as an inode or as a pathname. That's why >>I suggested /meta/vfs and /meta/inode -- sometimes I want to look up a >>file by name, and sometimes by inode, but either way, I should be able >>to get its metadata. > > > Well, it's not really 'as an inode or as a pathname'. It is 'as an > open file descriptor or as a path name' which is an important > difference. > > Maybe it is worth repeating Al Viro's suggestion at this point. I > don't have a reference but the idea was basically that if you open > "/foo" and get filedescriptor N, then > /proc/self/fds/N-meta How am I supposed to get there with a shell script? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQtROqXgHNmZLgCUhAQLRcg/+I9PWSmFXRwKtj7pnEeMXOCjiTo6GQE3O 61fjH3f6aL9Ydkip4eXum3S14cioiU9bzC11GA5kRIM+W1DKcYex1dIpivrtF9Ht Rvozn9x2TP5tacDmSfqRJXvAB+xTRtZOu+M/rDjXdLsriDJDA0AdyDH8Yo/8WMbU 6i1DWzLTO0vHS3kEb/8oqgBj7sQ63sS/4KVszBx6+bN0KOikXbORDu6efKjC9w21 3DZPnBG0B03smhdCygd0j0Zmh0JEaZHfuFgNK1ZmRzipbvvUBDtdKY5MJ6f4pHnA GBO8ybsXp9qxNQr6DNenF/wbAT6n3dMyE/AWuql+qx3iumSwx/Prh7xDAhBZBMXp Oin7hOa1i583cdju4ErSBPaciRzumGluY6gbFvVA8Yva+IjPxxjPtfLwalK11cH1 k4oQO5Par1W0TmMOpc0PQ/bEeEHHxUcn1ToeJa4NYJWIiBe+UHMb/AyI4hKjSIkt Kp0wrCPBjRfuBCHXXL89bWZoSeSFkN8fAyOxBV928naxxr8e+vCPUX1/H3ca7UsB Nxg0Vzt4V9tz4xCw4QAy810Uya8/HSm3aVhqEzjHKBoKboHrMVDJvxRQxfkqQcnC 4jIFYPBdHgGw7OONyhfbgTPLIm1OCNPpcRkS4aidHqg0DkDU50h6zFQkhG5Xwh5Z x5REgxbqD+A= =FGm4 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-12 23:13 ` David Masover @ 2005-07-12 23:40 ` Neil Brown 0 siblings, 0 replies; 620+ messages in thread From: Neil Brown @ 2005-07-12 23:40 UTC (permalink / raw) To: David Masover Cc: Stefan Smietanowski, Hans Reiser, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller On Tuesday July 12, ninja@slaphack.com wrote: > > > > Maybe it is worth repeating Al Viro's suggestion at this point. I > > don't have a reference but the idea was basically that if you open > > "/foo" and get filedescriptor N, then > > /proc/self/fds/N-meta > > How am I supposed to get there with a shell script? function get_meta() { var=$1 file=$2 meta=$3 val=`cat /proc/self/fd/3-meta/$meta < $file` eval var=\$val } then get_meta varname /home/foo/bar username will read the 'username' meta-file of 'home/foo/bar' and place it in varname. Is that what you wanted? NeilBrown ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 23:00 ` David Masover 2005-07-05 23:47 ` Hans Reiser @ 2005-07-05 23:56 ` Hans Reiser 2005-07-06 0:50 ` Alexander G. M. Smith 2005-07-06 1:59 ` Neil Brown 2005-07-06 2:55 ` Horst von Brand 2 siblings, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-07-05 23:56 UTC (permalink / raw) To: David Masover Cc: Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller I got it slightly wrong. One can have hardlinks to a directory without cycles provided that one does not have hardlinks from the children of that directory to any file not a child of that directory. (Mountpoints currently implement that restriction.) Question: can one implement that lesser restriction above with elegant code? Is the greater restriction below easier to code? (If no to the first and yes to the second is correct, then I can accept the greater restriction described below.) One can have hardlinks to directories without cycles provided that one does not allow any child of the directory to have a hardlink. Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 23:56 ` Hans Reiser @ 2005-07-06 0:50 ` Alexander G. M. Smith 2005-07-06 1:16 ` Hubert Chan 2005-07-06 1:59 ` Neil Brown 1 sibling, 1 reply; 620+ messages in thread From: Alexander G. M. Smith @ 2005-07-06 0:50 UTC (permalink / raw) To: Hans Reiser Cc: hubert, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, ninja Hans Reiser wrote on Tue, 05 Jul 2005 16:56:02 -0700: > One can have hardlinks to a directory without cycles provided that one > does not have hardlinks from the children of that directory to any file > not a child of that directory. (Mountpoints currently implement that > restriction.) > > Question: can one implement that lesser restriction above with elegant > code? Is the greater restriction below easier to code? (If no to the > first and yes to the second is correct, then I can accept the greater > restriction described below.) > > One can have hardlinks to directories > without cycles provided that one does not allow any child of the > directory to have a hardlink. That sounds equivalent to no hard links (other than the usual parent directory one). If there's any directory with two links to it, then there will be a cycle somewhere! The tradeoff is the size of the cycle to the amount of memory and other resources needed to clean up the cycle when rename/deletion changes the directory graph structure. Various artificial limits can be imposed, such as not creating a link which would result in a cycle of more than some number of files/directories (adjust so that cleanup fits in the memory size of the machine). Or just fail the cleanup if it is too complex (user deletes individual items in the cycle then tries again). Perhaps you are thinking about the speed improvement for cycle checking that comes from marking directories as known to not contain children pointing outside the graph rooted at that directory. Then that directory can be treated as an ordinary directory in the cycle cleanup procedure, avoiding the need to trace down the links to all its descendants. That's essentially what your mount point restriction is. - Alex ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 0:50 ` Alexander G. M. Smith @ 2005-07-06 1:16 ` Hubert Chan 2005-07-06 6:44 ` Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-07-06 1:16 UTC (permalink / raw) To: Alexander G. M. Smith Cc: ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, ninja On Tue, 05 Jul 2005 20:50:08 -0400 EDT, "Alexander G. M. Smith" <agmsmith@rogers.com> said: > That sounds equivalent to no hard links (other than the usual parent > directory one). If there's any directory with two links to it, then > there will be a cycle somewhere! What we want is no directed cycles. That is A is the parent of B is the parent of C is the parent of A. We don't care about A is the parent of B is the parent of C; A is the parent of B is the parent of C. OK, here's a random idea that just popped into my head, and to which I've given little thought (read: none whatsoever), and may be the stupidest idea ever proposed on LKML, but thought I would just toss it out to see if it could stimulate someone to come up with something better (read: sane): Conceptually, foo/.... is just a symlink to /meta/[filesystem]/[inode of foo]. And a question: is it feasible to store, for each inode, its parent(s), instead of just the hard link count? -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 1:16 ` Hubert Chan @ 2005-07-06 6:44 ` Hans Reiser 2005-07-06 13:09 ` David Masover 2005-07-06 18:52 ` Jonathan Briggs 0 siblings, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-07-06 6:44 UTC (permalink / raw) To: Hubert Chan Cc: Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, ninja, vitaly Hubert Chan wrote: >On Tue, 05 Jul 2005 20:50:08 -0400 EDT, "Alexander G. M. Smith" <agmsmith@rogers.com> said: > > > >>That sounds equivalent to no hard links (other than the usual parent >>directory one). If there's any directory with two links to it, then >>there will be a cycle somewhere! >> >> > >What we want is no directed cycles. That is A is the parent of B is the >parent of C is the parent of A. We don't care about A is the parent of >B is the parent of C; A is the parent of B is the parent of C. > >OK, here's a random idea that just popped into my head, and to which >I've given little thought (read: none whatsoever), and may be the >stupidest idea ever proposed on LKML, but thought I would just toss it >out to see if it could stimulate someone to come up with something >better (read: sane): Conceptually, foo/.... is just a symlink to >/meta/[filesystem]/[inode of foo]. > > Except that we want the metafiles to go away when the base file goes away. >And a question: is it feasible to store, for each inode, its parent(s), >instead of just the hard link count? > > Ooh, now that is an interesting old idea I haven't considered in 20 years.... makes fsck more robust too.... ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 6:44 ` Hans Reiser @ 2005-07-06 13:09 ` David Masover 2005-07-06 13:43 ` Adrian Ulrich 2005-07-06 18:07 ` Hans Reiser 2005-07-06 18:52 ` Jonathan Briggs 1 sibling, 2 replies; 620+ messages in thread From: David Masover @ 2005-07-06 13:09 UTC (permalink / raw) To: Hans Reiser Cc: Hubert Chan, Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, vitaly Hans Reiser wrote: > Hubert Chan wrote: > > >>On Tue, 05 Jul 2005 20:50:08 -0400 EDT, "Alexander G. M. Smith" <agmsmith@rogers.com> said: >> >> >> >> >>>That sounds equivalent to no hard links (other than the usual parent >>>directory one). If there's any directory with two links to it, then >>>there will be a cycle somewhere! >>> >>> >> >>What we want is no directed cycles. That is A is the parent of B is the >>parent of C is the parent of A. We don't care about A is the parent of >>B is the parent of C; A is the parent of B is the parent of C. >> >>OK, here's a random idea that just popped into my head, and to which >>I've given little thought (read: none whatsoever), and may be the >>stupidest idea ever proposed on LKML, but thought I would just toss it >>out to see if it could stimulate someone to come up with something >>better (read: sane): Conceptually, foo/.... is just a symlink to >>/meta/[filesystem]/[inode of foo]. >> >> > > Except that we want the metafiles to go away when the base file goes away. Only, /meta is a filesystem that already makes stuff go away for us, so all we have left is the issue of whether using /meta costs us performance, or whether breaking POSIX to add a symlink (such as foo/...) really gives us that much more usability. I don't know the first thing about whether it costs us performance, although it seems like it could be negligable considering the existance of mount --bind. I don't think file-as-dir gives us that much more usability, because we can always create a simple program or shell script that 'cd's us into metadata. It's still easier than having a simple program that manipulates the metadata directly, because this way we can do 'cd' and 'ls' and so on inside the metadata directory. And, once we start talking about applications, /meta will be more readily supported (as in, some apps will go through a pathname and stop when they get to a file, and then there's tar). On apps which don't have direct support for /meta, you'd be navigating to the file in question and then manually typing '...' into the dialog, so I don't see why typing '...' at the end is better than typing '/meta' or '/meta/vfs' at the beginning. That said, I'm still not entirely sure how we get /meta/vfs to work other than adding a '...' sort of delimiter anyway. >>And a question: is it feasible to store, for each inode, its parent(s), >>instead of just the hard link count? >> >> > > Ooh, now that is an interesting old idea I haven't considered in 20 > years.... makes fsck more robust too.... Doesn't it make directory operations slower, too? And, will it require a format change? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 13:09 ` David Masover @ 2005-07-06 13:43 ` Adrian Ulrich 2005-07-06 14:11 ` David Masover 2005-07-06 18:07 ` Hans Reiser 1 sibling, 1 reply; 620+ messages in thread From: Adrian Ulrich @ 2005-07-06 13:43 UTC (permalink / raw) To: David Masover Cc: reiser, hubert, agmsmith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, vitaly > so all we have left is the issue of whether using /meta costs us > performance, or whether breaking POSIX to add a symlink (such as > foo/...) really gives us that much more usability. IMHO '/meta' isn't such a good idea, because a chrooted application won't be able to use it. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 13:43 ` Adrian Ulrich @ 2005-07-06 14:11 ` David Masover 2005-07-06 14:55 ` Adrian Ulrich 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-06 14:11 UTC (permalink / raw) To: Adrian Ulrich Cc: reiser, hubert, agmsmith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, vitaly Adrian Ulrich wrote: >>so all we have left is the issue of whether using /meta costs us >>performance, or whether breaking POSIX to add a symlink (such as >>foo/...) really gives us that much more usability. > > > IMHO '/meta' isn't such a good idea, because a chrooted application > won't be able to use it. mount --bind. Is there a performance hit for having too many of those? mount --bind /meta/vfs/some/chroot /some/chroot/meta Also, maybe a separately-mounted /meta, maybe with something like '-o root=' I can't think of when you'd have a chrooted application that uses /meta, but wasn't written with /meta in mind, so as to have these mount commands in its init scripts. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 14:11 ` David Masover @ 2005-07-06 14:55 ` Adrian Ulrich 0 siblings, 0 replies; 620+ messages in thread From: Adrian Ulrich @ 2005-07-06 14:55 UTC (permalink / raw) To: David Masover Cc: reiser, hubert, agmsmith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, vitaly > mount --bind /meta/vfs/some/chroot /some/chroot/meta This maybe funny if you got 1-2 chrooted applications. But it will be a nightmare if you got 20-30 chrooted applications. -- We're working on it, slowly but surely...or not-so-surely in the spots we're not so sure... -- Larry Wall ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 13:09 ` David Masover 2005-07-06 13:43 ` Adrian Ulrich @ 2005-07-06 18:07 ` Hans Reiser 2005-07-06 20:47 ` David Masover 1 sibling, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-07-06 18:07 UTC (permalink / raw) To: David Masover Cc: Hubert Chan, Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, vitaly David Masover wrote: > And, once we start talking about applications, /meta will be more > readily supported (as in, some apps will go through a pathname and > stop when they get to a file, and then there's tar). On apps which > don't have direct support for /meta, you'd be navigating to the file > in question and then manually typing '...' into the dialog, so I don't > see why typing '...' at the end is better than typing '/meta' or > '/meta/vfs' at the beginning. Performance. If you type it at the end, and you already have done the lookup of the filename, then you can go from the file to one of its methods, instead of a complete new traversal of another tree under /meta > > That said, I'm still not entirely sure how we get /meta/vfs to work > other than adding a '...' sort of delimiter anyway. > >>> And a question: is it feasible to store, for each inode, its parent(s), >>> instead of just the hard link count? >>> >>> >> >> Ooh, now that is an interesting old idea I haven't considered in 20 >> years.... makes fsck more robust too.... > > > Doesn't it make directory operations slower, too? Not sure. It consumes space though. > > And, will it require a format change? > Yes, but we have plugins now, so..... ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 18:07 ` Hans Reiser @ 2005-07-06 20:47 ` David Masover 2005-07-06 22:49 ` Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-06 20:47 UTC (permalink / raw) To: Hans Reiser Cc: Hubert Chan, Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, vitaly Hans Reiser wrote: > David Masover wrote: > > >>And, once we start talking about applications, /meta will be more >>readily supported (as in, some apps will go through a pathname and >>stop when they get to a file, and then there's tar). On apps which >>don't have direct support for /meta, you'd be navigating to the file >>in question and then manually typing '...' into the dialog, so I don't >>see why typing '...' at the end is better than typing '/meta' or >>'/meta/vfs' at the beginning. > > > Performance. If you type it at the end, and you already have done the > lookup of the filename, then you can go from the file to one of its > methods, instead of a complete new traversal of another tree under /meta Only, it's a different view of the same tree. That may not matter performance-wise, though. Also, for performance-critical applications, the /meta tree is pretty easy -- it becomes more like /meta/inode/some_inode_number/. There's also the chroot issue, though. >>That said, I'm still not entirely sure how we get /meta/vfs to work >>other than adding a '...' sort of delimiter anyway. >> >> >>>>And a question: is it feasible to store, for each inode, its parent(s), >>>>instead of just the hard link count? >>>> >>>> >>> >>>Ooh, now that is an interesting old idea I haven't considered in 20 >>>years.... makes fsck more robust too.... >> >> >>Doesn't it make directory operations slower, too? > > > Not sure. It consumes space though. > > >>And, will it require a format change? >> > > Yes, but we have plugins now, so..... So, will the format change happen at mount time? Will it need a special mount flag? Will I need to use debugfs or some other offline tool? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 20:47 ` David Masover @ 2005-07-06 22:49 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-07-06 22:49 UTC (permalink / raw) To: David Masover Cc: Hubert Chan, Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, vitaly David Masover wrote: > So, will the format change happen at mount time? Will it need a > special mount flag? Will I need to use debugfs or some other offline > tool? First we make sure we have the right answer. Have we solved the cycle problem? Can we run out of memory as Horst/Nikita suggest? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 6:44 ` Hans Reiser 2005-07-06 13:09 ` David Masover @ 2005-07-06 18:52 ` Jonathan Briggs 2005-07-06 19:51 ` Hubert Chan 2005-07-07 6:42 ` Hans Reiser 1 sibling, 2 replies; 620+ messages in thread From: Jonathan Briggs @ 2005-07-06 18:52 UTC (permalink / raw) To: Hans Reiser Cc: Hubert Chan, Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, ninja, vitaly [-- Attachment #1: Type: text/plain, Size: 770 bytes --] On Tue, 2005-07-05 at 23:44 -0700, Hans Reiser wrote: > Hubert Chan wrote: > >And a question: is it feasible to store, for each inode, its parent(s), > >instead of just the hard link count? > > > > > Ooh, now that is an interesting old idea I haven't considered in 20 > years.... makes fsck more robust too.... Hey, sounds like the idea I proposed a couple months ago of storing the path names in each file, instead of in directories. Only better, since each path component isn't text but a link instead. It still has the performance and locking problem of having to update every child file when moving a directory tree to a new parent. On the other hand, maybe the benefit is worth the cost. -- Jonathan Briggs <jbriggs@esoft.com> eSoft, Inc. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 18:52 ` Jonathan Briggs @ 2005-07-06 19:51 ` Hubert Chan 2005-07-06 20:33 ` Jonathan Briggs 2005-07-06 20:33 ` Horst von Brand 2005-07-07 6:42 ` Hans Reiser 1 sibling, 2 replies; 620+ messages in thread From: Hubert Chan @ 2005-07-06 19:51 UTC (permalink / raw) To: Jonathan Briggs Cc: Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, ninja, vitaly On Wed, 06 Jul 2005 12:52:23 -0600, Jonathan Briggs <jbriggs@esoft.com> said: > On Tue, 2005-07-05 at 23:44 -0700, Hans Reiser wrote: >> Hubert Chan wrote: >>> And a question: is it feasible to store, for each >>> inode, its parent(s), instead of just the hard link count? >> Ooh, now that is an interesting old idea I haven't considered in 20 >> years.... makes fsck more robust too.... If you can store the parents, then finding cycles (relatively) quickly is pretty easy: before you try to make A the parent of B, walk up the parent pointers starting from A. If you ever reach B, you have a cycle. If not, you don't have a cycle. (Hmmm. Do I need a proof of correctness for this? It's just depth-first-search, which everybody learned in their first algorithms course.) Running time is (at most) just the sum of all (directed) path lengths from A to the root. Assuming people don't have too deep of a hierarchy, or too much branching with hard links (i.e. nodes with too many parents), which I think is a reasonable assumption, running time should be fairly quick. (People may have millions of files in a single directory, which is why doing the DFS in the forward direction is a bad idea. But people probably don't have millions of hard links to the same file, or hierarchies that are millions of levels deep.) And it only happens when a hard link is created -- probably not a very performance-dependent event (i.e. it's not the scheduling algorithm). I would imagine that some creative caching could be done to speed up the situation where you try to make a whole batch of hard links. (When you walk up the parent pointers starting from A, cache the inode numbers that you encounter. If you then try to make A the parent of C, check if C is one of the inode numbers in the cache. If yes, we have a cycle; if no, we don't have a cycle.) Extra disk storage is just about one extra word for most nodes (that only have one parent -- basically, one extra word per parent), if you can pack the data efficiently. (i.e. we don't want to waste a extra whole block per node.) Of course, since this requires extra storage on the part of the filesystem, the (kernel) VFS change would have to be something where the VFS asks the filesystem whether making A the parent of B will create a cycle; it's not something the VFS can do transparently for everybody. Filesystems that don't store this extra stuff just return "yes" if B is a directory (thus disallowing the link), and "no" if B is a file, and we get the same situation we have right now. It would require a disk format change (sort of) in Reiser4, but (as Hans mentioned) since we have plugins, shouldn't require a reformat (as I understand it). Just run a tool that builds the parent-pointer data; until you run that tool, the filesystem can just disallow hard links. After you run the tool, updating parent pointers is handled automatically by the filesystem. > Hey, sounds like the idea I proposed a couple months ago of storing > the path names in each file, instead of in directories. Only better, > since each path component isn't text but a link instead. > It still has the performance and locking problem of having to update > every child file when moving a directory tree to a new parent. On the > other hand, maybe the benefit is worth the cost. Every node should store the inode number(s) for its parent(s). Not the path name. So you don't need to do any updates, since when you move a tree, inode numbers don't change. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 19:51 ` Hubert Chan @ 2005-07-06 20:33 ` Jonathan Briggs 2005-07-06 20:53 ` David Masover 2005-07-06 20:57 ` Hubert Chan 2005-07-06 20:33 ` Horst von Brand 1 sibling, 2 replies; 620+ messages in thread From: Jonathan Briggs @ 2005-07-06 20:33 UTC (permalink / raw) To: Hubert Chan Cc: Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, ninja, vitaly [-- Attachment #1: Type: text/plain, Size: 833 bytes --] On Wed, 2005-07-06 at 15:51 -0400, Hubert Chan wrote: > On Wed, 06 Jul 2005 12:52:23 -0600, Jonathan Briggs <jbriggs@esoft.com> said: [snip] > > It still has the performance and locking problem of having to update > > every child file when moving a directory tree to a new parent. On the > > other hand, maybe the benefit is worth the cost. > > Every node should store the inode number(s) for its parent(s). Not the > path name. So you don't need to do any updates, since when you move a > tree, inode numbers don't change. You do need the updates if you change what inode is the parent. /a/b/c, /a/b/d, /a/b/e, /b mv /a/b /b Now you have to change the stored grand-parent inodes for /a/b/c, /a/b/d and /a/b/e so that they reference /b/b instead of /a/b. -- Jonathan Briggs <jbriggs@esoft.com> eSoft, Inc. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 20:33 ` Jonathan Briggs @ 2005-07-06 20:53 ` David Masover 2005-07-06 21:28 ` Jonathan Briggs 2005-07-06 20:57 ` Hubert Chan 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-06 20:53 UTC (permalink / raw) To: Jonathan Briggs Cc: Hubert Chan, Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, vitaly Jonathan Briggs wrote: > On Wed, 2005-07-06 at 15:51 -0400, Hubert Chan wrote: > >>On Wed, 06 Jul 2005 12:52:23 -0600, Jonathan Briggs <jbriggs@esoft.com> said: > > [snip] > >>>It still has the performance and locking problem of having to update >>>every child file when moving a directory tree to a new parent. On the >>>other hand, maybe the benefit is worth the cost. >> >>Every node should store the inode number(s) for its parent(s). Not the >>path name. So you don't need to do any updates, since when you move a >>tree, inode numbers don't change. > > > You do need the updates if you change what inode is the parent. > > /a/b/c, /a/b/d, /a/b/e, /b > mv /a/b /b > Now you have to change the stored grand-parent inodes for /a/b/c, /a/b/d > and /a/b/e so that they reference /b/b instead of /a/b. no, c, d, and e all would reference /a/b's *inode*, not *pathname*. Then /a/b would reference /a. So with the above mv, you just change the reference in /a/b (now /b/b) to point to /b as parent. Only way you would have to touch c, d, and e is if you did: mkdir /b/b mv /a/b/* /b/b/ but, since you're also essentially unlinking them from /a/b and linking them into /b, it's not that much more of a performance hit to update one pointer in the file. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 20:53 ` David Masover @ 2005-07-06 21:28 ` Jonathan Briggs 0 siblings, 0 replies; 620+ messages in thread From: Jonathan Briggs @ 2005-07-06 21:28 UTC (permalink / raw) To: David Masover; +Cc: Reiserfs mail-list [-- Attachment #1: Type: text/plain, Size: 721 bytes --] On Wed, 2005-07-06 at 15:53 -0500, David Masover wrote: > Jonathan Briggs wrote: [snip] > > /a/b/c, /a/b/d, /a/b/e, /b > > mv /a/b /b > > Now you have to change the stored grand-parent inodes for /a/b/c, /a/b/d > > and /a/b/e so that they reference /b/b instead of /a/b. > > no, c, d, and e all would reference /a/b's *inode*, not *pathname*. > Then /a/b would reference /a. So with the above mv, you just change the > reference in /a/b (now /b/b) to point to /b as parent. I misunderstood the reference to parent(s). It's a list of parent inodes, one parent per hard link, not a list of all (grand)*parents up to the root. But I see it now. -- Jonathan Briggs <jbriggs@esoft.com> eSoft, Inc. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 20:33 ` Jonathan Briggs 2005-07-06 20:53 ` David Masover @ 2005-07-06 20:57 ` Hubert Chan 1 sibling, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-07-06 20:57 UTC (permalink / raw) To: Jonathan Briggs Cc: Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, ninja, vitaly On Wed, 06 Jul 2005 14:33:18 -0600, Jonathan Briggs <jbriggs@esoft.com> said: > On Wed, 2005-07-06 at 15:51 -0400, Hubert Chan wrote: >> On Wed, 06 Jul 2005 12:52:23 -0600, Jonathan Briggs >> <jbriggs@esoft.com> said: > [snip] >>> It still has the performance and locking problem of having to >>> update every child file when moving a directory tree to a new >>> parent. On the other hand, maybe the benefit is worth the cost. >> Every node should store the inode number(s) for its parent(s). Not >> the path name. So you don't need to do any updates, since when you >> move a tree, inode numbers don't change. > You do need the updates if you change what inode is the parent. > /a/b/c, /a/b/d, /a/b/e, /b > mv /a/b /b > Now you have to change the stored grand-parent inodes for /a/b/c, > /a/b/d and /a/b/e so that they reference /b/b instead of /a/b. Don't store (great)*grand-parent pointers; only parent pointers. c points to b, b points to a, a points to root. c doesn't need to know about /a or root directly. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 19:51 ` Hubert Chan 2005-07-06 20:33 ` Jonathan Briggs @ 2005-07-06 20:33 ` Horst von Brand 2005-07-06 21:31 ` Hubert Chan 1 sibling, 1 reply; 620+ messages in thread From: Horst von Brand @ 2005-07-06 20:33 UTC (permalink / raw) To: Hubert Chan Cc: Jonathan Briggs, Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, ninja, vitaly Hubert Chan <hubert@uhoreg.ca> wrote: > On Wed, 06 Jul 2005 12:52:23 -0600, Jonathan Briggs <jbriggs@esoft.com> said: > > On Tue, 2005-07-05 at 23:44 -0700, Hans Reiser wrote: > >> Hubert Chan wrote: > >>> And a question: is it feasible to store, for each > >>> inode, its parent(s), instead of just the hard link count? > >> Ooh, now that is an interesting old idea I haven't considered in 20 > >> years.... makes fsck more robust too.... > If you can store the parents, then finding cycles (relatively) quickly > is pretty easy: before you try to make A the parent of B, walk up the > parent pointers starting from A. If you ever reach B, you have a cycle. > If not, you don't have a cycle. (Hmmm. Do I need a proof of > correctness for this? It's just depth-first-search, which everybody > learned in their first algorithms course.) Correct. And you need space for potentially a huge lot of up pointers for each file. And then there is the (very minor) problem is that meanwhile /nothing/ can touch the filesystem to do any change... How do you delete such an object? You will have to delete from each child down to the first object that has a pointer to it from the outside, if you don't want garbage left behind. And that means walking down to /each/ reachable object, then walking up from there to see if all its parents are in the DAG rooted at what you are going to delete. This means potentially walking through the whole filesystem (if you want to delete the root, or something near). You will run out of memory, and again, meanwhile no changes can be allowed. It is for a reason that Unix filesystems don't have up pointers for files, and just leaves (i.e., files) can have multiple hard links. And this /was/ explained in detail the last flamefest over this exact same point. I thought the ReiserFS people had learned from that... -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 20:33 ` Horst von Brand @ 2005-07-06 21:31 ` Hubert Chan 2005-07-07 2:33 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-07-06 21:31 UTC (permalink / raw) To: Horst von Brand Cc: Jonathan Briggs, Alexander G. M. Smith, ross.biro, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, ninja, vitaly On Wed, 06 Jul 2005 16:33:23 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: > Hubert Chan <hubert@uhoreg.ca> wrote: >> If you can store the parents, then finding cycles (relatively) >> quickly is pretty easy: before you try to make A the parent of B, >> walk up the parent pointers starting from A. If you ever reach B, >> you have a cycle. If not, you don't have a cycle. (Hmmm. Do I need >> a proof of correctness for this? It's just depth-first-search, which >> everybody learned in their first algorithms course.) > Correct. And you need space for potentially a huge lot of up pointers > for each file. People (that I know of) don't normally have a huge lot of hardlinks to a single file. > And then there is the (very minor) problem is that meanwhile /nothing/ > can touch the filesystem to do any change... If the DFS is quick, it shouldn't make much difference. Lock nodes as you walk up the tree, and people can change unlocked nodes without harming anything. All you need to do is make sure that no up pointers get added to nodes that you've already looked at. People can also still edit files without changing the tree (and hence not screwing up the DFS). > How do you delete such an object? You will have to delete from each > child down to the first object that has a pointer to it from the > outside, if you don't want garbage left behind. And that means walking > down to /each/ reachable object, then walking up from there to see if > all its parents are in the DAG rooted at what you are going to > delete. This means potentially walking through the whole filesystem > (if you want to delete the root, or something near). You will run out > of memory, and again, meanwhile no changes can be allowed. What does deleting have to do with up pointers? If you delete a directory that has refcount greater than 1, you decrease the refcount, remove the appropriate up pointer, and remove the inode entry from its parent. If you delete a directory that has refcount equal to 1, well, we're in the same situation we're in right now when you delete a directory; you need to "rm -r", which doesn't lock up the whole filesystem. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 21:31 ` Hubert Chan @ 2005-07-07 2:33 ` David Masover 2005-07-07 3:13 ` Jan Harkes 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-07 2:33 UTC (permalink / raw) To: Hubert Chan Cc: Horst von Brand, Jonathan Briggs, Alexander G. M. Smith, ross.biro, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, vitaly Hubert Chan wrote: > On Wed, 06 Jul 2005 16:33:23 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: > > >>Hubert Chan <hubert@uhoreg.ca> wrote: >> >>>If you can store the parents, then finding cycles (relatively) >>>quickly is pretty easy: before you try to make A the parent of B, >>>walk up the parent pointers starting from A. If you ever reach B, >>>you have a cycle. If not, you don't have a cycle. (Hmmm. Do I need >>>a proof of correctness for this? It's just depth-first-search, which >>>everybody learned in their first algorithms course.) > > >>Correct. And you need space for potentially a huge lot of up pointers >>for each file. > > > People (that I know of) don't normally have a huge lot of hardlinks to a > single file. And speaking of which, the only doomsday scenario (running out of RAM) that I can think of with this scheme is if we have a ton of hardlinks to the same file and we try to move one of them. But this scales linearly with the number of hardlinks, I think. Maybe not quite, but certainly not exponentially. The only other doomsday scenario is if we have a ludicrously deep tree. To make this work in real usage, not DOS testing, we really need both of those, and even then I'm not sure it can work. What's the maximum number of hardlinks supported to a single file? What's the maximum tree depth? Can these be limited to prevent actual DOS attempts? Now, if we were to ever actually *create* a cycle, then yes, we might end up traversing the whole tree to delete it, possibly running out of RAM. But we don't ever actually create cycles except if we were to have corruption, which could as easily create cycles in any other FS. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-07 2:33 ` David Masover @ 2005-07-07 3:13 ` Jan Harkes 0 siblings, 0 replies; 620+ messages in thread From: Jan Harkes @ 2005-07-07 3:13 UTC (permalink / raw) To: David Masover Cc: Hubert Chan, Horst von Brand, Jonathan Briggs, Alexander G. M. Smith, ross.biro, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, vitaly On Wed, Jul 06, 2005 at 09:33:13PM -0500, David Masover wrote: > And speaking of which, the only doomsday scenario (running out of RAM) > that I can think of with this scheme is if we have a ton of hardlinks to > the same file and we try to move one of them. But this scales linearly > with the number of hardlinks, I think. Maybe not quite, but certainly > not exponentially. > > The only other doomsday scenario is if we have a ludicrously deep tree. rename(a/b, c/b), if a and c are identical. End result, either you deadlock trying to lock the same object twice, or you end up removing b since the target of a rename is unlinked. The VFS uses dentries, there is one per hardlinked object, and they have a single parent only. So a/b and c/b are represented by the same inode, but they have completely different dentry-aliases associated with them. Similar things for removal, we can safely remove a file, but not a directory that still has children, if you have 'meta-files' hanging of a file, you'd have to get rid of the metadata objects first. If you want to use a file as a directory, it probably will need the same restrictions as a directory if you expect the dcache and VFS locking to work correctly. So that means, _no hardlinks to files_ (the file system could internally implement copy-on-write type links, or use a content addressable storage to deal with diskspace issues) and the file system probably has to d_unhash/destroy metadata objects before it can unlink the file object, etc. > To make this work in real usage, not DOS testing, we really need both of > those, and even then I'm not sure it can work. What's the maximum > number of hardlinks supported to a single file? I believe nlink is a 16-bit value, so that would be either 32K or 64K depending on signedness. > What's the maximum tree depth? Last time I checked, PATH_MAX is 4096, so that would be about 2048 single character directory names. > Can these be limited to prevent actual DOS attempts? Is that the 'nobody needs more than 64KB of memory' kind of DOS? Jan ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 18:52 ` Jonathan Briggs 2005-07-06 19:51 ` Hubert Chan @ 2005-07-07 6:42 ` Hans Reiser 2005-07-08 16:39 ` Hubert Chan 1 sibling, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-07-07 6:42 UTC (permalink / raw) To: Jonathan Briggs Cc: Hubert Chan, Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, ninja, vitaly Jonathan Briggs wrote: >On Tue, 2005-07-05 at 23:44 -0700, Hans Reiser wrote: > > >>Hubert Chan wrote: >> >> >>>And a question: is it feasible to store, for each inode, its parent(s), >>>instead of just the hard link count? >>> >>> >>> >>> >>Ooh, now that is an interesting old idea I haven't considered in 20 >>years.... makes fsck more robust too.... >> >> > >Hey, sounds like the idea I proposed a couple months ago of storing the >path names in each file, instead of in directories. Only better, since >each path component isn't text but a link instead. > >It still has the performance and locking problem of having to update >every child file when moving a directory tree to a new parent. On the >other hand, maybe the benefit is worth the cost. > > Oh no, don't store the whole path, store just the parent list. This will make fsck more robust in the event that the directory gets clobbered by hardware error. I don't think it affects the cost of detecting cycles though, I think it only makes fsck more robust. Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-07 6:42 ` Hans Reiser @ 2005-07-08 16:39 ` Hubert Chan 2005-07-08 16:45 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-07-08 16:39 UTC (permalink / raw) To: Hans Reiser Cc: Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, ninja, vitaly On Wed, 06 Jul 2005 23:42:50 -0700, Hans Reiser <reiser@namesys.com> said: > Oh no, don't store the whole path, store just the parent list. This > will make fsck more robust in the event that the directory gets > clobbered by hardware error. > I don't think it affects the cost of detecting cycles though, I think > it only makes fsck more robust. It doesn't affect the worst-case cost of detecting cycles; by necessity, any (static [1]) directed cycle detection algorithm must take O(n) time, n being the size of the tree (nodes + links). However, under "reasonable assumptions" (where the reasonableness of those assumptions may be questioned, but I think they're reasonable), I think that doing a depth-first-search through the parent links makes the algorithm not-too-terrible. Namely, the "reasonable assumptions" are that a node doesn't have "too many" parents, and the filesystem hierarchy is not "too deep". [1] BTW, I had also previously looked at online/dynamic algorithms, for those who are familiar with that area. The best known so far is still O(n) worst case, but much, much smaller in "most cases". -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-08 16:39 ` Hubert Chan @ 2005-07-08 16:45 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-07-08 16:45 UTC (permalink / raw) To: Hubert Chan Cc: Hans Reiser, Alexander G. M. Smith, ross.biro, vonbrand, mrmacman_g4, Valdis.Kletnieks, ltd, gmaxwell, jgarzik, hch, akpm, linux-kernel, reiserfs-list, zam, vs, ndiller, vitaly Hubert Chan wrote: > On Wed, 06 Jul 2005 23:42:50 -0700, Hans Reiser <reiser@namesys.com> said: > > >>Oh no, don't store the whole path, store just the parent list. This >>will make fsck more robust in the event that the directory gets >>clobbered by hardware error. > > >>I don't think it affects the cost of detecting cycles though, I think >>it only makes fsck more robust. > > > It doesn't affect the worst-case cost of detecting cycles; by necessity, > any (static [1]) directed cycle detection algorithm must take O(n) time, > n being the size of the tree (nodes + links). However, under > "reasonable assumptions" (where the reasonableness of those assumptions > may be questioned, but I think they're reasonable), I think that doing a > depth-first-search through the parent links makes the algorithm > not-too-terrible. Namely, the "reasonable assumptions" are that a node > doesn't have "too many" parents, and the filesystem hierarchy is not > "too deep". And, remember, there are other things affected by depth of the tree anyway. For that matter, most of the time, when you push a system past "reasonable assumptions", you hit performance issues, if not stability ones. Most everything but Reiser assumes that you won't have "too many" files in a directory, or "too many" small files in the FS as a whole. I think it's fair to assume people will keep things "reasonable", especially if we tell them what "reasonable" means. As in, "This is the kind of organization which Reiser4 handles really well, and this is the kind of organization which will absolutely kill your performance." ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 23:56 ` Hans Reiser 2005-07-06 0:50 ` Alexander G. M. Smith @ 2005-07-06 1:59 ` Neil Brown 2005-07-06 16:06 ` Nate Diller 1 sibling, 1 reply; 620+ messages in thread From: Neil Brown @ 2005-07-06 1:59 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller On Tuesday July 5, reiser@namesys.com wrote: > I got it slightly wrong. > > One can have hardlinks to a directory without cycles provided that one > does not have hardlinks from the children of that directory to any file > not a child of that directory. (Mountpoints currently implement that > restriction.) > > Question: can one implement that lesser restriction above with elegant > code? Is the greater restriction below easier to code? (If no to the > first and yes to the second is correct, then I can accept the greater > restriction described below.) <technical-content> I think the "lesser restriction above" can be implemented elegantly, but it would require major dcache surgery. Currently all the dentries of names in a directory are linked to the dentry of the directory. As you would have to let a directory have multiple dentries, it would be best to change that linkage so that the dentries of names in a directory were linked to the "inode" of that directory (of which there is still only one). Thus instead of just having a dentry tree with inodes attached at each point, you would have a dentry/inode tree with inodes a more integral part of the tree. (i.e. the path from the root down would be dentry -> inode -> dentry ->inode -> dentry etc). This would have major implications for the current code. The "greater restriction below" should be easy to code providing you were willing to have two sorts of directories: those which could be linked (ie. they sometimes look like files) and those which cannot (they never look like files). Then for each dentry, you remember the closest parent which is a can-be-linked directory an make sure a hard-link will never want to change the can-be-linked directory for the target. If you want to be more general, and have only one sort of directory that just behaves differently in different situations, then it would be much harder. You have to make sure both a/ that you never hard link a file that is under a hard-linked directory to somewhere outside of that hard-linked directory and b/ that you never hard link a directory that contains a file which is hard-linked to somewhere outside that directory. The first is probably quite manageable. The second is essentially the cycle-detection problem. </technical-content> <humour> SUN used to advertise: "The network is the computer" However I think we have all come to realise that the network is the network, and the computer is the computer. Now Hans wants to tells that "The directory is the file" but I don't find it any more convincing than SUN's message... </humour> <opinion> If you really want to change traditional Unix semantics, I would suggest you get rid of hard-links. They really are more trouble than they are worth, and discarding them makes this whole issue moot. </opinion> NeilBrown ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 1:59 ` Neil Brown @ 2005-07-06 16:06 ` Nate Diller 2005-07-06 18:13 ` Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: Nate Diller @ 2005-07-06 16:06 UTC (permalink / raw) To: Neil Brown Cc: Hans Reiser, David Masover, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller Neil Brown wrote: >On Tuesday July 5, reiser@namesys.com wrote: > > >>I got it slightly wrong. >> >>One can have hardlinks to a directory without cycles provided that one >>does not have hardlinks from the children of that directory to any file >>not a child of that directory. (Mountpoints currently implement that >>restriction.) >> >>Question: can one implement that lesser restriction above with elegant >>code? Is the greater restriction below easier to code? (If no to the >>first and yes to the second is correct, then I can accept the greater >>restriction described below.) >> >> > ><technical-content> >I think the "lesser restriction above" can be implemented elegantly, >but it would require major dcache surgery. > >Currently all the dentries of names in a directory are linked to the >dentry of the directory. As you would have to let a directory have >multiple dentries, it would be best to change that linkage so that the >dentries of names in a directory were linked to the "inode" of that >directory (of which there is still only one). >Thus instead of just having a dentry tree with inodes attached at each >point, you would have a dentry/inode tree with inodes a more integral >part of the tree. (i.e. the path from the root down would be dentry -> >inode -> dentry ->inode -> dentry etc). >This would have major implications for the current code. > >The "greater restriction below" should be easy to code providing you >were willing to have two sorts of directories: those which could be >linked (ie. they sometimes look like files) and those which cannot >(they never look like files). Then for each dentry, you remember the >closest parent which is a can-be-linked directory an make sure a >hard-link will never want to change the can-be-linked directory for >the target. > >If you want to be more general, and have only one sort of directory >that just behaves differently in different situations, then it would >be much harder. >You have to make sure both > a/ that you never hard link a file that is under a hard-linked > directory to somewhere outside of that hard-linked directory and > b/ that you never hard link a directory that contains a file which is > hard-linked to somewhere outside that directory. > >The first is probably quite manageable. The second is essentially the >cycle-detection problem. > ></technical-content> > ><humour> >SUN used to advertise: > "The network is the computer" >However I think we have all come to realise that the network is the >network, and the computer is the computer. > >Now Hans wants to tells that > "The directory is the file" > >but I don't find it any more convincing than SUN's message... > ></humour> > ><opinion> >If you really want to change traditional Unix semantics, I would >suggest you get rid of hard-links. They really are more trouble than >they are worth, and discarding them makes this whole issue moot. ></opinion> > > have you read Giampaolo's BeFS design book? he talks (ch 11) about this big argument they had at Be about how their API would let programs store references to files. the macintosh people wanted to have the program store an inode number, and the posix guys wanted it to store a path. in my mind, the only reason posix is right on this is because it allows hard links AND symlinks as a way of giving a choice in the matter, since hard links increment the reference count and softlinks don't. it seems much *more* important to be able to have this choice for file attributes than for the traditional posix namespace. as an example, if a program were to store some things it needs access to in its executable's attributes, it should have the option of keeping a hard reference to something, so that it can't be deleted out from underneath. this enables sane sharing of resources without ownership tracking problems (see windows DLL hell for details). the attribute space should be indistinguishable from the rest of the namespace, and should be able to link (soft or hard) anywhere in the FS. anything less is too much work for too little reward. NATE >NeilBrown > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 16:06 ` Nate Diller @ 2005-07-06 18:13 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-07-06 18:13 UTC (permalink / raw) To: Nate Diller Cc: Neil Brown, David Masover, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List, Alexander Zarochentcev, vs, Nate Diller Nate Diller wrote: > > > as an example, if a program were to store some things it needs access > to in its executable's attributes, it should have the option of > keeping a hard reference to something, so that it can't be deleted out > from underneath. this enables sane sharing of resources without > ownership tracking problems (see windows DLL hell for details). the > attribute space should be indistinguishable from the rest of the > namespace, and should be able to link (soft or hard) anywhere in the > FS. anything less is too much work for too little reward. > You already have a problem with hardlinks not crossing mount points, but I understand your point. If we can write code for solving the cycle problem cleanly, it would be best. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 23:00 ` David Masover 2005-07-05 23:47 ` Hans Reiser 2005-07-05 23:56 ` Hans Reiser @ 2005-07-06 2:55 ` Horst von Brand 2005-07-06 13:18 ` David Masover 2 siblings, 1 reply; 620+ messages in thread From: Horst von Brand @ 2005-07-06 2:55 UTC (permalink / raw) To: David Masover Cc: Hans Reiser, Hubert Chan, Ross Biro, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover <ninja@slaphack.com> wrote: [...] > Just don't allow user-created hardlinks inside either metafs (/meta) or > the magical meta directory inside files. And what is it useful for, after its advantage was that it was /exactly/ like regular files &c, and now it is severely crippled? -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 2:55 ` Horst von Brand @ 2005-07-06 13:18 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-07-06 13:18 UTC (permalink / raw) To: Horst von Brand Cc: Hans Reiser, Hubert Chan, Ross Biro, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Horst von Brand wrote: > David Masover <ninja@slaphack.com> wrote: > > [...] > > >>Just don't allow user-created hardlinks inside either metafs (/meta) or >>the magical meta directory inside files. > > > And what is it useful for, after its advantage was that it was /exactly/ > like regular files &c, and now it is severely crippled? The advantage was never that meta files look exactly like regular files, but that they look enough like regular files that you can use existing tools on them. Of course, there are some times when you want meta files to look exactly like regular files, as in (I hate to bring it up again, but...) zip files. So, a zip file (/meta/vfs/some/zip/file/.../contents/) would allow hardlinks between files inside the zipfile, but not outside of it. This would be like creating a separate mountpoint for the zipfile's contents, and doing "mount --bind" when a hardlink to the zipfile is created. I'm not sure if we actually have to pretend it's a mountpoint, or if we can just take the checking that mountpoints already do and generalize it. Can you think of a way in which hardlinks are useful in /meta, in a *general* way, instead of within a specific directory controlled by a specific plugin? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 21:34 ` Ross Biro 2005-06-29 23:29 ` Hubert Chan @ 2005-06-30 3:04 ` Hans Reiser 2005-06-30 4:33 ` Hubert Chan 2005-07-05 15:46 ` Martin Waitz 1 sibling, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-30 3:04 UTC (permalink / raw) To: Ross Biro Cc: Hubert Chan, Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Ross Biro wrote: >On 6/29/05, Hubert Chan <hubert@uhoreg.ca> wrote: > > >>On Wed, 29 Jun 2005 01:09:05 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: >> >> >> >>>Hubert Chan <hubert@uhoreg.ca> wrote: [...] >>>And doing "tar cf /dev/tape /usr/games/tetris" gives you a nice tangle >>>of undecipherable junk. >>> >>> > >I'm confused. Can someone on one of these lists enlighten me? > >How is directories as files logically any different than putting all >data into .data files and making all files directories (yes you would >need some sort of special handling for files that were really called >.data). > Add to this that you make .data the default if the file within the directory is not specified, and define a stanadard set of names for metafiles, and you've got the essential idea, and any differences are details. > Then it's just a matter of deciding what happens when you >call open and stat on one of these files? > >For backwards compatibility, current existing system calls have to >treat these things as directories. Perhaps an exception could be made >for exec. > >But we could have a whole new set of system calls that treat things as >magic, and if files as directories is as cool as many people think, >apps will start using the new api. If not, they won't and the new api >can be deprecated. > > Ross > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 3:04 ` Hans Reiser @ 2005-06-30 4:33 ` Hubert Chan 2005-06-30 4:48 ` Hans Reiser 2005-06-30 6:29 ` David Weinehall 2005-07-05 15:46 ` Martin Waitz 1 sibling, 2 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-30 4:33 UTC (permalink / raw) To: Hans Reiser Cc: Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Wed, 29 Jun 2005 20:04:58 -0700, Hans Reiser <reiser@namesys.com> said: > Ross Biro wrote: >> How is directories as files logically any different than putting all >> data into .data files and making all files directories (yes you would >> need some sort of special handling for files that were really called >> .data). > Add to this that you make .data the default if the file within the > directory is not specified, It's sort of like the way web servers handle index.html, for those who think it's a stupid idea. (Of course, some people may still think it's a stupid idea... ;-) ) > and define a stanadard set of names for metafiles, and you've got the > essential idea, and any differences are details. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 4:33 ` Hubert Chan @ 2005-06-30 4:48 ` Hans Reiser 2005-06-30 6:29 ` David Weinehall 1 sibling, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-30 4:48 UTC (permalink / raw) To: Hubert Chan Cc: Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hubert Chan wrote: >On Wed, 29 Jun 2005 20:04:58 -0700, Hans Reiser <reiser@namesys.com> said: > > > >>Ross Biro wrote: >> >> >>>How is directories as files logically any different than putting all >>>data into .data files and making all files directories (yes you would >>>need some sort of special handling for files that were really called >>>.data). >>> >>> > > > >>Add to this that you make .data the default if the file within the >>directory is not specified, >> >> > >It's sort of like the way web servers handle index.html, for those who >think it's a stupid idea. (Of course, some people may still think it's >a stupid idea... ;-) ) > > > Good analogy. >>and define a stanadard set of names for metafiles, and you've got the >>essential idea, and any differences are details. >> >> > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 4:33 ` Hubert Chan 2005-06-30 4:48 ` Hans Reiser @ 2005-06-30 6:29 ` David Weinehall 2005-06-30 15:57 ` Hubert Chan 1 sibling, 1 reply; 620+ messages in thread From: David Weinehall @ 2005-06-30 6:29 UTC (permalink / raw) To: Hubert Chan Cc: Hans Reiser, Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Thu, Jun 30, 2005 at 12:33:10AM -0400, Hubert Chan wrote: > On Wed, 29 Jun 2005 20:04:58 -0700, Hans Reiser <reiser@namesys.com> said: > > > Ross Biro wrote: > >> How is directories as files logically any different than putting all > >> data into .data files and making all files directories (yes you would > >> need some sort of special handling for files that were really called > >> .data). > > > Add to this that you make .data the default if the file within the > > directory is not specified, > > It's sort of like the way web servers handle index.html, for those who > think it's a stupid idea. (Of course, some people may still think it's > a stupid idea... ;-) ) And guess what? That's handled on the web server level (userland), not by the file system. So different web servers can handle it differently (think index.html.sv, index.html.zh, index.php, etc). Amazing isn't it? [snip] Regards: David Weinehall -- /) David Weinehall <tao@acc.umu.se> /) Northern lights wander (\ // Maintainer of the v2.0 kernel // Dance across the winter sky // \) http://www.acc.umu.se/~tao/ (/ Full colour fire (/ ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 6:29 ` David Weinehall @ 2005-06-30 15:57 ` Hubert Chan 0 siblings, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-30 15:57 UTC (permalink / raw) To: Hans Reiser Cc: Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Thu, 30 Jun 2005 08:29:56 +0200, David Weinehall <tao@acc.umu.se> said: > On Thu, Jun 30, 2005 at 12:33:10AM -0400, Hubert Chan wrote: >> It's sort of like the way web servers handle index.html, for those >> who think it's a stupid idea. (Of course, some people may still >> think it's a stupid idea... ;-) ) > And guess what? That's handled on the web server level (userland), > not by the file system. So different web servers can handle it > differently (think index.html.sv, index.html.zh, index.php, etc). >From the web *browser*'s point of view, it is handled by the "filesystem" (which is provided by the various servers). The browser doesn't care how or where the data is stored. It just requests a file, and gets some data back. So the browser doesn't have to check for http://www.example.com/, get a failure (trying to read a directory), check for http://www.example.com/index.html, etc. In this way, the web server controls (which I think takes the place of the filesystem in this case) what gets shown (index.html.sv, etc.), instead of the leaving it up to the browser. In the same way, you don't want every single user program to have to guess whether it should look at .data, or .contents, or whatever. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-30 15:57 ` Hubert Chan 0 siblings, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-30 15:57 UTC (permalink / raw) To: Hans Reiser Cc: Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Thu, 30 Jun 2005 08:29:56 +0200, David Weinehall <tao@acc.umu.se> said: > On Thu, Jun 30, 2005 at 12:33:10AM -0400, Hubert Chan wrote: >> It's sort of like the way web servers handle index.html, for those >> who think it's a stupid idea. (Of course, some people may still >> think it's a stupid idea... ;-) ) > And guess what? That's handled on the web server level (userland), > not by the file system. So different web servers can handle it > differently (think index.html.sv, index.html.zh, index.php, etc). From the web *browser*'s point of view, it is handled by the "filesystem" (which is provided by the various servers). The browser doesn't care how or where the data is stored. It just requests a file, and gets some data back. So the browser doesn't have to check for http://www.example.com/, get a failure (trying to read a directory), check for http://www.example.com/index.html, etc. In this way, the web server controls (which I think takes the place of the filesystem in this case) what gets shown (index.html.sv, etc.), instead of the leaving it up to the browser. In the same way, you don't want every single user program to have to guess whether it should look at .data, or .contents, or whatever. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 15:57 ` Hubert Chan (?) @ 2005-06-30 17:10 ` David Weinehall 2005-06-30 18:47 ` Horst von Brand 2005-06-30 19:00 ` Hubert Chan -1 siblings, 2 replies; 620+ messages in thread From: David Weinehall @ 2005-06-30 17:10 UTC (permalink / raw) To: Hubert Chan Cc: Hans Reiser, Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Thu, Jun 30, 2005 at 11:57:27AM -0400, Hubert Chan wrote: > On Thu, 30 Jun 2005 08:29:56 +0200, David Weinehall <tao@acc.umu.se> said: > > > On Thu, Jun 30, 2005 at 12:33:10AM -0400, Hubert Chan wrote: > >> It's sort of like the way web servers handle index.html, for those > >> who think it's a stupid idea. (Of course, some people may still > >> think it's a stupid idea... ;-) ) > > > And guess what? That's handled on the web server level (userland), > > not by the file system. So different web servers can handle it > > differently (think index.html.sv, index.html.zh, index.php, etc). > > >From the web *browser*'s point of view, it is handled by the > "filesystem" (which is provided by the various servers). The browser > doesn't care how or where the data is stored. It just requests a file, > and gets some data back. So the browser doesn't have to check for > http://www.example.com/, get a failure (trying to read a directory), > check for http://www.example.com/index.html, etc. In this way, the web > server controls (which I think takes the place of the filesystem in this > case) what gets shown (index.html.sv, etc.), instead of the leaving it > up to the browser. > > In the same way, you don't want every single user program to have to > guess whether it should look at .data, or .contents, or whatever. For the analogy to be complete: User has a file browser (say Nautilus) The file browser sees the userland VFS (say a unified VFS between GNOME and KDE) The VFS sees the real file system This way the userland VFS can be ported to almost any platform. When toying around on the desktop, an abstraction of what a file is works fine with me. When doing serious work (programming, tar:ing up stuff, etc) I want to be bloody sure that I see the files in the same way always. I don't want surprises such as files suddenly behaving as directories or vice versa. Regards: David -- /) David Weinehall <tao@acc.umu.se> /) Northern lights wander (\ // Maintainer of the v2.0 kernel // Dance across the winter sky // \) http://www.acc.umu.se/~tao/ (/ Full colour fire (/ ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 17:10 ` David Weinehall @ 2005-06-30 18:47 ` Horst von Brand 2005-06-30 20:08 ` Kevin Bowen 2005-06-30 20:37 ` Zan Lynx 2005-06-30 19:00 ` Hubert Chan 1 sibling, 2 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-30 18:47 UTC (permalink / raw) To: Hubert Chan, Hans Reiser, Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Weinehall <tao@acc.umu.se> wrote: > On Thu, Jun 30, 2005 at 11:57:27AM -0400, Hubert Chan wrote: > > On Thu, 30 Jun 2005 08:29:56 +0200, David Weinehall <tao@acc.umu.se> said: [...] > > >From the web *browser*'s point of view, it is handled by the > > "filesystem" (which is provided by the various servers). The browser > > doesn't care how or where the data is stored. It just requests a file, > > and gets some data back. So the browser doesn't have to check for > > http://www.example.com/, get a failure (trying to read a directory), > > check for http://www.example.com/index.html, etc. In this way, the web > > server controls (which I think takes the place of the filesystem in this > > case) what gets shown (index.html.sv, etc.), instead of the leaving it > > up to the browser. > > In the same way, you don't want every single user program to have to > > guess whether it should look at .data, or .contents, or whatever. > > For the analogy to be complete: > User has a file browser (say Nautilus) > The file browser sees the userland VFS (say a unified VFS between GNOME > and KDE) > The VFS sees the real file system > This way the userland VFS can be ported to almost any platform. That is /very/ nice to have. > When toying around on the desktop, an abstraction of what a file > is works fine with me. When doing serious work (programming, > tar:ing up stuff, etc) I want to be bloody sure that I see the > files in the same way always. I don't want surprises such as > files suddenly behaving as directories or vice versa. Let me moderate that a bit: I'd be happy to have (some) files behaving strangely, /if in exchange I get something very worthwhile/. I.e., Unix filesystem sockets, named pipes, virtual filesystems all behave in weird ways, but this downside is more than compensated by huge advantages (even being able to do things that would otherwise be impossible). But all I see is that file-is-directory advocates explain over and over how they are bending over backards to get the whole mess working exactly like true directories (nothing more, in the end quite a bit less). The uses I've seen discussed don't really need files-as-directories (you get most of the advantages by just tar(1)-ing up the contents, or packing them in some other way; OpenOffice /has/ structured files, XML inside zipped files, Java also uses zip files for its structuring needs, etc), or are ideas that might be nice to have on exclusively one-user, isolated machines, like "keep /my/ annotations/icon/application name/whatever with the file's data", but that break down in multiuser (even serially, one user after the other) systems or when files are shared (via network, or simply by sending by mail or by copying from one machine to the other). In my rather ample experience, that situation is rare today, rapidly dwindling in the arena where Linux is mostly used. So this particular case of strange semantics for files has no upsides I can see, only downsides. The downsides have been discussed to death, and AFAICS there are fundamental problems with the idea that can't be fixed or sanely worked around. So why bother? -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 18:47 ` Horst von Brand @ 2005-06-30 20:08 ` Kevin Bowen 2005-07-01 3:28 ` Horst von Brand 2005-06-30 20:37 ` Zan Lynx 1 sibling, 1 reply; 620+ messages in thread From: Kevin Bowen @ 2005-06-30 20:08 UTC (permalink / raw) To: Horst von Brand Cc: Hubert Chan, Hans Reiser, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List > might be nice to have on exclusively one-user, isolated machines, like > "keep /my/ annotations/icon/application name/whatever with the file's > data", but that break down in multiuser (even serially, one user after the If this is really the core of your (and the rest of the reiser- obstructionist crowd's) objection to the file-as-directory concept, then you just haven't thought it through thoroughly enough. Ignore for the moment the case of system-wide or network-wide shared data, and think about it just limited to a user's home directory, and the storage and organization of actual *data* (as opposed to system files). The desire amongst users for ubiquitous metadata is very real - the current wave of "desktop search" products and technologies demonstrates this - but search is really only the lowest-hanging fruit of this new way of looking at data. Application-layer solutions like Beagle, Google Desktop Search et al allow for querying on metadata, but actually *acting* on the results of those queries requires that they be exposed via first class primitives which can be manipulated with arbitrary tools, not via some proprietary userland api which only one tool ever actually implements. As to the case of system-wide shared files, there is already a mechanism to prevent users from inappropriately annotating files that don't belong to them: file permissions. If you're sysadmining a multiuser reiser4 box, and your users are able to modify the metadata of files they don't own, then you go to sysadmin purgatory. > other way; OpenOffice /has/ structured files, XML inside zipped files, Java > also uses zip files for its structuring needs, etc), or are ideas that And as a Java developer, I can tell you that the wide consensus is that this solution is half-assed and insufficient for both developers and users needs. In fact, I believe there is currently a JSR in progress to develop a more sophisticated Java packaging model. -- kevin@ucsd.edu ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 20:08 ` Kevin Bowen @ 2005-07-01 3:28 ` Horst von Brand 2005-07-01 6:56 ` David Masover 2005-07-01 7:41 ` Chet Hosey 0 siblings, 2 replies; 620+ messages in thread From: Horst von Brand @ 2005-07-01 3:28 UTC (permalink / raw) To: Kevin Bowen Cc: Horst von Brand, Hubert Chan, Hans Reiser, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Kevin Bowen <kevin@ucsd.edu> wrote: > > might be nice to have on exclusively one-user, isolated machines, like > > "keep /my/ annotations/icon/application name/whatever with the file's > > data", but that break down in multiuser (even serially, one user after the > If this is really the core of your (and the rest of the reiser- > obstructionist crowd's) objection to the file-as-directory concept, then > you just haven't thought it through thoroughly enough. Ignore for the > moment the case of system-wide or network-wide shared data, I.e., something like 90% of the use of Linux here. OK. > and think > about it just limited to a user's home directory, and the storage and > organization of actual *data* (as opposed to system files). Who is to say what is "data" and what is "system files"? And if you are limiting yourself to /user/ data, why not have the /user/ decide how they want to organize it, and give them the tools? > The desire > amongst users for ubiquitous metadata is very real - the current wave of > "desktop search" products and technologies demonstrates this - Just like each previous claim "this /must/ be the next cool technology!", also called later the "dotcom crash"... > but > search is really only the lowest-hanging fruit of this new way of > looking at data. Application-layer solutions like Beagle, That works without screwing up the whole system, AFAIU. > Google Desktop > Search et al allow for querying on metadata, but actually *acting* on > the results of those queries requires that they be exposed via first > class primitives which can be manipulated with arbitrary tools, not via > some proprietary userland api which only one tool ever actually > implements. Could you please explain in plain english? The only part I get out is "propietary API", and I don't see anybody advocating such here. > As to the case of system-wide shared files, there is already a mechanism > to prevent users from inappropriately annotating files that don't belong > to them: file permissions. And who says that a normal user isn't allowed to annotate each and every file with its purpose or something else? I can very well imagine a system in which users (say students in a Linux class) want to do so... on a shared machine. Or users having a shared MP3 or photograph or ... collection, with individual notes on each. Or even developers wanting to annotate source code files with their comments, but leave them read-only (or have them under SCM). > If you're sysadmining a multiuser reiser4 > box, and your users are able to modify the metadata of files they don't > own, then you go to sysadmin purgatory. Bingo! And thus goes much of the supposed advantage of this nonsense. [I see that /opponents/ are accused here of lack of imagination, while I see that the /proponents/ lack imagination... or perhaps just real-world experience.] > > other way; OpenOffice /has/ structured files, XML inside zipped files, > > Java also uses zip files for its structuring needs, etc), or are ideas > > that > And as a Java developer, I can tell you that the wide consensus is that > this solution is half-assed and insufficient for both developers and > users needs. In fact, I believe there is currently a JSR in progress to > develop a more sophisticated Java packaging model. Presumably based on ReiserFS 4, which then has to be ported to whatever platform you want to run Java on ASAP? Great for you! Wait a bit, and you'll get what you want then, even across the board! -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 3:28 ` Horst von Brand @ 2005-07-01 6:56 ` David Masover 2005-07-01 20:26 ` Kevin Bowen 2005-07-01 7:41 ` Chet Hosey 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-07-01 6:56 UTC (permalink / raw) To: Horst von Brand Cc: Kevin Bowen, Hubert Chan, Hans Reiser, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Horst von Brand wrote: > Kevin Bowen <kevin@ucsd.edu> wrote: [...] >> The desire >>amongst users for ubiquitous metadata is very real - the current wave of >>"desktop search" products and technologies demonstrates this - > > > Just like each previous claim "this /must/ be the next cool technology!", > also called later the "dotcom crash"... Have you used this technology? For that matter, some of these claims have been true. The Internet, despite the dotcom crash, is now a requirement -- few people would use an OS with no networking support on a desktop PC. Same goes for GUIs. Both of these technologies had to have at least some amount of kernel support, both were once weird and experimental, and both are now in demand by users and likely to be for some time. If we're right, and searching is the next best thing, then it would be a good idea to get started now. If we're wrong, what's the harm? (Assuming /meta, of course. I accept that the issues with file-as-directory may never be solved to the point where we can do it at the kernel's FS/VFS level.) >> Google Desktop >>Search et al allow for querying on metadata, but actually *acting* on >>the results of those queries requires that they be exposed via first >>class primitives which can be manipulated with arbitrary tools, not via >>some proprietary userland api which only one tool ever actually >>implements. > > > Could you please explain in plain english? The only part I get out is > "propietary API", and I don't see anybody advocating such here. I don't understand it much, but I think the point being made is that while desktop searches may be able to search metadata (not sure how they do this -- their own plugins or just full-text searching?), they cannot write to said metadata. To change metadata on a file, one currently needs to use a proprietary userland api which is only implemented by one tool. So, for instance, if I want to grab all mp3s with Artist "Paul Oakenfold" and change the genre to "techno" (can you do that?), I can use Beagle's search tool to find all mp3s by Oakenfold, but to change the genre, I have to use some separate tool to manipulate id3 tags, which may or may not exist in an easily scriptable form. I know there's a decent tool for id3 tags, but there's so many kinds of metadata and so many tools out there that the user can't possibly memorize them all. This is why I use the zipfile example, despite its flaws. Now, the "arbitrary tools" bit is about how one could do this kind of thing with file-as-dir or /meta. With file-as-dir, it might be as simple as: $ for i in `magical-find-tool artist='Paul Oakenfold'`; do echo 'techno' > "$i/.../genre"; done Or even $ echo 'techno' > /.../search/artist='Paul Oakenfold'/*/.../genre It doesn't get too much more complicated for /meta, which has a nice side effect of not killing things like tar and generally not messing with POSIX. One could also use vim, emacs, or whatever else to edit this metadata, instead of being limited to one program. This is especially nice considering you might have to look up which one program to use. >>As to the case of system-wide shared files, there is already a mechanism >>to prevent users from inappropriately annotating files that don't belong >>to them: file permissions. > > > And who says that a normal user isn't allowed to annotate each and every > file with its purpose or something else? I can very well imagine a system > in which users (say students in a Linux class) want to do so... on a shared > machine. Or users having a shared MP3 or photograph or ... collection, with > individual notes on each. Or even developers wanting to annotate source > code files with their comments, but leave them read-only (or have them > under SCM). File-as-dir is actually pretty nice for this, I think. Since the annotations appear as files, if the app supports it, the users can each store individual annotations, world-readable, but only user-writable. With something like Gnome-VFS, I doubt there's even a mechanism whereby such annotations can be shared to begin with. But this is just a thought experiment atm, don't pick it apart too brutally... ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 6:56 ` David Masover @ 2005-07-01 20:26 ` Kevin Bowen 2005-07-02 2:12 ` Horst von Brand 0 siblings, 1 reply; 620+ messages in thread From: Kevin Bowen @ 2005-07-01 20:26 UTC (permalink / raw) To: Horst von Brand, Hubert Chan, Hans Reiser, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List > > Could you please explain in plain english? The only part I get out is > > "propietary API", and I don't see anybody advocating such here. Proprietary was a poor choice of words on my part. > I don't understand it much, but I think the point being made is that ... > So, for instance, if I want to grab all mp3s with Artist "Paul > Oakenfold" and change the genre to "techno" (can you do that?), I can > use Beagle's search tool to find all mp3s by Oakenfold, but to change > the genre, I have to use some separate tool to manipulate id3 tags, Yes, this is basically what I was getting at, although I was thinking more in terms of the reiser5/6/whatever set theoretic semantics, which, from my point of view at least, reiser4 is simply the first step towards building the enabling infrastructure of. But the fact that reiser4 semantics + trivial shell scripting enables, as illustrated by David, the rough equivalent of set-theoretic semantics, demonstrates how reiser4 is in fact a step in this direction. > > moment the case of system-wide or network-wide shared data, > I.e., something like 90% of the use of Linux here. OK. 90% of *what* exactly? 90% of what machines deal with, or 90% of what humans interact with? > > users needs. In fact, I believe there is currently a JSR in > > progress to develop a more sophisticated Java packaging model. > > Presumably based on ReiserFS 4, which then has to be ported to > whatever platform you want to run Java on ASAP? Great for you! Wait a > bit, and you'll get what you want then, even across the board! No, obviously that's not what I was saying. But the need for these kinds of domain-specific packaging/metadata formats, each requiring their own tools to work with, would be alleviated if there were simply a more powerful filesystem semantic. Clearly forcing reiser on the world is a non-starter, but try extending your imagination a little bit to a future in which there's a 'new POSIX' specifying a set-theoretic filesystem model. So-called 'database-filesystems' ARE coming, whether from Microsoft, Apple, or us. Who gets there first will determine who gets to make the rules. -- kevin@ucsd.edu ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 20:26 ` Kevin Bowen @ 2005-07-02 2:12 ` Horst von Brand 2005-07-04 19:05 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Horst von Brand @ 2005-07-02 2:12 UTC (permalink / raw) To: Kevin Bowen Cc: Horst von Brand, Hubert Chan, Hans Reiser, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Kevin Bowen <kevin@ucsd.edu> wrote: [...] > > So, for instance, if I want to grab all mp3s with Artist "Paul > > Oakenfold" and change the genre to "techno" (can you do that?), I can > > use Beagle's search tool to find all mp3s by Oakenfold, but to change > > the genre, I have to use some separate tool to manipulate id3 tags, > Yes, this is basically what I was getting at, although I was thinking > more in terms of the reiser5/6/whatever set theoretic semantics, which, > from my point of view at least, reiser4 is simply the first step towards > building the enabling infrastructure of. But the fact that reiser4 > semantics + trivial shell scripting enables, as illustrated by David, > the rough equivalent of set-theoretic semantics, demonstrates how > reiser4 is in fact a step in this direction. I don't see any "trivial shell scripting", just need for a plethora of magic extract-this-or-that-metadata programs. Which can very well work exactly the same on independent files, no need to shove junk /into/ the indexed files. > > > moment the case of system-wide or network-wide shared data, > > I.e., something like 90% of the use of Linux here. OK. > 90% of *what* exactly? 90% of what machines deal with, or 90% of what > humans interact with? Both. Most use here is in computer labs, where most is shared via NFS (readonly), plus user data mounted read-write. > > > users needs. In fact, I believe there is currently a JSR in > > > progress to develop a more sophisticated Java packaging model. > > Presumably based on ReiserFS 4, which then has to be ported to > > whatever platform you want to run Java on ASAP? Great for you! Wait a > > bit, and you'll get what you want then, even across the board! > No, obviously that's not what I was saying. But the need for these kinds > of domain-specific packaging/metadata formats, each requiring their own > tools to work with, would be alleviated if there were simply a more > powerful filesystem semantic. *Please explain HOW*!! The domain-specific formats /will/ stay (if nothing else, because the /domains/ have /specific/ needs), the special tools to work with them /will/ have to be written exactly the same (because of the above). All for use on /one/ non-standard filesystem. Plus exactly the same stuff to work on sane filesystems. The kernel is *not* a magic piece of software that solves the problem of world hunger if you just can figure out what strange extension to force onto Linus' kernel version. > Clearly forcing reiser on the world is a > non-starter, but try extending your imagination a little bit to a future > in which there's a 'new POSIX' specifying a set-theoretic filesystem > model. Sorry, I just don't see any need of shoving Oracle into the kernel. It leads a quite confortable life in userland. > So-called 'database-filesystems' ARE coming, whether from > Microsoft, Apple, or us. IBM did it, long ago (AS/400 and OS/400 ring a bell?). And is now slowly moving away from it (and other structured filesystems), AFAICS, towards (guess what!) POSIX and Linux... Again, Linux is what it is today in large part because "We have to get $FEATURE, because if not, $COMPETITOR will win on us!" arguments have no traction. > Who gets there first will determine who gets to > make the rules. So what? Let /them/ make the mistakes and pay for them, and learn how to do it /right/, even if later! -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-02 2:12 ` Horst von Brand @ 2005-07-04 19:05 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-07-04 19:05 UTC (permalink / raw) To: Horst von Brand Cc: Kevin Bowen, Hubert Chan, Hans Reiser, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Horst von Brand wrote: > Kevin Bowen <kevin@ucsd.edu> wrote: > > [...] > > >>>So, for instance, if I want to grab all mp3s with Artist "Paul >>>Oakenfold" and change the genre to "techno" (can you do that?), I can >>>use Beagle's search tool to find all mp3s by Oakenfold, but to change >>>the genre, I have to use some separate tool to manipulate id3 tags, > > >>Yes, this is basically what I was getting at, although I was thinking >>more in terms of the reiser5/6/whatever set theoretic semantics, which, >>from my point of view at least, reiser4 is simply the first step towards >>building the enabling infrastructure of. But the fact that reiser4 >>semantics + trivial shell scripting enables, as illustrated by David, >>the rough equivalent of set-theoretic semantics, demonstrates how >>reiser4 is in fact a step in this direction. > > > I don't see any "trivial shell scripting", just need for a plethora of > magic extract-this-or-that-metadata programs. Which can very well work > exactly the same on independent files, no need to shove junk /into/ the > indexed files. > > >>>>moment the case of system-wide or network-wide shared data, >>> >>>I.e., something like 90% of the use of Linux here. OK. > > >>>>users needs. In fact, I believe there is currently a JSR in >>>>progress to develop a more sophisticated Java packaging model. > > >>>Presumably based on ReiserFS 4, which then has to be ported to >>>whatever platform you want to run Java on ASAP? Great for you! Wait a >>>bit, and you'll get what you want then, even across the board! > > >>No, obviously that's not what I was saying. But the need for these kinds >>of domain-specific packaging/metadata formats, each requiring their own >>tools to work with, would be alleviated if there were simply a more >>powerful filesystem semantic. > > > *Please explain HOW*!! The domain-specific formats /will/ stay (if nothing > else, because the /domains/ have /specific/ needs), the special tools to > work with them /will/ have to be written exactly the same (because of the > above). All for use on /one/ non-standard filesystem. One filesystem that exists right now. /proc, as people have made clear, was originally implemented on *one* "non-standard" Unix. Others then copied it. I see no reason why if /meta is a good idea, and programs start using it, that other FSes won't implement it. > Plus exactly the same > stuff to work on sane filesystems. "sane" -- I assume you mean "traditional". Well, yeah. So? How big a program do you need to create a new interface to an existing system? Consider: BitTorrent has at least three interfaces that I can remember at the moment. btdownloadgui.py, btdownloadcurses.py, btdownloadheadless.py. Are you saying that btdownloadgui.py was so much work if you start from btdownloadcurses.py that there should be no GTK library, because it takes so much work to convert existing curses apps to GTK? Remember, even if I have GTK installed, I can still get to my curses apps, and they may not even know that GTK exists. Even if I have /meta enabled, I can still use my POSIX apps, and they may not even know that /meta exists. >> So-called 'database-filesystems' ARE coming, whether from >>Microsoft, Apple, or us. > > > IBM did it, long ago (AS/400 and OS/400 ring a bell?). And is now slowly > moving away from it (and other structured filesystems), AFAICS, towards > (guess what!) POSIX and Linux... Linux over POSIX, I think. Because Linux is big and open-source, not beacuse POSIX is so great. > Again, Linux is what it is today in large part because "We have to get > $FEATURE, because if not, $COMPETITOR will win on us!" arguments have no > traction. I agree with that, at least. But Linux is also what it is today in large part because "I don't need $FEATURE, so it should be kept out" isn't a particularly good argument either. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 3:28 ` Horst von Brand 2005-07-01 6:56 ` David Masover @ 2005-07-01 7:41 ` Chet Hosey 2005-07-05 20:55 ` Hubert Chan 1 sibling, 1 reply; 620+ messages in thread From: Chet Hosey @ 2005-07-01 7:41 UTC (permalink / raw) To: Horst von Brand Cc: Kevin Bowen, Hubert Chan, Hans Reiser, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Horst von Brand wrote: >Kevin Bowen <kevin@ucsd.edu> wrote: > > >> and think >>about it just limited to a user's home directory, and the storage and >>organization of actual *data* (as opposed to system files). >> >> > >Who is to say what is "data" and what is "system files"? And if you are >limiting yourself to /user/ data, why not have the /user/ decide how they >want to organize it, and give them the tools? > > That's exactly what the idea of exporting the metadata as files within a file-as-dir is intended to do: let the user decide how to manage metadata. They have the tools -- anything which can interact with a file. This includes users who never start X. > > >> The desire >>amongst users for ubiquitous metadata is very real - the current wave of >>"desktop search" products and technologies demonstrates this - >> >> > >Just like each previous claim "this /must/ be the next cool technology!", >also called later the "dotcom crash"... > > How does that analogy hold? An example of a technology which is actually in demand is similar to the tech bubble exactly how? >> but >>search is really only the lowest-hanging fruit of this new way of >>looking at data. Application-layer solutions like Beagle, >> >> > >That works without screwing up the whole system, AFAIU. > > > If you define "works" as "implements a proprietary layer which can't be used naturally by the same tools UNIX users have used for years to manage data". >And who says that a normal user isn't allowed to annotate each and every >file with its purpose or something else? I can very well imagine a system >in which users (say students in a Linux class) want to do so... on a shared >machine. Or users having a shared MP3 or photograph or ... collection, with >individual notes on each. Or even developers wanting to annotate source >code files with their comments, but leave them read-only (or have them >under SCM). > > > This same argument could be used to attack the idea of group permissions -- that groups of users might have conflicting goals. Implementing metadata in userspace via bundled files has the same drawback. >> If you're sysadmining a multiuser reiser4 >>box, and your users are able to modify the metadata of files they don't >>own, then you go to sysadmin purgatory. >> >> > >Bingo! And thus goes much of the supposed advantage of this nonsense. > > > You seem to be implying that on a Reiser4 filesystem used by multiple users that people other than file owners will be about to apply arbitrary metadata to any object. It seems to me that adding metadata to a file-as-dir will require the same permissions as writing to the file itself, giving Reiser4 *zero* disadvantage against other multiuser systems. What gives you the impression that adding metadata to a file-as-dir is any harder to secure against modification from other users than the files themselves are in the first place? >[I see that /opponents/ are accused here of lack of imagination, while I > see that the /proponents/ lack imagination... or perhaps just real-world > experience.] > > > Huh? The idea of arbitrary metadata isn't just with annotating MP3 files. One seemingly important consideration would be the flexible implementation of add-on mechanisms, like security data, that could be implemented with zero changes to the filesystem. And you get to use existing tools to deal with new and arbitrary interfaces instead of having to deal with GUI garbage just to interact with metadata. >>>other way; OpenOffice /has/ structured files, XML inside zipped files, >>>Java also uses zip files for its structuring needs, etc), or are ideas >>>that >>> >>> > > > >>And as a Java developer, I can tell you that the wide consensus is that >>this solution is half-assed and insufficient for both developers and >>users needs. In fact, I believe there is currently a JSR in progress to >>develop a more sophisticated Java packaging model. >> >> > >Presumably based on ReiserFS 4, which then has to be ported to whatever >platform you want to run Java on ASAP? Great for you! Wait a bit, and >you'll get what you want then, even across the board! > > Are unable to differentiate between a statement that a given solution is non-ideal and a suggestion that everything be done in a way that requires Reiser4? Someone mentions that the current solution isn't quite meeting all of their needs, and you respond with rhetoric accusing them of wanting everything to depend on Reiser4 and bloat the kernel. How is this productive? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 7:41 ` Chet Hosey @ 2005-07-05 20:55 ` Hubert Chan 2005-07-06 2:51 ` Horst von Brand 0 siblings, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-07-05 20:55 UTC (permalink / raw) To: Chet Hosey Cc: Kevin Bowen, Hans Reiser, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Fri, 01 Jul 2005 03:41:00 -0400, Chet Hosey <chosey@nauticom.net> said: > Horst von Brand wrote: >> And who says that a normal user isn't allowed to annotate each and >> every file with its purpose or something else? Explain how you currently allow users to annotate arbitrary files. >> I can very well imagine a system in which users (say students in a >> Linux class) want to do so... on a shared machine. Or users having a >> shared MP3 or photograph or ... collection, with individual notes on >> each. Or even developers wanting to annotate source code files with >> their comments, but leave them read-only (or have them under SCM). > This same argument could be used to attack the idea of group > permissions -- that groups of users might have conflicting > goals. Implementing metadata in userspace via bundled files has the > same drawback. The situation is even better with file-as-dir. If the administrator wants to allow users to edit the description metadata for the file foo, the administrator can set the appropriate permissions for foo/.../description, and keep foo read-only. >>Kevin Bowen <kevin@ucsd.edu> wrote: >>> If you're sysadmining a multiuser reiser4 box, and your users are >>> able to modify the metadata of files they don't own, then you go to >>> sysadmin purgatory. Actually, you could use something like unionfs to allow users to keep their own annotations without affecting everyone else's. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 20:55 ` Hubert Chan @ 2005-07-06 2:51 ` Horst von Brand 2005-07-06 13:26 ` David Masover 2005-07-06 20:10 ` Hubert Chan 0 siblings, 2 replies; 620+ messages in thread From: Horst von Brand @ 2005-07-06 2:51 UTC (permalink / raw) To: Hubert Chan Cc: Chet Hosey, Kevin Bowen, Hans Reiser, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hubert Chan <hubert@uhoreg.ca> wrote: > On Fri, 01 Jul 2005 03:41:00 -0400, Chet Hosey <chosey@nauticom.net> said: > > Horst von Brand wrote: > >> And who says that a normal user isn't allowed to annotate each and > >> every file with its purpose or something else? > Explain how you currently allow users to annotate arbitrary files. By keeping annotations /outside/ the files. [...] > The situation is even better with file-as-dir. If the administrator > wants to allow users to edit the description metadata for the file foo, > the administrator can set the appropriate permissions for > foo/.../description, and keep foo read-only. So now root is responsible in exquisite detail for random other users being able to keep info about my files? [...] > Actually, you could use something like unionfs to allow users to keep > their own annotations without affecting everyone else's. Again, root has to mount that stuff for each and every user? -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 2:51 ` Horst von Brand @ 2005-07-06 13:26 ` David Masover 2005-07-06 20:10 ` Hubert Chan 1 sibling, 0 replies; 620+ messages in thread From: David Masover @ 2005-07-06 13:26 UTC (permalink / raw) To: Horst von Brand Cc: Hubert Chan, Chet Hosey, Kevin Bowen, Hans Reiser, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Horst von Brand wrote: > Hubert Chan <hubert@uhoreg.ca> wrote: > >>On Fri, 01 Jul 2005 03:41:00 -0400, Chet Hosey <chosey@nauticom.net> said: >> >>>Horst von Brand wrote: >>> >>>>And who says that a normal user isn't allowed to annotate each and >>>>every file with its purpose or something else? > > >>Explain how you currently allow users to annotate arbitrary files. > > > By keeping annotations /outside/ the files. > > [...] > > >>The situation is even better with file-as-dir. If the administrator >>wants to allow users to edit the description metadata for the file foo, >>the administrator can set the appropriate permissions for >>foo/.../description, and keep foo read-only. > > > So now root is responsible in exquisite detail for random other users being > able to keep info about my files? If it's the general info that's associated with the file, and may even be stored inside the file, then yes, that's fair. Although I could certainly imagine foo/.../descriptions being a directory that's world-writable, allowing each user to maintain their own file inside of it. You can even set these per-user descriptions to be stored somewhere else, like the user's home directory, and that could work for CDs. >>Actually, you could use something like unionfs to allow users to keep >>their own annotations without affecting everyone else's. > > > Again, root has to mount that stuff for each and every user? Why is that a problem? Put it in a script. Mount each user's unionfs at boot. And it's "something like unionfs" -- maybe it's a feature of metafs or reiserfs that we haven't thought of yet. It certainly can't be unionfs as it stands, as unionfs doesn't work on top of any reiser. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 2:51 ` Horst von Brand 2005-07-06 13:26 ` David Masover @ 2005-07-06 20:10 ` Hubert Chan 1 sibling, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-07-06 20:10 UTC (permalink / raw) To: Horst von Brand Cc: Chet Hosey, Kevin Bowen, Hans Reiser, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Tue, 05 Jul 2005 22:51:07 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: > Hubert Chan <hubert@uhoreg.ca> wrote: >> On Fri, 01 Jul 2005 03:41:00 -0400, Chet Hosey <chosey@nauticom.net> >> said: >> > Horst von Brand wrote: >> >> And who says that a normal user isn't allowed to annotate each and >> >> every file with its purpose or something else? >> Explain how you currently allow users to annotate arbitrary files. > By keeping annotations /outside/ the files. So if I want to share annotations, I have to look in 20 different places? > [...] >> The situation is even better with file-as-dir. If the administrator >> wants to allow users to edit the description metadata for the file >> foo, the administrator can set the appropriate permissions for >> foo/.../description, and keep foo read-only. > So now root is responsible in exquisite detail for random other users > being able to keep info about my files? I can grant people permissions to write random info into my own files. Or they can use unionfs if I don't grant them permissions. Remember: the above argument was citing an advantage of file-as-dir over packed files (storing metadata as a tar or zip file, similar to what OpenOffice.org does, or even something like exif or id3). In a packed file, I can't allow people to edit the description attribute without allowing them to edit the entire file. With file-as-dir, I get much finer grained permissions. > [...] >> Actually, you could use something like unionfs to allow users to keep >> their own annotations without affecting everyone else's. > Again, root has to mount that stuff for each and every user? suid program that allows union mount into a directory within my own tree (or just into any directory that I have write permissions should be sufficiently secure). -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 18:47 ` Horst von Brand 2005-06-30 20:08 ` Kevin Bowen @ 2005-06-30 20:37 ` Zan Lynx 2005-07-01 7:16 ` David Masover 1 sibling, 1 reply; 620+ messages in thread From: Zan Lynx @ 2005-06-30 20:37 UTC (permalink / raw) To: Horst von Brand Cc: Hubert Chan, Hans Reiser, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 2764 bytes --] On Thu, 2005-06-30 at 14:47 -0400, Horst von Brand wrote: [snip] > Let me moderate that a bit: I'd be happy to have (some) files behaving > strangely, /if in exchange I get something very worthwhile/. I.e., Unix > filesystem sockets, named pipes, virtual filesystems all behave in weird > ways, but this downside is more than compensated by huge advantages (even > being able to do things that would otherwise be impossible). But all I see > is that file-is-directory advocates explain over and over how they are > bending over backards to get the whole mess working exactly like true > directories (nothing more, in the end quite a bit less). The uses I've seen > discussed don't really need files-as-directories (you get most of the > advantages by just tar(1)-ing up the contents, or packing them in some > other way; OpenOffice /has/ structured files, XML inside zipped files, Java > also uses zip files for its structuring needs, etc), or are ideas that > might be nice to have on exclusively one-user, isolated machines, like > "keep /my/ annotations/icon/application name/whatever with the file's > data", but that break down in multiuser (even serially, one user after the > other) systems or when files are shared (via network, or simply by sending > by mail or by copying from one machine to the other). In my rather ample > experience, that situation is rare today, rapidly dwindling in the arena > where Linux is mostly used. So this particular case of strange semantics > for files has no upsides I can see, only downsides. The downsides have been > discussed to death, and AFAICS there are fundamental problems with the idea > that can't be fixed or sanely worked around. So why bother? Structured files are fine when they're small but quickly become disgusting as they get bigger. Either you slowly rewrite the whole thing or you "fast save" by writing new sections to the end of it that replace existing sections. That's how you end up with documents that contain "deleted" information that was supposed to be confidential. Having the filesystem track each part for you and then creating a structured file when needed (and without needing to remember to run a special tool) is a far better idea. (reiser4 doesn't do this but its a possible idea) Another advantage to file-as-directory is being able to access all the file's meta-data through a single channel: file names and text data. It removes the need for chmod, chown, touch, getxattr, chacl and all the rest of the special tools needed to work with Unix files. It also makes it far easier to add new meta-data in the future, because no new tools have to be written to use it. So that's two reasons to bother. -- Zan Lynx <zlynx@acm.org> [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 20:37 ` Zan Lynx @ 2005-07-01 7:16 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-07-01 7:16 UTC (permalink / raw) To: Zan Lynx Cc: Horst von Brand, Hubert Chan, Hans Reiser, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Zan Lynx wrote: > On Thu, 2005-06-30 at 14:47 -0400, Horst von Brand wrote: > [snip] [snip-some-more] > Structured files are fine when they're small but quickly become > disgusting as they get bigger. Either you slowly rewrite the whole > thing or you "fast save" by writing new sections to the end of it that > replace existing sections. That's how you end up with documents that > contain "deleted" information that was supposed to be confidential. Doesn't OpenOffice already do something similar with zipfiles? I remember noticing huge differences in save times after adding large images to a presentation versus just changing some text. So, it's obviously not repacking each image every time I save. But, as the internal text is all XML, I doubt it's leaving "deleted" sections lying around. > Having the filesystem track each part for you and then creating a > structured file when needed (and without needing to remember to run a > special tool) is a far better idea. (reiser4 doesn't do this but its a > possible idea) Been discussed to death, and chances are, Reiser4 will do this with plugins. I imagine it being implemented in a more general way, though. For example, a meta-file which, when read, produces a zipfile/tarball/dump of the directory it belongs to. Then, to send such a document via email, you just navigate to the directory where it lives while looking for files to attach, only from /meta/vfs as a base, then attach the meta-file. The zipfile gets built automatically. To simplify the process, all you *really* need is a button on all FS navigation windows (Nautilus, Open/Save dialogs, etc) to switch back and forth between the meta view and the normal view. That, or a separate pane. This way, instead of patching the mail client to support all possible transformations you'd want to do before sending, or patching all applications to use zip (and any other transformation a user might want to do), or forcing the user to run another app, you automatically get *all* available transformations, in a way that could potentially look as smooth as adding zip support directly to the app. > Another advantage to file-as-directory is being able to access all the > file's meta-data through a single channel: file names and text data. > It removes the need for chmod, chown, touch, getxattr, chacl and all the > rest of the special tools needed to work with Unix files. It also makes > it far easier to add new meta-data in the future, because no new tools > have to be written to use it. All good points except the last one. New tools vs new plugins? People already know how to write the tools... > So that's two reasons to bother. That, and the possible paradigm shift. Navigating to the /meta dir for a file could be the commandline equivalent of right-clicking on the file. Of course, to some people, paradigm shifts are a reason *not* to bother... ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 17:10 ` David Weinehall 2005-06-30 18:47 ` Horst von Brand @ 2005-06-30 19:00 ` Hubert Chan 1 sibling, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-30 19:00 UTC (permalink / raw) To: Hans Reiser Cc: Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Thu, 30 Jun 2005 19:10:57 +0200, David Weinehall <tao@acc.umu.se> said: > For the analogy to be complete: > User has a file browser (say Nautilus) > The file browser sees the userland VFS (say a unified VFS between > GNOME and KDE) > The VFS sees the real file system I would say that this only works if everyone is forced to use the same VFS. In the web example, everyone is forced to use the same API (HTTP), and so everyone gets the same view. For the real filesystem, not everyone is forced to use the hypothetical unified GNOME/KDE VFS. So if I try to edit with gedit or kate, I get something different than if I try to edit with vi or emacs. As I see it: web browser <-> Nautilus/user apps web server <-> filesystem web server's filesystem/database/etc. <-> physical disk storage Of course, we all know that analogies are not perfect. The layering in both sides isn't exactly the same. And other people could assign different equivalences (e.g. web browser <-> Nautilus/user apps; web server <-> VFS; web server's filesystem <-> filesystem). Anyways, the analogy wasn't supposed to be about where to handle the magic extra functionality, whether userspace or kernel space. The analogy was for people who might think that it's stupid to return foo/.data when the user tries to open the directory foo; it was meant to illustrate that that idea isn't completely braindead. > This way the userland VFS can be ported to almost any platform. I think GNOME and KDE will always need a VFS if it wants to be cross-platform. But I think that if the kernel provides extra functionality, GNOME may be better off. For example, glib provides its own threading abstraction. But on systems that use pthreads, glib's threading library uses it for its implementation. And I think it can even be used on systems that don't offer threading, by doing its own emulation of threading. > When toying around on the desktop, an abstraction of what a file is > works fine with me. When doing serious work (programming, tar:ing up > stuff, etc) I want to be bloody sure that I see the files in the same > way always. I don't want surprises such as files suddenly behaving as > directories or vice versa. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 15:57 ` Hubert Chan (?) (?) @ 2005-07-01 7:47 ` David Masover -1 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-07-01 7:47 UTC (permalink / raw) To: Hubert Chan Cc: Hans Reiser, Horst von Brand, Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hubert Chan wrote: > On Thu, 30 Jun 2005 08:29:56 +0200, David Weinehall <tao@acc.umu.se> said: > > >>On Thu, Jun 30, 2005 at 12:33:10AM -0400, Hubert Chan wrote: >> >>>It's sort of like the way web servers handle index.html, for those >>>who think it's a stupid idea. (Of course, some people may still >>>think it's a stupid idea... ;-) ) > > >>And guess what? That's handled on the web server level (userland), >>not by the file system. So different web servers can handle it >>differently (think index.html.sv, index.html.zh, index.php, etc). > > > From the web *browser*'s point of view, it is handled by the > "filesystem" (which is provided by the various servers). The browser > doesn't care how or where the data is stored. It just requests a file, > and gets some data back. So the browser doesn't have to check for > http://www.example.com/, get a failure (trying to read a directory), > check for http://www.example.com/index.html, etc. In this way, the web > server controls (which I think takes the place of the filesystem in this > case) what gets shown (index.html.sv, etc.), instead of the leaving it > up to the browser. Somewhat flawed analogy, though. After the protocol definition, the browser proper will take any URL that the protocol handler likes, which is why file:// works. After the domain, the http/https handler will take any URL at all, except for maybe some character set issues. So assuming the server is compatible with itself, we don't have to worry about whether the browser supports going to a directory and having it behave as a file (index.html behavior), or going to a file and having it behave as a directory (as some scripts do -- I've seen urls that look like http://example.com/foo.cgi/bar.html) Among protocols that behave more like filesystems, such as FTP, you can't really pull the same tricks. When an FTP client asks for a directory, it's probably asking for a directory listing, and would be quite surprised to find a file there -- or the user would when binary data floods their terminal. I *think* this is how FTP works, but I haven't used it in years, except through a web browser. I still get the feeling that even a web browser expects an FTP file to behave as a file and an FTP directory to behave as a directory. But I'm also pretty sure that FTP would be much more receptive to file-as-directory than your average sysadmin would. For one, breaking tar is unforgivable, and the only ways I can think of fixing that issue are shaky at best when you consider how many apps might do things oh-so-slightly different than tar. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-30 3:04 ` Hans Reiser 2005-06-30 4:33 ` Hubert Chan @ 2005-07-05 15:46 ` Martin Waitz 2005-07-05 22:32 ` Jonathan Briggs 1 sibling, 1 reply; 620+ messages in thread From: Martin Waitz @ 2005-07-05 15:46 UTC (permalink / raw) To: Hans Reiser Cc: Ross Biro, Hubert Chan, Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 908 bytes --] hoi :) On Wed, Jun 29, 2005 at 08:04:58PM -0700, Hans Reiser wrote: > >How is directories as files logically any different than putting all > >data into .data files and making all files directories (yes you would > >need some sort of special handling for files that were really called > >.data). > > > Add to this that you make .data the default if the file within the > directory is not specified, and define a stanadard set of names for > metafiles, and you've got the essential idea, and any differences are > details. sure, that would work more or less, but what would it give you? You haven't introduced anything new that userspace couldn't do before. Just write a library which mangles pathnames and treats files and directories the same. You don't need the kernel to do that. Filesystems are there to store files. Everything else can be done in userspace. -- Martin Waitz [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 15:46 ` Martin Waitz @ 2005-07-05 22:32 ` Jonathan Briggs 2005-07-06 7:20 ` Martin Waitz 0 siblings, 1 reply; 620+ messages in thread From: Jonathan Briggs @ 2005-07-05 22:32 UTC (permalink / raw) To: Martin Waitz Cc: Hans Reiser, Ross Biro, Hubert Chan, Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 499 bytes --] On Tue, 2005-07-05 at 17:46 +0200, Martin Waitz wrote: [snip] > Filesystems are there to store files. > Everything else can be done in userspace. You could do filesystems in userspace too and just use the kernel's block layer. In fact you can reduce the OS kernel to just interrupts, memory management, context switches and message passing. Everything else can be done in userspace, but that doesn't always make it a good idea. -- Jonathan Briggs <jbriggs@esoft.com> eSoft, Inc. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-05 22:32 ` Jonathan Briggs @ 2005-07-06 7:20 ` Martin Waitz 2005-07-06 9:02 ` Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: Martin Waitz @ 2005-07-06 7:20 UTC (permalink / raw) To: Jonathan Briggs Cc: Hans Reiser, Ross Biro, Hubert Chan, Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 713 bytes --] hoi :) On Tue, Jul 05, 2005 at 04:32:00PM -0600, Jonathan Briggs wrote: > You could do filesystems in userspace too and just use the kernel's > block layer. but you can't do that in an library, you have to use a filesystem server in order to get access control. But you can build a library that handles uniform access to files and directories. Don't get me wrong, I'm all for a uniform interface for files and metadata, but I don't think that it has to be in the kernel. Gnome and KDE already have their own userspace VFS, something like that should be used. One has to distinguish between the low-level filesystem and the storage system which is presented to the user. -- Martin Waitz [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 7:20 ` Martin Waitz @ 2005-07-06 9:02 ` Hans Reiser 2005-07-06 17:30 ` Horst von Brand 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-07-06 9:02 UTC (permalink / raw) To: Martin Waitz Cc: Jonathan Briggs, Ross Biro, Hubert Chan, Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Martin Waitz wrote: >hoi :) > >On Tue, Jul 05, 2005 at 04:32:00PM -0600, Jonathan Briggs wrote: > > >>You could do filesystems in userspace too and just use the kernel's >>block layer. >> >> > >but you can't do that in an library, you have to use a filesystem >server in order to get access control. >But you can build a library that handles uniform access to >files and directories. > >Don't get me wrong, I'm all for a uniform interface for files and >metadata, but I don't think that it has to be in the kernel. >Gnome and KDE already have their own userspace VFS, something >like that should be used. > >One has to distinguish between the low-level filesystem and >the storage system which is presented to the user. > > > Yes, but to do what you advocate properly, the existing semantics currently in the kernel should be moved out of it into user space. I think the exokernel approach by Frans is a very interesting approach. I wish I had the experience with it necessary to know if it was effective. I do NOT take the position that name resolution should be in the kernel. I DO take the position that it should be either in the kernel or out of the kernel, and should constitute one cohesive and coherent body of code. If someone talks Linus into trying the exokernel approach, I will be happy to educate myself to where I have an opinion on whether that works. It is easy to see powerful advantages to the exokernel approach: I wish I understood the security model for it, and I wish I was sure that name resolution would not require too many context switches as one fetches each storage component required by a name resolution. Masover's words about HURD earlier in this thread were very well said. Context switching is expensive, and I find it easier to be sure my code will both perform well and be cohesive if it is all done in the kernel by one body of code. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 9:02 ` Hans Reiser @ 2005-07-06 17:30 ` Horst von Brand 2005-07-07 6:41 ` Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: Horst von Brand @ 2005-07-06 17:30 UTC (permalink / raw) To: Hans Reiser Cc: Martin Waitz, Jonathan Briggs, Ross Biro, Hubert Chan, Horst von Brand, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hans Reiser <reiser@namesys.com> wrote: [...] > I think the exokernel approach by Frans is a very interesting approach. > I wish I had the experience with it necessary to know if it was > effective. I do NOT take the position that name resolution should be in > the kernel. I DO take the position that it should be either in the > kernel or out of the kernel, and should constitute one cohesive and > coherent body of code. Right. > If someone talks Linus into trying the exokernel > approach, Are you nuts?! Such radical experiments do /not/ belong in the kernel on which millions of machines depend! Go and fork off a branch to play around with this, and if it does show real promise, you can then come back and try to integrate this into the official kernel. > I will be happy to educate myself to where I have an opinion > on whether that works. It is easy to see powerful advantages to the > exokernel approach: I wish I understood the security model for it, and I > wish I was sure that name resolution would not require too many context > switches as one fetches each storage component required by a name > resolution. Exactly the kinds of questions that have to get solid answers before any experimental patches can get off the ground. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-06 17:30 ` Horst von Brand @ 2005-07-07 6:41 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-07-07 6:41 UTC (permalink / raw) To: Horst von Brand Cc: Martin Waitz, Jonathan Briggs, Ross Biro, Hubert Chan, Kyle Moffett, David Masover, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Horst von Brand wrote: >Hans Reiser <reiser@namesys.com> wrote: > >[...] > > > >>I think the exokernel approach by Frans is a very interesting approach. >>I wish I had the experience with it necessary to know if it was >>effective. I do NOT take the position that name resolution should be in >>the kernel. I DO take the position that it should be either in the >>kernel or out of the kernel, and should constitute one cohesive and >>coherent body of code. >> >> > >Right. > > > >> If someone talks Linus into trying the exokernel >>approach, >> >> > >Are you nuts?! Such radical experiments do /not/ belong in the kernel on >which millions of machines depend! > > Your response is a flame, and I will not respond to it. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 1:40 ` Hubert Chan 2005-06-29 5:09 ` Horst von Brand @ 2005-07-01 9:08 ` David Masover 2005-07-01 18:55 ` Jeremy Maitin-Shepard 2005-07-02 12:28 ` Pierre Etchemaïté 1 sibling, 2 replies; 620+ messages in thread From: David Masover @ 2005-07-01 9:08 UTC (permalink / raw) To: Hubert Chan Cc: Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hubert Chan wrote: > On Tue, 28 Jun 2005 15:59:03 -0400, Kyle Moffett <mrmacman_g4@mac.com> said: > > >>On Jun 28, 2005, at 13:51:04, Hubert Chan wrote: > > >>> I don't know if VFS is the right place for it, but I agree that it >>>would be good to make it accessible to all filesystems. > > >>That's somewhat of a contradiction in terms. The whole point of the >>VFS is to hold all of the things that multiple filesystems want to >>share :-D. > > > VFS provides a common interface to the filesystem. I don't think metafs > needs any VFS changes. It may be able to get by without making changes > to the VFS, and if so, it shouldn't touch the VFS. It should just be > its own separate filesystem. > > I imagine most of it could be implemented by a FUSE filesystem. "could", yes. "should", no. I'll refer you to my HURD comment. That's not to say that none of this should be userspace, just that some of it most certainly *never* needs to touch userspace, such as cryptocompress. I'm not guessing that you wanted to make it FUSE, I just want to be pre-emptive here. FUSE will NOT work well for this. >>Maybe we just need better regular applications? > > > You mean patch them all so that they understand and can edit > xattr/substreams/etc.? The file-as-dir interface is meant to avoid > having to do that. metafs also avoids having to patch all the > applications by exposing them as regular files. Metafs also avoids having to patch tar. It's assumed that legacy backup systems can always avoid metafs and still catch almost everything important, and certainly everything they already do catch. With a hybrid or an entirely new backup system, we could catch everything, including any new ACL-like animals that people invent. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 9:08 ` David Masover @ 2005-07-01 18:55 ` Jeremy Maitin-Shepard 2005-07-02 12:28 ` Pierre Etchemaïté 1 sibling, 0 replies; 620+ messages in thread From: Jeremy Maitin-Shepard @ 2005-07-01 18:55 UTC (permalink / raw) To: linux-kernel David Masover <ninja@slaphack.com> writes: [snip] > That's not to say that none of this should be userspace, just that some > of it most certainly *never* needs to touch userspace, such as > cryptocompress. It seems that transparent encryption and compression really has little to do file-as-directory and metafs. There needs to be a mechanism for specifying (per-process or per-user) keys to the kernel, but other than that, the file or directory on which this transparent encryption/compression has been enabled can just be accessed normally. An attempt to access a file or directory for which a key is not available would give EPERM (or possibly the encrypted data, but this would be problematic for directories; it also might be the case that only encryption of file contents and file names would be supported). > I'm not guessing that you wanted to make it FUSE, I just want to be > pre-emptive here. FUSE will NOT work well for this. In suggesting the file-path-based search facility (which, by the way, is an interface that does not allow for searching with terms that include '/'), you surely cannot be suggesting that search-by-artist or IDv3 tag support be added to the kernel. [snip] -- Jeremy Maitin-Shepard ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-01 9:08 ` David Masover 2005-07-01 18:55 ` Jeremy Maitin-Shepard @ 2005-07-02 12:28 ` Pierre Etchemaïté 2005-07-02 23:09 ` David Masover 1 sibling, 1 reply; 620+ messages in thread From: Pierre Etchemaïté @ 2005-07-02 12:28 UTC (permalink / raw) To: reiserfs-list Le Fri, 01 Jul 2005 04:08:20 -0500, David Masover <ninja@slaphack.com> a écrit : Hi all, > Metafs also avoids having to patch tar. It's assumed that legacy backup > systems can always avoid metafs and still catch almost everything > important, and certainly everything they already do catch. With a > hybrid or an entirely new backup system, we could catch everything, > including any new ACL-like animals that people invent. As an administrator, a way to perfectly save and restore the content of file systems is a requirement. Crashes do happen, so do migration to new disk subsystems, system cloning, etc, etc. If restoration can be done with the standard tools you find on any live rescue CD, it's even better. All what's probably needed is some readable-writable non-confusing (fully POSIX compliant) "view" that's guaranteed to contain all existing "raw" data and metadata). BR, Pierre. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-07-02 12:28 ` Pierre Etchemaïté @ 2005-07-02 23:09 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-07-02 23:09 UTC (permalink / raw) To: Pierre Etchemaïté; +Cc: reiserfs-list Pierre Etchemaïté wrote: > Le Fri, 01 Jul 2005 04:08:20 -0500, David Masover <ninja@slaphack.com> a écrit : > > Hi all, > > >>Metafs also avoids having to patch tar. It's assumed that legacy backup >>systems can always avoid metafs and still catch almost everything >>important, and certainly everything they already do catch. With a >>hybrid or an entirely new backup system, we could catch everything, >>including any new ACL-like animals that people invent. > > > As an administrator, a way to perfectly save and restore the content of > file systems is a requirement. Crashes do happen, so do migration to new > disk subsystems, system cloning, etc, etc. I would suggest a plugin which produces a dump of the active FS. It's how XFS seems to do it, after all, only you don't need the commandline tool. > If restoration can be done with the standard tools you find on any live > rescue CD, it's even better. Any good XFS-enabled CD will have xfsdump. Similarly, any good Reiser4-enabled CD will have whatever we feel like putting there. Can ACLs currently be backed up with any other tool than some sort of dump? > All what's probably needed is some readable-writable non-confusing > (fully POSIX compliant) "view" that's guaranteed to contain all existing > "raw" data and metadata). The danger here is that things beyond POSIX must either be encoded into the file somehow. If we add a thumbnail property to a certain type of file, and the file format itself doesn't allow for any extra data, we can either choose to have a separate view entirely for backup so that the "backup" view (which forces the metadata into the file anyway) is separate, or we can choose to accept that existing backup tools will catch existing attributes that they were programmed to expect, and all new stuff can be handled by some sort of dump. I say "some sort of dump" because a separate backup view, because it makes some files unusable except for transport, is effectively a dump, only shown as a tree, instead of a single file. But as long as we're doing that, we may as well provide an actual dump plugin so that one can do "gzip /metas/some/where/.../dump > /mnt/some/backup/medium". It'd be more efficient than tar for that sort of thing. Both are useful, though -- the tree is better for rsync backups, if you're either using /meta on the other end or are planning to only restore to a /meta enabled system. I think we should do both ways. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 2:59 ` Kyle Moffett 2005-06-28 3:45 ` Hubert Chan @ 2005-06-28 20:07 ` David Masover 1 sibling, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-28 20:07 UTC (permalink / raw) To: Kyle Moffett Cc: Hubert Chan, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Kyle Moffett wrote: > On Jun 27, 2005, at 22:21:35, Hubert Chan wrote: > >> On Mon, 27 Jun 2005 18:27:26 -0500, David Masover >> <ninja@slaphack.com> said: >> >>> Kyle Moffett wrote: >>> >>>> I think the '...' is just a bad idea in general, because it breaks >>>> tar and such. An interface like this might be simpler to design and >>>> use: >>>> >>>> # mkdir /attr >>>> # mount -t attrfs attrfs /attr >>>> >>>> /attr/fd/$FD_NUM == '...' directory for a filedescriptor >>>> /attr/fs/$DEV_NUM/$INODE_NUM == '...' directory for an inode >> >> >> The most obvious (at least obvious for me) complaint about this is that >> the attributes are separate from the file. (In other words, it's a bit >> ugly. ;-) ) So if you want to backup a file, along with all its >> (extended) attributes (or substreams, or ... etc. ...), you need to >> backup the file, and find the appropriate /attr/fs/$DEV_NUM/$INODE_NUM >> and back that up. If I want to edit an attribute, I need to find >> $DEV_NUM and $INODE_NUM (which I have no idea how to do), rather than >> just "edit foo/.../extended_attribute". (Or using your "getattrpath" >> command, it would be a two-step process -- run getattrpath, then run >> editor -- rather than a one-step process. Of course, this is only >> mildly annoying.) > > > These are not really "attributes" so much as they are "metadata", for > example, a "contents" subdirectory, if one existed, would be based on > the original file, and therefore non-unique, but would be looked up > based on information about the original file. If you say so. I like the name "meta" because it looks cooler and is easier to type, but I could live with "attr". >> It also exposes a difference between attributes and regular files. >> e.g. can I add attributes to an attribute? (say, I have a thumbnail >> attribute for the file ~/foo, and I want to add a mime-type attribute to >> that thumbnail attribute.) If you want to allow it, you have to be >> careful not to run into a big loop generating an infinite number of >> inodes in the attrfs. (e.g. >> /attr/fs/$(getattrpath /attr/fs/$(getattrpath ~/foo)/thumbnail)/ mimetype >> -- attrfs shouldn't generate the inode for that until >> /attr/fs/$(getattrpath~/foo)/thumbnail is accessed, and maybe not even >> then...) > > > Agreed. I discuss this more below > >> That said, I prefer that interface over xattr or openat. > > > Absolutely. On the other hand, that's not to say it can't be improved. > >> >>>> It would be usable from a shell with a simple "getattrpath" command >>>> which returns "fs/$DEV_NUM/$INODE_NUM" by stat-ing any given path. >> >> >>> Still pretty annoying, but maybe a good idea, especially if the shell >>> gets extended to support it. Not sure I like using inode numbers, >>> though -- I like the idea of being able to symlink to stuff inside the >>> meta-file dir. >>> >> >> The advantage of using inodes rather than pathnames is that it is robust >> with respect to file renaming/moving. It also allows things like >> adding attributes to symlinks (since the inode number for a symlink is >> different from the inode number for the file that it points to). >> >> You can still symlink. It just takes a little more effort to figure out >> what $DEV_NUM/$INODE_NUM to use. > > > Also, unlike a symlink, if the path doesn't change and the file does, it > will break, also, if the file is removed and another created elsewhere, it > will be redirected improperly. Perhaps a new symlink syntax is needed to > allow attribute specification (Ick, more changes :-\). There are symlinks, and there are hardlinks. When I want a link to a file which doesn't change even if the file is moved, I use a hardlink. When I want a link to a particular location, even if the file there is deleted and then replaced, or if the file is not likely to move anytime soon, I use a symlink. Besides, can't we have it automagically figure out what the fs is once we give it the device number? I propose we should be able to have: /meta/$DEV_NUM/$INODE_NUM /meta/vfs/path/to/file That way, I can symlink to whichever is appropriate. >>> Hans, thoughts? Seems to be namespace fragmentation, but seems >>> usable, less breakage, and so on. And should it be /attr or /meta? >> >> >> For the mount point, it doesn't matter; it's up to the user. It's the >> attrfs or metafs or ???fs that matters (but which will greatly influence >> whether people user /attr or /meta). > > > "meta" seems the more descriptive name. There should also probably be a > somewhat standardized location for this, such that programs can locate it > without much trouble. This mechanism would be usable from all FSs, and > could be built into the VFS. Maybe. I think it should be its own FS, mountable and unmountable, just like /proc or /sys. It's a standardized location anyway, but easier to admin that way. But we definitely want to support multiple FSes. Even more reason to make it a separate system, accessible as its own mountable FS. > Also, it would allow one to access the meta > data of meta data (if supported by the filesystem, and possibly only > through the file descriptor lookup, due to numbering limitations.) Or just use: /meta/vfs/meta/$DEV_NUM/$INODE_NUM /meta/vfs/meta/vfs/path/to/file/some/meta/file Loop as many times as the FS supports -- if the FS doesn't like a particular amount of looping, the file will simply not be there. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsGuHngHNmZLgCUhAQLFDA//etF3JuQVLCz4XV60yDWZ3rEtf9mHew7F Ki0ON1q2P94TppOhjhUWPUQ8v53K5AIRkXyr0Uc1AVxqzj8ff5IHxYRXOJqB1OAC 3sJpRawIW+4g7Odwej8320gVrSPGJ4xMOm5HN2QFWhjlqPdAcMIydoZnM5WFAV/N 0/iaHA54wIrxGepCp0KPmyW3PJTRQj3PiBInvEER6dSnPpjKScEbSGLgVXHaHVpA X3ujOvNjFzmPrbmytQuJY3lB3dFSOXVarv0RKOaCqbDUW1KoWLiXnXD1t+9qLqRy w3LlRFMTooh5jsmWsorEVDqIS2ZsPL5fnYOkx5pTgUU2/pVLyedJWnB3zbzOqZ4T kypdr9yZZqoVakYNTmHqEGQp4z5BuKzZgoAjAJXJrST1+IhdDfjAWQ61NrSsFP3C XIVfsuY2WwUiy3wUrATUgIL8cpeKz9WRzJOsnD7b5e8YB8ceR8uElSD2VB5RKF+q v7E1chqofOWgIgclSPMq74jCa3iD6PF5W8ohS5J0P4TafyTr8KGB17Is81O55d5D 3aoioC6JzKW9iQ4SM+CxeS5gTNqI/O9cA/NU+aZzrz9P6xQSB5bOoMMfm1N3pzLD obhZaGyVZtpn5mc+AoVi9JLKs105JFZzHgdZYiwsJdIR07oMZYfOU34As0dgp78+ O6Em4T/9YAQ= =hvkI -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 2:21 ` Hubert Chan 2005-06-28 2:59 ` Kyle Moffett @ 2005-06-28 20:22 ` David Masover 2005-06-29 1:51 ` Hubert Chan 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-28 20:22 UTC (permalink / raw) To: Hubert Chan Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hubert Chan wrote: > On Mon, 27 Jun 2005 18:27:26 -0500, David Masover <ninja@slaphack.com> said: > > >>Kyle Moffett wrote: >> >>>I think the '...' is just a bad idea in general, because it breaks >>>tar and such. An interface like this might be simpler to design and >>>use: >>> >>># mkdir /attr >>># mount -t attrfs attrfs /attr >>> >>>/attr/fd/$FD_NUM == '...' directory for a filedescriptor >>>/attr/fs/$DEV_NUM/$INODE_NUM == '...' directory for an inode > > > The most obvious (at least obvious for me) complaint about this is that > the attributes are separate from the file. (In other words, it's a bit > ugly. ;-) ) So if you want to backup a file, along with all its > (extended) attributes (or substreams, or ... etc. ...), you need to > backup the file, and find the appropriate /attr/fs/$DEV_NUM/$INODE_NUM > and back that up. If I want to edit an attribute, I need to find > $DEV_NUM and $INODE_NUM (which I have no idea how to do), rather than > just "edit foo/.../extended_attribute". (Or using your "getattrpath" > command, it would be a two-step process -- run getattrpath, then run > editor -- rather than a one-step process. Of course, this is only > mildly annoying.) Especially considering you could add a tar/rsync flag to handle it for you. And most likely, each meta dir would have its own serialization method (read: file) so that you didn't find yourself trying to back up huge infinitely recursive structures, or accidentally triggering a huge tar operation, or something... > It also exposes a difference between attributes and regular files. > e.g. can I add attributes to an attribute? Under my scheme, yes, and it works the same way. > (say, I have a thumbnail > attribute for the file ~/foo, and I want to add a mime-type attribute to > that thumbnail attribute.) If you want to allow it, you have to be > careful not to run into a big loop generating an infinite number of > inodes in the attrfs. I'm not sure if inodes are a problem, but this kind of stuff would be up to the fs, not the attrfs or metafs. > That said, I prefer that interface over xattr or openat. Absolutely. But I think both should exist, so that programs currently using xattr will still work. Maybe you'd have: /meta/vfs/home/foo/attrs or something... Come to think of it, that changes the proposal a bit. You need a different way to access the meta-dir for files than for directories, if we're going to support /meta/vfs. No biggie: /meta/vfs/file/home/bob/sue.mpg/acl /meta/vfs/dir/home/bob/acl >>>It would be usable from a shell with a simple "getattrpath" command >>>which returns "fs/$DEV_NUM/$INODE_NUM" by stat-ing any given path. > > >>Still pretty annoying, but maybe a good idea, especially if the shell >>gets extended to support it. Not sure I like using inode numbers, >>though -- I like the idea of being able to symlink to stuff inside the >>meta-file dir. > > > The advantage of using inodes rather than pathnames is that it is robust > with respect to file renaming/moving. It also allows things like > adding attributes to symlinks (since the inode number for a symlink is > different from the inode number for the file that it points to). I think we need both interfaces, especially because the /meta/vfs interface makes looping easy. > You can still symlink. It just takes a little more effort to figure out > what $DEV_NUM/$INODE_NUM to use. Doesn't have the same effect. >>Actually, I like this. Give me some time to let it percolate. > > >>Hans, thoughts? Seems to be namespace fragmentation, but seems >>usable, less breakage, and so on. And should it be /attr or /meta? > > > For the mount point, it doesn't matter; it's up to the user. No, it's not. It's up to the user and the application developers. Sure, I could mount /proc somewhere else, but that would confuse some things. Mounting /meta somewhere else would confuse even more things. > It's the > attrfs or metafs or ???fs that matters (but which will greatly influence > whether people user /attr or /meta). True, but I think they should still hand down a decree of where it is to be mounted by default. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsGxn3gHNmZLgCUhAQLWkg/+MIzXbygGoHubwLcw5SI1eeanVssukzF7 9kl+3qYcpbC6l/4fFkezBA8YCb8yy/VCfadAJ4Vgsr7T0rvGq1fvKIER6Q4LfU22 YCkHfU6dCnTmTjqHTFyYqZX95J6JXKUT/HCXn5AaYC5DIOYMp1gFHgj6P1wKOb/o KRvuOSIO0ta8fpPg6tBe6ve92sp99zqbacUplN+38+eBEIZUI7s7Dkg1lWX7ScyZ LVtHDEDJUuP0/Frq3xZHFKdgsVhoDtCm5Cl6EBX0OSubExPa/7ID2ANA+W01lqbm 9KMtXAcp/K8uOVkM8YDpFvvP2Ru/gp+iKy993lpGY9bEhIZ6BVz4zpVMvloHQe3a GCdsvub1yP7qQa/7LpkN6Shbw8W25iXcG0QUXa6XuiaJZIsVvRThrWOPXstx9GGQ KDZtMGIzH5vJcXQzIKzHMrfJBIKMmu3c/Et1EmdJy1t1PkH95+gGCrYc/nfBizfk NMYIp8S+RdGXwIcOWcwu4WspTJEZg1w7spXuydBZHd55YCq5JJE7lLNoK8Y3ZXks xC7b6Eo/0FgsxUmoQ9QYl9kzrxAejBGWmmTfsQGjFhu0Ep7jVCZHRRSq8r2RwRj/ AYY2JeFu7yRHsKy29BYipo0etR38Z5+Z0ZOn3iZZHHVz7D6u12U+69LXUEeyp46H mYzXXJOmIVA= =h/Gv -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-28 20:22 ` David Masover @ 2005-06-29 1:51 ` Hubert Chan 2005-07-01 9:00 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-06-29 1:51 UTC (permalink / raw) To: David Masover Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Tue, 28 Jun 2005 15:22:55 -0500, David Masover <ninja@slaphack.com> said: > Come to think of it, that changes the proposal a bit. You need a > different way to access the meta-dir for files than for directories, > if we're going to support /meta/vfs. No biggie: > /meta/vfs/file/home/bob/sue.mpg/acl > /meta/vfs/dir/home/bob/acl What if /home has an extended attribute named bob? Then is /meta/vfs/dir/home/bob a file or a directory? -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-29 1:51 ` Hubert Chan @ 2005-07-01 9:00 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-07-01 9:00 UTC (permalink / raw) To: Hubert Chan Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hubert Chan wrote: > On Tue, 28 Jun 2005 15:22:55 -0500, David Masover <ninja@slaphack.com> said: > > >>Come to think of it, that changes the proposal a bit. You need a >>different way to access the meta-dir for files than for directories, >>if we're going to support /meta/vfs. No biggie: > > >>/meta/vfs/file/home/bob/sue.mpg/acl >>/meta/vfs/dir/home/bob/acl > > > What if /home has an extended attribute named bob? Then is > /meta/vfs/dir/home/bob a file or a directory? Ah, you caught it. I knew it didn't feel right. So /meta/vfs/file and /meta/vfs/dir are worthless. Ok, there has to be a delimiter of some kind. That, or there has to be a way we can pick up front how deep we're going. Or, someone else has to come up with a better idea. Or, start out with the assumption that we're talking about metadata, instead of the other way around. I don't like delimiters because the way we usually deal with these is escaping them when we need to. For instance, if I want to actually pass a literal " character to some command, I can wrap it in ' characters or add a \ to the front of it. But we're expecting this to be simpler than that. A program trying to access the metas for some file shouldn't have to worry about escaping. Last time we tried to come up with a delimiter, we ended up with ... and ..metas as possible candidates, and ..metas was in the source. As unlikely as it is to hit either of those, I still don't like them. Programs would mostly use the inodes anyway, and users would mostly use the /meta/vfs interface, but it's still not pretty. But, the only alternative I can think of now is /meta/vfs/2/home/bob is a file and /meta/vfs/3/home/bob is a directory. That's not something we want our users to have to do, especially considering we want to be able to have metadata of metadata. That, or we can try something like: /meta/vfs/#/home/#/bob is a directory /home/vfs/#/home/bob is a file where we add a delimiter to the /meta dir which identifies where each directory begins, not each set of metadata. To clarify a bit: $ ls /meta/vfs/ mime permissions acls # [snip] $ ls /meta/vfs/#/ proc sys tmp etc home [snip] $ ls /meta/vfs/#/home mime permissions acls # [snip] $ ls /meta/vfs/#/home/#/ bob billy hank sue [snip] This is pretty annoying for the user, though, because it's more verbose. But, at least this way, we can guarentee that no one will kill our delimiter, because it exists in a new namespace we're creating anyway. That is, if we have to, the old xattrs stuff can go in its own folder, and so even if some app currently depends on a '#' app, it won't kill our delimiter. Now, if only there was an easier delimiter to type... Ok, since file-as-dir was going to do this anyway, I think the way to go is probably the original '...' delimiter. It's safer, because it stays in /meta, but it's dangerous, because someone might actually try to create a '...' file on an existing system. But, I think we can live with introducing a delimiter, more than we can live with uglifying the /meta/vfs interface, especially because "touch ..." wouldn't make the system blow up, you just couldn't easily get metadata for "..." ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 23:27 ` David Masover 2005-06-28 2:21 ` Hubert Chan @ 2005-06-28 2:34 ` Lee Revell 1 sibling, 0 replies; 620+ messages in thread From: Lee Revell @ 2005-06-28 2:34 UTC (permalink / raw) To: David Masover Cc: Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Mon, 2005-06-27 at 18:27 -0500, David Masover wrote: > Right on all points. Just remember that some change is good. Why do > we > have ALSA now? Everything a user can do with ALSA, they can do with > OSS, AFAIK. > Wrong, you have it backwards. The ALSA API is a superset of the OSS API. Otherwise, what would the point have been? Lee ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 3:24 ` David Masover 2005-06-27 3:40 ` Kyle Moffett @ 2005-06-27 5:38 ` Horst von Brand 2005-06-27 6:18 ` David Masover 1 sibling, 1 reply; 620+ messages in thread From: Horst von Brand @ 2005-06-27 5:38 UTC (permalink / raw) To: David Masover Cc: Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover <ninja@slaphack.com> wrote: > Kyle Moffett wrote: > > On Jun 26, 2005, at 22:37:48, David Masover wrote: [...] > That just means the zip plugin has to be more carefully written than I > thought. It would have to be sandboxed and limited to prevent buffer > overruns and zip bombs. [...] > Remember that zip, at least, would not be in the kernel. I think what > is going in the kernel is gzip and lzo, and it's being done so > transparently that you never actually see the compressed version. Doesn't matter if it is zip of some other compression, the problem is exactly the same. > > Can you illustrate for me with precise, clear, and unambiguous arguments > That will take some time. And some thinking. See? Exactly what has been demanded by all the "unfair", "ReiserFS-racist", "shove-any-junk-but-do-not-accept-perfect-filesystems-into-the-kernel" etc people here on LKML from the very start. > > how this can avoid all possible pitfalls, > Especially if you want perfection. Perfection would be nice, but (IMHO) not required. [...] > Now, the cryptocompress as it currently stands does not involve ... and > does not introduce any new security holes in the way that you are > describing. There might be some issues with key management (someone > hinted vaguely at that), but nothing insurmountable. OK, I see a week of flamefest going by until you admit it has the same problemas as compression (Hint: Most encryption compresses first, in order to give would-be cryptoanalysts less of a toehold via non-uniform distributions, and having less work to do), and then a whole lot of its /own/ problems (that somebody can peek at a cached uncompressed copy of my files is not so bad, if they can peek at my decrypted files I'd be rather less pleased... and here you have to include a malicious root (Yes, I'm paranoid. Doesn't mean root is not out to get me.).). Perhaps this can't be all solved, but the exact boundaries of what /is/ provided, and what /isn't/ must be made clear. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 5:38 ` Horst von Brand @ 2005-06-27 6:18 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-27 6:18 UTC (permalink / raw) To: Horst von Brand Cc: Kyle Moffett, Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Horst von Brand wrote: > David Masover <ninja@slaphack.com> wrote: > >>Kyle Moffett wrote: >> >>>On Jun 26, 2005, at 22:37:48, David Masover wrote: > > > [...] > > >>That just means the zip plugin has to be more carefully written than I >>thought. It would have to be sandboxed and limited to prevent buffer >>overruns and zip bombs. > > > [...] > > >>Remember that zip, at least, would not be in the kernel. I think what >>is going in the kernel is gzip and lzo, and it's being done so >>transparently that you never actually see the compressed version. > > > Doesn't matter if it is zip of some other compression, the problem is > exactly the same. zlib is in the kernel. Isn't lzo also in the kernel? The problem is still the same, but not as bad. And it's completely irrelevant to the current cryptocompress. Saying that someone could create a malicious transparently-compressed file that blows up when decompressed (inflated?) is like saying that my /dev/hda5 is no longer trusted. Because that's the only layer at which a userland program can see the compressed version of a transparently-compressed file. Now, if you're talking about going the other way, I think the problems were less about buffer overflows and more about bombs where 1k compressed data = 1 gig uncompressed data. Going the other way is kind of pointless if you're an attacker. Unless there are buffer overflows in the kernel's zlib. I hope not! After all, I don't necessarily trust all the CDs that I try to mount! >>>Can you illustrate for me with precise, clear, and unambiguous arguments > > >>That will take some time. And some thinking. > > > See? Exactly what has been demanded by all the "unfair", "ReiserFS-racist", > "shove-any-junk-but-do-not-accept-perfect-filesystems-into-the-kernel" etc > people here on LKML from the very start. Back at ya. Many of the people I can't abide here are people who don't seem to actually understand anywhere near as much of Reiser4 as I do. And I've never read much of its source code. It would be nice if this was approached more along the lines of helping get good stuff into the kernel than keeping anything bad out. But that's too much to ask, no sarcasm intended. Didn't Linus say that his job is more to keep stuff out anyway? > [...] > > >>Now, the cryptocompress as it currently stands does not involve ... and >>does not introduce any new security holes in the way that you are >>describing. There might be some issues with key management (someone >>hinted vaguely at that), but nothing insurmountable. > > > OK, I see a week of flamefest going by until you admit it has the same > problemas as compression Did I admit that compression has problems? I admitted that zip has problems, not the planned cryptocompress plugin. > and then a whole lot of its > /own/ problems (that somebody can peek at a cached uncompressed copy of my > files is not so bad, if they can peek at my decrypted files I'd be rather > less pleased... and here you have to include a malicious root (Yes, I'm > paranoid. Doesn't mean root is not out to get me.).). Perhaps this can't be > all solved, but the exact boundaries of what /is/ provided, and what > /isn't/ must be made clear. Can't trust the kernel at all if root is malicious. For that matter, can't trust anything if root is malicious. Or is there a system which blocks such things as keyloggers, kernel modifications, eavesdropping on random ptys, or simply brute forcing /dev/mem? I don't see any existing solutions other than heavily patched everything (text editors included) for even blocking simple temporary file hijacking. Now, if root is not malicious, we can just go back to UNIX permissions to keep your files safe -- only now your stuff is safe if the FBI steals your disk, and I assume your root can be trusted to power off the system before they can touch it in that event? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr+aM3gHNmZLgCUhAQLEGw//YgLNHfZNRBBD9wFd6Si0xnl+75eAwqOm 7WMDPdtXeZORhUnmNnS+EO71nMupUQmOaMI/AGbAJgTRHJKAiVKz1rt25TtMfkMg tOeZ4PGOmI2eMe2Ltw8ocR6YDYLc/2VTLCE51pCvVswHPbcY2W+j1JHoIwo/y/3f /WnO8QYNdGnvlYJB+smNEpO8ggwPItK5Ge2PoK1+3A+e+boX2xZyk3RzZ5Oth3Se H8oW9wfNXoXp50BjVRXcCcOSbHiFFYWMRUD/i3izFwB3JNS523rMLjyJH/5zeciL lW+b9dCjHqt0ULtkuw0gVbEQh4LTPBKM8WIKDRNkuFl9kz8FQk0BuQrpvmr8JZ3z 4S4etxhdnmuxMkXznJ0ioUTa+p7hXjRxpN1wZLXQi9xJfnOEkUHazI8GZui7qrlQ UlRyxyZTezEfROk+Ova0DWfAJEV4qY2SktxXZeMr7Z2a8WdkdozfPOFHt1GG38c4 xc/L5z2maXAG0fQbZPZtrYi65ES5MQ472T6KNnwNb3nFJr1CRO0l4PpZa6kNLQ1G XDKKpsPUR9A1/oOh81Ep1YN+RfJKWJCU3O59z1+ro3eKa1ckWEjX2TEwKIjeUaGd B8n5FenOsSslHw1of1aVCRyNVUMFH2wTRf0CcIVBIMxZJ/RhJ6m2tqbOIsIK4V+H rMEvogPEbtM= =1+LH -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 2:37 ` David Masover 2005-06-27 3:10 ` Kyle Moffett @ 2005-06-27 4:23 ` Valdis.Kletnieks 2005-06-27 5:31 ` David Masover 2005-06-27 4:27 ` Valdis.Kletnieks 2 siblings, 1 reply; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 4:23 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 898 bytes --] On Sun, 26 Jun 2005 21:37:48 CDT, David Masover said: > Assume we can do on-disk caching, similar to fscache/cachefs for nfs. > Now, benchmark: > > $ unzip linux-2.6.12.zip && make -C linux-2.6.12 > > versus the hypothetical > > $ make -C linux-2.6.12.zip/.../contents > > This is an automatic performance gain, in theory, because the second > command is identical to unzipping just the parts you need into > linux-2.6.12, then running "make". Nope, they're not identical. The first specifically unzips it into the file system, leaving the zip file intact. The second, you're having to take all those .o files and other stuff that the 'make' generates and put them back into the .zip file *on the fly* - when the 'make' is half done, the .zip should reflect a directory tree that has had half the make execute.... (Think - after that hyptothetical 'make' completes, where is 'vmlinux'? ;) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 4:23 ` Valdis.Kletnieks @ 2005-06-27 5:31 ` David Masover 2005-06-27 5:41 ` Valdis.Kletnieks 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-27 5:31 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Sun, 26 Jun 2005 21:37:48 CDT, David Masover said: > > >>Assume we can do on-disk caching, similar to fscache/cachefs for nfs. >>Now, benchmark: >> >>$ unzip linux-2.6.12.zip && make -C linux-2.6.12 >> >>versus the hypothetical >> >>$ make -C linux-2.6.12.zip/.../contents >> >>This is an automatic performance gain, in theory, because the second >>command is identical to unzipping just the parts you need into >>linux-2.6.12, then running "make". > > > Nope, they're not identical. The first specifically unzips it into the file > system, leaving the zip file intact. The second, you're having to take all > those .o files and other stuff that the 'make' generates and put them back > into the .zip file *on the fly* - when the 'make' is half done, the .zip should > reflect a directory tree that has had half the make execute.... This assumes we limit ourselves that way. I doubt something I think of in five minutes is the final API. > (Think - after that hyptothetical 'make' completes, where is 'vmlinux'? ;) In the cache where the stuff was unpacked to. Anyone who wants to discuss things that will actually be done soon, or anything relevant to merging reiser4 in the near future, can stop reading now. *If* we decide that this must go both ways, *then* we either turn off write support inside the zipfile and do "make" with a symlink farm (cp - -as isn't hard), or (better) we can set things up so that only on access (most likely a read) of the original zipfile do we re-add all the changes. I think copy-on-write files are planned sometime, too. So one could imagine doing a COW of the original zipfile, so that you have: linux-2.6.12.zip # original, for redistributing and patching linux-2.6.12-mine.zip # your own patchset, ready for redistributing And, of course, there's always an option to treat the cache as its own COW of the original zipfile. This system (which I did discuss on "Silent Semantic") is also useful for various other ways one would want to package something. Imagine someone's made a bootable iso which uses zisofs for transparent compression. You can set things up so that the iso->files plugin works as zip does above, grabbing the files you need on the fly and putting them in an on-disk cache, and files->iso takes only the files which have changed and runs them through mkzftree, then takes the whole tree and runs it through mkisofs. This actually isn't a performance win at all, if you know what you're doing. You could always set up a script (as I did when playing with this) which uses mkzftree's --crib-path option. But, transparency is nice. This way, all that's needed to build a new image is to chroot into some_file.iso/.../contents and change whatever you wanted, even use a package manager from the image, and when you're done, "cdrecord some_file.iso" will build the image and burn it. It also doesn't have to be lazy. Files could be compressed as you go, either in a blocking way or at a low priority, or they could be put off until you actually want to burn the image. I can imagine all kinds of packaging getting easier this way. A universal interface to peek inside of some sort of package, image, or software distribution and change its contents, then build a modified image. I guess that makes it a sort of "make" plugin. OK, here's an example where the "zip"-like behavior makes sense: Suppose you buy a computer with a desktop Linux pre-installed. But, on your first boot, the system is entirely composed of one giant copy-on-write tree, which refers to all the packages needed to build the entire system. All the packages in the standard distribution are pre-loaded onto the hard drive as (say) .deb files. In this situation, every time the user runs a program for the first time, it unpacks from a (presumably compressed) package. When a package is entirely unpacked, it may be removed. Since most users won't use a majority of the software, there's a huge amount of disk space saved. Yet when a user does want to use some random piece of software, they simply run it, and it behaves as if it was already installed. Download a package, and it can be run almost immediately after "installation". A bit of a slow startup, but maybe not as much as trying to unpack absolutely everything in the package all at once and get it set up. For example, on Debian, the Foomatic PPDs all come in one package, making it easy to find the right one from the CUPS web interface -- but you only need a maximum of one PPD file per printer you want to use, and there are hundreds of PPD files in the foomatic-ppds package. There are plenty of other programs (Mozilla, OpenOffice) which install lots of components which you may never use, and certainly won't use immediately on your very first run. This way you save space and installation time. I'm sure this was also discussed in "Silent Semantic", and I think it is mostly not my idea. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr+PQXgHNmZLgCUhAQLwoQ//f2vNSd4ZGBPjoaZBq8VlZ5l+wWCzinCF h21YI5f7l3d1uGO/oo/E4AESrAFN0A410fNNfAVEtNPJy64SlrrLY/PMBuOB5w7k 83ZNLSqExdblA64FkPkTB6/l+g1fmhey2xAIRvTUrSyAwnbs6fYD6sNq2oVV/czR G4t01iWyEGNyt3Ik41Pl/UqVCepHWcNNRUJL5eQYEakbp1hSwTU3NVkzwHeq/MJ/ OPP81Pq5NMfUUkH5h+G3/zghM44czSTPezC2Be/VS7vx0BvLWOSmS2qaPCogr35o QJwksZqaUirHtVotG353vShTyOSXlkEfhelyEKrMp7V+dK9ChP4hJeNTrS4eAxsF Wg25ZhFWN2D3ALdYdiG4dG2qFqSgR/mM2wwxzgumXI1YY1g0ppHDBtgRi8feXfjn COB+G0w1cM2ofGz2v2eCZIps2cS/5bWG6eyjdFN/z+KnlO6vuDiQ/z9y1KwUKURs u8+VLr9aWDNKuFisDbRoIc2Emq9sHUFNVG9WxrTY2Yd7RXWQfv1+gzvJt8FOwKVW 49L1jU2YLR31Kj6cc7HnHowzzVBTuZxJV9nnZkvpVQNVa/NFM2n44Og+Uvm06ikG 1JK2oLNgQGwRWW+UXJIBDqABYqh5COx4PMeRI3/cRPOxBLUdk34Hp+35PnvNSx3D pffWHDlDaew= =MtUX -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 5:31 ` David Masover @ 2005-06-27 5:41 ` Valdis.Kletnieks 2005-06-27 5:57 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 5:41 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 657 bytes --] On Mon, 27 Jun 2005 00:31:46 CDT, David Masover said: > *If* we decide that this must go both ways, *then* we either turn off > write support inside the zipfile Oh, *that* will do wonders for command symmetry. And you just shot down the whole 'mv foo bar' being equivalent to 'zip bar foo' concept. ;) > and do "make" with a symlink farm (cp > - -as isn't hard), or (better) we can set things up so that only on access > (most likely a read) of the original zipfile do we re-add all the changes. Those chuckleheads who have filled up a disk by saying 'tar cvf foo.tar .' just got a whole new way to fill the disk... ;) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 5:41 ` Valdis.Kletnieks @ 2005-06-27 5:57 ` David Masover 2005-06-27 6:12 ` Valdis.Kletnieks 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-27 5:57 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Mon, 27 Jun 2005 00:31:46 CDT, David Masover said: > > >>*If* we decide that this must go both ways, *then* we either turn off >>write support inside the zipfile > > > Oh, *that* will do wonders for command symmetry. And you just shot down > the whole 'mv foo bar' being equivalent to 'zip bar foo' concept. ;) In one of three possible settings for the imaginary zipfile plugin, yes. But if we're talking about a kernel source tree, how many of us actually build zipfiles/tarballs of their kernel source trees, rather than unpack existing ones? >> and do "make" with a symlink farm (cp >>- -as isn't hard), or (better) we can set things up so that only on access >>(most likely a read) of the original zipfile do we re-add all the changes. > > > Those chuckleheads who have filled up a disk by saying 'tar cvf foo.tar .' just > got a whole new way to fill the disk... ;) Just a new interface :P -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr+VYngHNmZLgCUhAQK/bQ/+MbWZrE+sg3te+YWCajaZhsZLyM4CZrGe BrvUb3dLSKLLg+2CzsKXQp9u3bavJg7OEEmnzgY4dK3FaOHZSATzbwmRgi1hYtfe TfVWKQb4z93zzS6MXafRkgL98qEIUV4n0aFKIEzDgeXAwMYx+Gv5Q7YMky7DEBNq HV5ZiKxhsqCEJdsUKo80G2R12XBm9reunJM/xRt+iA7WfsevBi+/Fl7qbAIDG+sd t83kZHiHQROTEmyyMj4zsJ0j7gRkJsXWKng3HEMl08AkRBIZxQRx0xmzWyZEs+wW AaPPFvsFo04tfwplasvwZ7EcLoCgFP2rfnjx190wov6ZvEWwIbKepsyLqusEuRcR hY0ohrWzYmQkHDXxs4FW+/QMsDF0L7YojHJgHOI1xSeCMOEgGPmHG9Jmf9FeK97Q 4xBXLDGXjlcDBZvDZciLzJCloH3pb+3TrzSkbIkTGnGbgtsS46bWRUAFOY1d2ynM C1FTebc5LPGoDetsu9/cLLuJrCJbbmpN/AWR33WUOnu9/O+H0lWNhiYyOg0x2G/z p6toqr95RPKO6tC5OliU7GfEpqjDhr+RQkjSUbrHqoNjZp3P07nCwGCfwMTCdsBx AkpQyuyfn/RfZvaFOnpUSOTKCWfZKYB2/J8ij+FH8tcPpYl5uzb6H6mHNyEPlvAa DOlFJixKQ7o= =6LFS -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 5:57 ` David Masover @ 2005-06-27 6:12 ` Valdis.Kletnieks 2005-06-27 6:27 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 6:12 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 833 bytes --] On Mon, 27 Jun 2005 00:57:54 CDT, David Masover said: > In one of three possible settings for the imaginary zipfile plugin, yes. > But if we're talking about a kernel source tree, how many of us > actually build zipfiles/tarballs of their kernel source trees, rather > than unpack existing ones? I dunno. I'll often build a tarball of "-mm plus local patches" known to be working at the moment, precisely so I can just untar that as a known good base for the next kernel-hackfest, rather than untar Linus's tree, apply all of the -mm patch, then all my local patches again... And even if I'm not *that* ambitious, I'll at least tar up a clean -mm tree to use as a base. :) And even if I didn't do that, you *do* have to do something when the disk gets backed up. You *do* intend for sensible things to happen then, right? ;) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 6:12 ` Valdis.Kletnieks @ 2005-06-27 6:27 ` David Masover 2005-06-27 6:43 ` Valdis.Kletnieks 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-27 6:27 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Mon, 27 Jun 2005 00:57:54 CDT, David Masover said: > > >>In one of three possible settings for the imaginary zipfile plugin, yes. >> But if we're talking about a kernel source tree, how many of us >>actually build zipfiles/tarballs of their kernel source trees, rather >>than unpack existing ones? > > > I dunno. I'll often build a tarball of "-mm plus local patches" known to > be working at the moment, precisely so I can just untar that as a known good > base for the next kernel-hackfest, rather than untar Linus's tree, apply all > of the -mm patch, then all my local patches again... What you really want is a copy-on-write tree, or a simpe cp. Or are you that short on space that you need compression? Speaking of which, I know copy-on-write files are planned. What about whole trees? But, short of that, you can always do a poor man's copy-on-write with "cp -al" or "cp -as", assuming you remember to copy when you write... > And even if I'm not *that* ambitious, I'll at least tar up a clean -mm tree > to use as a base. :) I just keep a -mm patch around and the original vanilla tarball, but to each his own. > And even if I didn't do that, you *do* have to do something when the disk > gets backed up. You *do* intend for sensible things to happen then, right? ;) I back up with rsync, actually. Speaking of backup, that's another nice place for a plugin. Imagine a dump that didn't have to be of the entire FS, but rather an arbitrary tree... That might be a nice new archive format. I know Apple already uses something like this for their dmg packages. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr+cTXgHNmZLgCUhAQIvvxAAk70+cz4yOZaMX6TDIuUWNsPjqM890FMa jpNE5I2K3ZV91yJFAMdSZW1fQSQakorJyt0DV6wnxu3EbIZV8ATeIkNgJsqcDxAD O6dZoIQnFZND90Fh3HdJb46DRZyml4NbeEKhQRzIAnuANIAa4+6it2aERcpbDNxE ijOiVVpjTkUEutI2uuCJTYSXOGDa+rI0Jmth/9VnPAb5r+wqUb49wUziaSslnZs5 Jt6UlIcXU/OymzhWe+9JM0gydP+V2nP4N14oYoiXaqINPCA9OHV7BudClkKDeCyv bYXyRfjmuiDNoOel6ZfURqUFR2GK/dPqI1PBV6Vc4UMsUqIKkGLAuQ5rU0csb39f eS+6Dp8DJ7tOvQp73x1KTMJWP0lha1VRAj8s3SwdCp0Xar8YHymfCDjHvx/iJe50 rY5aZfWXGCzVmbKVETE8ACWeF5bgpnzwMDwU2RlaWhV/1yIZSOttuJFWMHNG4Rns ajUrRsV9cmmngIJBMcM6XaZD9eRsJ9gb+E9POpsFbo6OwhS0IpRlzu/AGGoEiHSW ZGNBoSDuLDluHP1jiXrnM7ONcFPEe2opUD7ltgSkYPKpya9C7lEfsEBzXh5Gqc5X Iw8P0Md7U6rjCJYGb0pwnJQa0MYMimXq3hOTVsLdoj4A1Nn0EVntHzpB/QTe4UX2 NjMlwpPIXkU= =fqm7 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 6:27 ` David Masover @ 2005-06-27 6:43 ` Valdis.Kletnieks 2005-06-27 7:00 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 6:43 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 503 bytes --] On Mon, 27 Jun 2005 01:27:25 CDT, David Masover said: > I back up with rsync, actually. Doesn't matter what it is. You still need to define sane semantics for it.. ;) > Speaking of backup, that's another nice place for a plugin. Imagine a > dump that didn't have to be of the entire FS, but rather an arbitrary > tree... That might be a nice new archive format. I know Apple already > uses something like this for their dmg packages. Hmm.. you mean like 'tar' or 'cpio' or 'pax' or 'rsync'? :) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 6:43 ` Valdis.Kletnieks @ 2005-06-27 7:00 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-27 7:00 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Mon, 27 Jun 2005 01:27:25 CDT, David Masover said: [...] >>Speaking of backup, that's another nice place for a plugin. Imagine a >>dump that didn't have to be of the entire FS, but rather an arbitrary >>tree... That might be a nice new archive format. I know Apple already >>uses something like this for their dmg packages. > > > Hmm.. you mean like 'tar' or 'cpio' or 'pax' or 'rsync'? :) No, a dmg is an OS X program installer. It appears to be a disk image of sorts. So this is the backup idea in reverse. They even "optimize" (repack? defrag?) the hard drive after each update. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr+kIXgHNmZLgCUhAQKMbg/9GMCtXtlC76wYEvzALVQROXh5xTKHJg8q +sngIMVKfCPOwGVe62tUfrzGaQ0/t9FJ/p40yAzRrWHV7msJWwQZYjf918AA1Vmg IMmsbzFPH+3oRROSfwWcPB/MPG5n1YVfsaq09BUSFM7pasubGgEiFz3TA7gwRTh1 sOw/3J1mlhcxUUbQJrLPAe6e4u59h6MwUZVSdnF2D0Gnnxgwvl8ZemLVSqDaaMGs F7fHJWUd7pZO+d4h2c9AnFLQQL5TvJHDOWuHcGVHZbt/Vaz7F79TWWyC74MeDUA6 ErSGbrZ4fLnocP8zqmDkWJ9GnqE/w9HAhDC2vPtRqCl4z8Mbc/0e5FOxi3M+7WVH TXXBTKnBQG97DWEwneVRFndmhzP/rccnCYIlWhTugiATkStNkOKabTHVp2tpC7lB VP+NwBsMFoaS3vvKcYMF9709OglLRdGmHoju87UIvyLLUZ1fYvZ2+7WlEez0cqSh I0GTCq8gshagIAKhjR/vhVT0i20xyrS/oJEIYdtXz470E2S9tf1NvIwlokO153wL 6PaAN2okmC+YLG+lMeHDZs2lKDfmPzOtbzxR8XyEQHLxsbdWJBnm9B6Epa02iiE1 wZ8feMMsAOjzb/8iiZFqMRROe+j9m+beDLMdc3fYVrNQM72EyPAMw1zUr1GgXeVA USJDqNWK5S4= =BpTW -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-27 7:00 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-27 7:00 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Mon, 27 Jun 2005 01:27:25 CDT, David Masover said: [...] >>Speaking of backup, that's another nice place for a plugin. Imagine a >>dump that didn't have to be of the entire FS, but rather an arbitrary >>tree... That might be a nice new archive format. I know Apple already >>uses something like this for their dmg packages. > > > Hmm.. you mean like 'tar' or 'cpio' or 'pax' or 'rsync'? :) No, a dmg is an OS X program installer. It appears to be a disk image of sorts. So this is the backup idea in reverse. They even "optimize" (repack? defrag?) the hard drive after each update. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr+kIXgHNmZLgCUhAQKMbg/9GMCtXtlC76wYEvzALVQROXh5xTKHJg8q +sngIMVKfCPOwGVe62tUfrzGaQ0/t9FJ/p40yAzRrWHV7msJWwQZYjf918AA1Vmg IMmsbzFPH+3oRROSfwWcPB/MPG5n1YVfsaq09BUSFM7pasubGgEiFz3TA7gwRTh1 sOw/3J1mlhcxUUbQJrLPAe6e4u59h6MwUZVSdnF2D0Gnnxgwvl8ZemLVSqDaaMGs F7fHJWUd7pZO+d4h2c9AnFLQQL5TvJHDOWuHcGVHZbt/Vaz7F79TWWyC74MeDUA6 ErSGbrZ4fLnocP8zqmDkWJ9GnqE/w9HAhDC2vPtRqCl4z8Mbc/0e5FOxi3M+7WVH TXXBTKnBQG97DWEwneVRFndmhzP/rccnCYIlWhTugiATkStNkOKabTHVp2tpC7lB VP+NwBsMFoaS3vvKcYMF9709OglLRdGmHoju87UIvyLLUZ1fYvZ2+7WlEez0cqSh I0GTCq8gshagIAKhjR/vhVT0i20xyrS/oJEIYdtXz470E2S9tf1NvIwlokO153wL 6PaAN2okmC+YLG+lMeHDZs2lKDfmPzOtbzxR8XyEQHLxsbdWJBnm9B6Epa02iiE1 wZ8feMMsAOjzb/8iiZFqMRROe+j9m+beDLMdc3fYVrNQM72EyPAMw1zUr1GgXeVA USJDqNWK5S4= =BpTW -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 7:00 ` David Masover (?) @ 2005-06-27 13:47 ` David Weinehall 2005-06-27 15:08 ` David Masover -1 siblings, 1 reply; 620+ messages in thread From: David Weinehall @ 2005-06-27 13:47 UTC (permalink / raw) To: David Masover Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Mon, Jun 27, 2005 at 02:00:49AM -0500, David Masover wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Valdis.Kletnieks@vt.edu wrote: > > On Mon, 27 Jun 2005 01:27:25 CDT, David Masover said: > > [...] > > >>Speaking of backup, that's another nice place for a plugin. Imagine a > >>dump that didn't have to be of the entire FS, but rather an arbitrary > >>tree... That might be a nice new archive format. I know Apple already > >>uses something like this for their dmg packages. > > > > > > Hmm.. you mean like 'tar' or 'cpio' or 'pax' or 'rsync'? :) > > No, a dmg is an OS X program installer. It appears to be a disk image > of sorts. So this is the backup idea in reverse. Yeah, disk images are really a new invention... It's not like creating an arbitrarily large solid file and then doing mkfs on it would accomplish the same thing =) > They even "optimize" (repack? defrag?) the hard drive after each update. That's what they call the updating of the prelinking information, AFAIK. Regards: David Weinehall -- /) David Weinehall <tao@acc.umu.se> /) Northern lights wander (\ // Maintainer of the v2.0 kernel // Dance across the winter sky // \) http://www.acc.umu.se/~tao/ (/ Full colour fire (/ ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 13:47 ` David Weinehall @ 2005-06-27 15:08 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-27 15:08 UTC (permalink / raw) To: David Weinehall Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 David Weinehall wrote: > On Mon, Jun 27, 2005 at 02:00:49AM -0500, David Masover wrote: > >>-----BEGIN PGP SIGNED MESSAGE----- >>Hash: SHA1 >> >>Valdis.Kletnieks@vt.edu wrote: >> >>>On Mon, 27 Jun 2005 01:27:25 CDT, David Masover said: >> >>[...] >> >> >>>>Speaking of backup, that's another nice place for a plugin. Imagine a >>>>dump that didn't have to be of the entire FS, but rather an arbitrary >>>>tree... That might be a nice new archive format. I know Apple already >>>>uses something like this for their dmg packages. >>> >>> >>>Hmm.. you mean like 'tar' or 'cpio' or 'pax' or 'rsync'? :) >> >>No, a dmg is an OS X program installer. It appears to be a disk image >>of sorts. So this is the backup idea in reverse. > > > Yeah, disk images are really a new invention... It's not like creating > an arbitrarily large solid file and then doing mkfs on it would > accomplish the same thing =) Scroll up a little... "Imagine a dump that didn't have to be of the entire FS, but rather an arbitrary tree..." Such a dump might produce the same thing as mkfs would, but it would do it faster, maybe even copy-on-write. The OS X example is to point out that using some chunk of an FS as an archive format isn't necessarily a bad idea. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsAWa3gHNmZLgCUhAQJnRw//Sh0L38YZ/1fuxjo1TXQtfpMs2YpOIwSm brJtx53rJD8nvAnOP1rwtWc5RiIl+YDzlWIO31L+U45Ap4B9+M4f7elqOlXXyrhw HC/PIu3kai+MZYV05As8IaQNlU7fDPygnYVfIdFzz1NfwwiRo7TPXjjoPlkwaN8w SluYj9CtwTUarNc7Nqct4gmmoxZ20YuyTmpQLyYkU4UMFmGcUfokhg03WUaFPGZr bJL5TjW4YePyFtPcU53JcztjLD2z5pQrj1QdzK5hE2FM0UAM+0mWVEalkR19bNk3 z/lLEkfWBHP3gib9mdC5RsT/aik8nwRdr6X80WtPEMPwqAgSN+7x37u8Jp3paqLc iQu54zVb7h9GJQfoi7sRCfO3GXxRKm3HDB8RnLZD39RKq3duA9M5hN+X9hMYVjd7 KE8kH387QvhLH7TwlfCyIQ2yY6/9abw4Jw9kdoBcX1pq5HpSf1s9TreA/w+89gan W5NoIMo9VPR6Lhx9h6f7O07kxqVNdpjgw6w1LAGmgXKtlT/ojZ3lunXXPsjKknWN M11XYiG6wBRNWJKsWsbh19O9FdcjpNknCxvrSLdiQDD44BTqc7l+Kt6dhACiyigs BBMG3ceowL+vh52spCkakksqCIU4fb6+ndv2g43Q0d2Sk9tpCFUakzEugGfXrHYR 0qNwqhqbe2o= =lP7Z -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 7:00 ` David Masover @ 2005-06-27 16:40 ` Valdis.Kletnieks -1 siblings, 0 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 16:40 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 681 bytes --] On Mon, 27 Jun 2005 02:00:49 CDT, David Masover said: > >>Speaking of backup, that's another nice place for a plugin. Imagine a > >>dump that didn't have to be of the entire FS, but rather an arbitrary > >>tree... That might be a nice new archive format. I know Apple already > >>uses something like this for their dmg packages. > > Hmm.. you mean like 'tar' or 'cpio' or 'pax' or 'rsync'? :) > No, a dmg is an OS X program installer. It appears to be a disk image > of sorts. So this is the backup idea in reverse. I was addressing the ability to deal with an arbitrary tree. By that definition, a dmg, being a disk image and not a tree image, is *not* what you want.... [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-27 16:40 ` Valdis.Kletnieks 0 siblings, 0 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 16:40 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 681 bytes --] On Mon, 27 Jun 2005 02:00:49 CDT, David Masover said: > >>Speaking of backup, that's another nice place for a plugin. Imagine a > >>dump that didn't have to be of the entire FS, but rather an arbitrary > >>tree... That might be a nice new archive format. I know Apple already > >>uses something like this for their dmg packages. > > Hmm.. you mean like 'tar' or 'cpio' or 'pax' or 'rsync'? :) > No, a dmg is an OS X program installer. It appears to be a disk image > of sorts. So this is the backup idea in reverse. I was addressing the ability to deal with an arbitrary tree. By that definition, a dmg, being a disk image and not a tree image, is *not* what you want.... [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 16:40 ` Valdis.Kletnieks @ 2005-06-27 18:25 ` David Masover -1 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-27 18:25 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Mon, 27 Jun 2005 02:00:49 CDT, David Masover said: > > >>>>Speaking of backup, that's another nice place for a plugin. Imagine a >>>>dump that didn't have to be of the entire FS, but rather an arbitrary >>>>tree... That might be a nice new archive format. I know Apple already >>>>uses something like this for their dmg packages. >>> >>>Hmm.. you mean like 'tar' or 'cpio' or 'pax' or 'rsync'? :) >> >>No, a dmg is an OS X program installer. It appears to be a disk image >>of sorts. So this is the backup idea in reverse. > > > I was addressing the ability to deal with an arbitrary tree. By that definition, > a dmg, being a disk image and not a tree image, is *not* what you want.... Not exactly what I want, no. I was just trying to avoid the "people will never adopt a new archive format" argument by pointing out that a similar archive format was recently created and adopted. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsBEingHNmZLgCUhAQLTCw//XdrVgjaxvz9jzbiiSXVY60ItW5aechNV JdvHBf1edVOCHs6OLil/cioK4Pjsfw3pKU+aBLN+eroS0RDomQkrqcy2zmFQ5JW6 oFJQEnb/uA95AhFtdCrbOF3rKABfJNo0TjwsRBuKyiabpYTGjfTJ0gDaQGAtBoOb JmycKjZlJxhQgef9Q3LhG6UQaszCQrKX++pakKYaqOlughIpZ4O2AJTQWEq5ujdu N9GNtO2DGd2sumWdxNpb0KSISZIs6PmPVthPsHwOvD0E6533q9a2AJH49j8AcuOz 1FCU7oFtpm994lwvc4G7eubRMbJ5Xgyy+suqfhTbumOgKzzBw8cKgpxkXFlcyDte 4WyNUlyIqAn0n9GAEVHWSDZ+vqN3E1u+bgLWJq0PEJJSjv9dJzhtH1WPu1bDA2v2 s1qNQDQromF+VfE1QF/fhy1Ttpqh7xAjBmxdxi+g3OU6Vlij7/j0fpMurIELknI0 MRFmw63TzMnFB07Vwi00L7ZR8GKSWDT0/smYF8R1V7stRqrAxHKDlT6E526ESMv+ ALa2pa1lGKWyCSgBMwPC0Cm9FBcOfaqxi60/5KE+bQOQc3Yc+20HCH5BIIQcNgWq axOUO4txg9uz0GxfuvHWaa2tKE1vpbNO0Oy92QqdpCRmssyTNxs7EyhmdUB92Xar SZVTSjpEmv0= =WLzu -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-27 18:25 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-27 18:25 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Mon, 27 Jun 2005 02:00:49 CDT, David Masover said: > > >>>>Speaking of backup, that's another nice place for a plugin. Imagine a >>>>dump that didn't have to be of the entire FS, but rather an arbitrary >>>>tree... That might be a nice new archive format. I know Apple already >>>>uses something like this for their dmg packages. >>> >>>Hmm.. you mean like 'tar' or 'cpio' or 'pax' or 'rsync'? :) >> >>No, a dmg is an OS X program installer. It appears to be a disk image >>of sorts. So this is the backup idea in reverse. > > > I was addressing the ability to deal with an arbitrary tree. By that definition, > a dmg, being a disk image and not a tree image, is *not* what you want.... Not exactly what I want, no. I was just trying to avoid the "people will never adopt a new archive format" argument by pointing out that a similar archive format was recently created and adopted. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQsBEingHNmZLgCUhAQLTCw//XdrVgjaxvz9jzbiiSXVY60ItW5aechNV JdvHBf1edVOCHs6OLil/cioK4Pjsfw3pKU+aBLN+eroS0RDomQkrqcy2zmFQ5JW6 oFJQEnb/uA95AhFtdCrbOF3rKABfJNo0TjwsRBuKyiabpYTGjfTJ0gDaQGAtBoOb JmycKjZlJxhQgef9Q3LhG6UQaszCQrKX++pakKYaqOlughIpZ4O2AJTQWEq5ujdu N9GNtO2DGd2sumWdxNpb0KSISZIs6PmPVthPsHwOvD0E6533q9a2AJH49j8AcuOz 1FCU7oFtpm994lwvc4G7eubRMbJ5Xgyy+suqfhTbumOgKzzBw8cKgpxkXFlcyDte 4WyNUlyIqAn0n9GAEVHWSDZ+vqN3E1u+bgLWJq0PEJJSjv9dJzhtH1WPu1bDA2v2 s1qNQDQromF+VfE1QF/fhy1Ttpqh7xAjBmxdxi+g3OU6Vlij7/j0fpMurIELknI0 MRFmw63TzMnFB07Vwi00L7ZR8GKSWDT0/smYF8R1V7stRqrAxHKDlT6E526ESMv+ ALa2pa1lGKWyCSgBMwPC0Cm9FBcOfaqxi60/5KE+bQOQc3Yc+20HCH5BIIQcNgWq axOUO4txg9uz0GxfuvHWaa2tKE1vpbNO0Oy92QqdpCRmssyTNxs7EyhmdUB92Xar SZVTSjpEmv0= =WLzu -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 18:25 ` David Masover @ 2005-06-28 6:47 ` Valdis.Kletnieks -1 siblings, 0 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-28 6:47 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 815 bytes --] On Mon, 27 Jun 2005 13:25:14 CDT, David Masover said: > I was just trying to avoid the "people will never adopt a new archive > format" argument by pointing out that a similar archive format was > recently created and adopted. Out of curiosity, adopted by popular acclaim, or because an 800 pound gorilla said "This is the format we're shipping <XYZ> in, learn to deal with it"? (I've seen both happen multiple times in the last quarter century, on many different operating systems....) (For that matter, all of my production boxes are backed up by either Tivoli or Legato, and I haven't a *clue* what format those tapes are in. As a practical matter, it doesn't really matter - after the first quarter petabyte or so of backed up data, you're not going to do a restore without the software's help anyhow.. ;) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-28 6:47 ` Valdis.Kletnieks 0 siblings, 0 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-28 6:47 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 815 bytes --] On Mon, 27 Jun 2005 13:25:14 CDT, David Masover said: > I was just trying to avoid the "people will never adopt a new archive > format" argument by pointing out that a similar archive format was > recently created and adopted. Out of curiosity, adopted by popular acclaim, or because an 800 pound gorilla said "This is the format we're shipping <XYZ> in, learn to deal with it"? (I've seen both happen multiple times in the last quarter century, on many different operating systems....) (For that matter, all of my production boxes are backed up by either Tivoli or Legato, and I haven't a *clue* what format those tapes are in. As a practical matter, it doesn't really matter - after the first quarter petabyte or so of backed up data, you're not going to do a restore without the software's help anyhow.. ;) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 2:37 ` David Masover 2005-06-27 3:10 ` Kyle Moffett 2005-06-27 4:23 ` Valdis.Kletnieks @ 2005-06-27 4:27 ` Valdis.Kletnieks 2005-06-27 4:51 ` David Masover 2005-06-27 5:09 ` Hans Reiser 2 siblings, 2 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 4:27 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 870 bytes --] On Sun, 26 Jun 2005 21:37:48 CDT, David Masover said: > > Go read http://www.tux.org/lkml/#s7-7 and ponder until enlightenment arrives. > > So what? I don't intend to convince anyone based on how much > slower/faster their kernel compiles. It's meant to illustrate the > principle of the thing. No, you seemed convinced that you'd have a big win based on the fact that big chunks don't get unpacked - when in fact it's not as much of a win as you might think. And at least in the real world, performance *does* matter - if doing it the traditional way is 3 times faster, nobody's going to be interested. > Besides, your point was that you could not run make inside of a kernel > tarball/zipfile. Nobody ever suggested that you would actually want to. "Here's a new facility. Don't bother trying to actually use it". Is that the message you're trying to send? [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 4:27 ` Valdis.Kletnieks @ 2005-06-27 4:51 ` David Masover 2005-06-27 5:09 ` Hans Reiser 1 sibling, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-27 4:51 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Sun, 26 Jun 2005 21:37:48 CDT, David Masover said: > > >>>Go read http://www.tux.org/lkml/#s7-7 and ponder until enlightenment arrives. >> >>So what? I don't intend to convince anyone based on how much >>slower/faster their kernel compiles. It's meant to illustrate the >>principle of the thing. > > > No, you seemed convinced that you'd have a big win based on the fact that > big chunks don't get unpacked - when in fact it's not as much of a win as > you might think. Not in this case. > And at least in the real world, performance *does* matter - if doing it the > traditional way is 3 times faster, nobody's going to be interested. Not true. I've said it before, and I'll say it again: http://www.apple.com/macosx/features/spotlight/ Users *are* willing to trade speed for features, under the right circumstances. Tiger *is* noticeably slower. People *do* make webservers out of it anyway. Or, for an extreme example: http://www.doom3.com/ Doom 3 can be made to run on much older hardware (the Voodoo3, I think), by disabling things like the dynamic lighting. The old way of doing lighting is probably more than 3 times faster, I know it's more than twice as fast. The new way looks much, much better, and many people upgraded their hardware -- at least a couple hundred dollars upgrade, sometimes an entirely new box -- in order to play this game. For that matter, the old way of doing this kind of discussion is email, the new way seems to be a PHP Wiki, which has got to be more of a load on the server, but people use it anyway. In the real world, killer features trump performance. All of this is irrelevent. The performance isn't that bad. It may even be better, even if it's not *much* better. >>Besides, your point was that you could not run make inside of a kernel >>tarball/zipfile. Nobody ever suggested that you would actually want to. > > > "Here's a new facility. Don't bother trying to actually use it". Not the facility. The specific example. > Is that the message you're trying to send? I'm lacking a little imagination right now. It's Sunday night, after all. Actually, I think it might be a big win for Debian-like package management systems. That's also somewhere in the "Silent Semantic" archives. Here's another, somewhat better example: modifying a zisofs CD image. I'm going to go reply to your other mail now; I think the details belong there. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr+F3XgHNmZLgCUhAQJWDA/+NmbL7kCANpEEeV+4zKx3MUmABuoRRo6Y hbRPDlBk9PCPM8GWQx+XwRh4XXb17FQklP7oIJHnnGtQelwDtcvbX8bSsDUTPd3s Gt8a11GMAJkGcPAHh2oBtlGjslJj7O/eJRlj3eeYv5LIRxq/KEfvBt0wvx2W73uN d8Mm65baeq7ipWDqKf0yQTcJT8M8kQQczO5gb9E5Q68fL+8rO7diQEOCm2A+Sp2G rlKpvcQuhQ/P7IBOnX8ABUG4UphQfMqtICIwMmkLimLfjO+ID9pnO516K7O4erMI Q2deFh6+KQ62Ngmokkh6GOu7VEmUD4XZubJpJknGWWBaKdCz12LsvHt2DeucKD5e 5mWi4oVwSInlxIRboso0CmRkoJZdEmqBJWFPtjXEtGGdufqJFySIHxPXpsqwRsa4 SuIIa2SK5vDXoc9URqKtPZtGkrGjG4VL1fxLNp9hD91hp+T5bLhsO+/8v2ZqfyRC UvTKb9u3mqdu//UxJhhj0LTTjNv3eNc+2UQFhAg7uSpa7FZjV+nEvD5ZwlpimQL1 InDt+nd5wQWWQ+SfjbZNvdvE/XGfmMArT/Ik7tYv9LkSLp0ohwOVQw3fThPGY+BD 1SW0E5FLruAMh/cRl0mwIhDQ9CZO/jrKjRbueyC7CYFMjmyBLUxbt0vUIJqsB8rJ P58IcgUaWFs= =R57s -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 4:27 ` Valdis.Kletnieks 2005-06-27 4:51 ` David Masover @ 2005-06-27 5:09 ` Hans Reiser 2005-06-27 6:02 ` David Masover 1 sibling, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-27 5:09 UTC (permalink / raw) To: Valdis.Kletnieks Cc: David Masover, Lincoln Dale, Gregory Maxwell, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Valdis.Kletnieks@vt.edu wrote: >>tarball/zipfile. Nobody ever suggested that you would actually want to. >> >> > Besides, your point was that you could not run make inside of a kernel > Umm, try it when we have it working, on a 1-4GB RAM machine it might not be so bad..... We have the compression (albeit still with bugs) but not the tar plugin. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 5:09 ` Hans Reiser @ 2005-06-27 6:02 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-27 6:02 UTC (permalink / raw) To: Hans Reiser Cc: Valdis.Kletnieks, Lincoln Dale, Gregory Maxwell, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hans Reiser wrote: > Valdis.Kletnieks@vt.edu wrote: > > >>>tarball/zipfile. Nobody ever suggested that you would actually want to. >>> >>> >> >>Besides, your point was that you could not run make inside of a kernel >> > > Umm, try it when we have it working, on a 1-4GB RAM machine it might not > be so bad..... We have the compression (albeit still with bugs) but not > the tar plugin. We *desperately* need either more plugin-like "plugins" or a FUSE-like way of writing plugins. I mean, tar is small enough, but I don't want you chipping away at my (soon-to-be) 2 gigs of desktop RAM -- I need that for Doom 3! Or, do you mean using the RAM for keeping the working copy of the zipfile? And here I was trying to establish that it was better to cache this. If the cache appears as a separate Reiser tree, it's not much different than keeping it in RAM anyway, but it can be flushed if I need the space for... work... ;) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr+WdngHNmZLgCUhAQLYrw//V8z1ywRTyXk7zaHU+COPO2PCV8kKXtWJ 7BkDXm65cVCUVNN6hgiJxMGEx7fe8ZWNJRHhvdIWGt5G68qI6G0F/tYQajtT59mk E7AF2WWJ2ip0E0WKzBVVYKYnaeB+w9rF/8RJ/0Zj7C3zQ+fzvmVCGw6y9bLWYQle VCUYjSMKaQz1cU4RqtDd+pIgr9w5/kxfhd30Aj9nK2qaQZLSxeFTv7p9c2vW5ZwM YeQG8K4EZ0gO1GH0e85BAa9McLnDurosxeeLumaPqbIbFzc+1YXy9+5PsXjPKGDN IjKVm/R0HpPP9KpCLdpRvp3GyWz1EuYSp9zpUEh5mDPH/gOSgRCva3Vx/2TW+YTk 3U73tHbTI5b9Mf/dlK3GPAoJ1krRm83sNMMAyGaTv+n6kRVWaSWIelZrVEJ5/jHO pgY+5mDJwROlXdDqO1CV9PJHx4DivyODOrKzFnIykfQ2MO1vs1+Fy94LyYGkrPGd Nq6y6XGxjx5MZ1fX3vJfovX0z+eR9s3R1uIgCMXlVoELEh9XKoFGGWIhrb7MhUdK Q+n2ljixMZhIakeEUflhycZw0CY8jlgTK+vULUS7mDhOExVxombdAalVhhQ3cK0+ ZasRPyS3A3mCE7U6AKt8oc1AjRpdNIR1YKmfh+C44Yy+2e2wIWAoIb4mg0uY6jEa GJl0uZIj4rY= =uAm0 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 0:40 ` Valdis.Kletnieks 2005-06-27 2:37 ` David Masover @ 2005-06-27 2:49 ` David Masover 2005-06-27 3:10 ` Hubert Chan 2 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-27 2:49 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Sun, 26 Jun 2005 17:35:48 CDT, David Masover said: > > >>>Right. So please explain what crypto/raw/foo and crypto/inflated/foo.gz give you. >> >>In that example (shouldn't have used the name "crypto", but oh well), it >>should be crypto/raw/foo.gz and crypto/inflated/foo -- where foo.gz is >>the gzip'ed file and foo is the transparently compressed/decompressed >>file. Basically, these are equivalent: >> >>$ zcat crypto/raw/foo.gz >>$ cat crypto/inflated/foo > > > I'm *quite* aware of what your preconceived notions think it *should* be. So, by "what would ... give you", you meant what the benefits are? I was just clarifying how they work, I *thought* that's what you meant... > Maybe the two examples I asked for have *real-world* meanings that you should > be allowing for. Like, for instance, on a mail server, where the A/V software > may need to unzip a file 5 or 6 times to find out if there's malicious content. Why would it want to unzip the *same* file that many times? If you're talking about nested zipfiles, we can do nested plugins just as easily. > Or seeing if it's a ".zip bomb", where a small .zip will decompress to gigabytes. > Or I'm testing a new compression algorithm, to see if multiple compressions help > (yes, I know that it *shouldn't* help - but I've seen real-world cases where the > algorithm could only look at a 4K or 8K window at a time, and if you hit a *very* > long run of duplicate 4K segments, a second compression would compress all the > identical or near-identical "this is a 4K chunk" tokens...) Tune the plugin, or use zip directly. I'm not proposing necessarily to embed zip in the kernel and drop it from userland, just to have a kernel interface so that apps/people don't have to all know about the zip program and how to use it. Besides, in the zip example, I made very sure that unless you point a program specifically at where the zip plugin unpacks stuff, you can easily get the zipfile. Of course, that could be turned on its head if it was more low-level transparent compression, the kind where you just compress everything in /etc because it's all tiny text files. In that case, you would want a setup where you have to work to get the raw data. But in that case, it's a different kind of plugin anyway -- a plugin for storing data, rather than just accessing it. Stop worrying about automagic stuff, though. It won't trigger unless you go looking for it. It will just be easier to find if you do. >>I guess I need a new name for this approach. That's three possible ways >>of doing this? > > > I *said* you need to think this through in detail, didn't I? ;) I said we already did, which is true. I don't think I'm making these up, I just don't remember where they were in Silent Semantic. (oops, I fired the last one off before I was done...) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr9pUXgHNmZLgCUhAQKQ8w/+OQXWdt1pEF6t/0D+x3mPYWq2lhlUbchZ BYywcGN6Z/yFZQrZovJABZVnB+CXlUlx8DGqQN+Lj9+8HLko/P75ErTWHfuiscpS wRJIN5dVeZ4f1RImEa8PjQf3c+n+B/ft9hq28yaR58C9vBQwAWOPVL0/4n4unNAV 49odg/IKJM9sdSF+6sVwgWNfuacRcZ5/jXtkDFXIIyJzKl6r45r3HmTkbgRdxqpU p1aOIsXa7fciN+UK+eiw5jruzJsKHaJtBdISsMWbPdVBFjsDVZmhsdOMv2RZMPo6 2V78OQ4g7pJHSMZsaWQ/vvD3PuHxZm9qglJcdGnL6jNk8OXkKzxXaGDn/SHjYFTu c8keR1KYu+U5r5RmTiihavFPpN3zefS5W5o5IyLvEZkApRCngDFitq4t2tRfDbfT zV1JZWz7/bqtoY0zdaZ3gFwxXxh8FMw/LCnsjKkBQ7etlvnvcW2IPssd9rTIV0+4 9CmA2EaYIiXAwGoFzLbbPoKf0a+6tB38oHanBrRBZuTzu9mKHZvyefQa0j2+1KOp iiCmrK49/APXfp4IUu284r2dPUqAu33LtomxwFbNLaPDZFWgQ9qPjafJBO3L8Y/9 HlFe46Jo5tL7YAEq/UyKpDYWxVcUiDZ40z2w/WKq+v9JjmUZMm+jonXHa9Nq0cQn YlE9bxtEac8= =5eSh -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 0:40 ` Valdis.Kletnieks 2005-06-27 2:37 ` David Masover 2005-06-27 2:49 ` David Masover @ 2005-06-27 3:10 ` Hubert Chan 2005-06-27 4:59 ` Valdis.Kletnieks 2 siblings, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-06-27 3:10 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Sun, 26 Jun 2005 20:40:29 -0400, Valdis.Kletnieks@vt.edu said: > On Sun, 26 Jun 2005 17:35:48 CDT, David Masover said: >> The kernel does not decide that. You do. And it doesn't >> automatically decide that every time you create a file. You have to >> use some interface to trigger the plugins. > Oh, I'm waiting for the fun the first time somebody deploys a plugin > that has similar semantics to 'chmod g+s dirname/' ;) Reiser4 plugins have to be compiled into the kernel. (They're not plugins in the sense that most people use that word.) And any admin who would compile that kind of plugin into the kernel needs to have his head examined. Not to mention that plugins must first go through Hans and/or Linus before they can get included into the kernel. The kernel defines the set of plugins available to the user. The user selects (to a certain degree) which plugins to use. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 3:10 ` Hubert Chan @ 2005-06-27 4:59 ` Valdis.Kletnieks 2005-06-27 5:54 ` David Masover 2005-06-27 14:58 ` Hubert Chan 0 siblings, 2 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 4:59 UTC (permalink / raw) To: Hubert Chan Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 3125 bytes --] On Sun, 26 Jun 2005 23:10:43 EDT, Hubert Chan said: > On Sun, 26 Jun 2005 20:40:29 -0400, Valdis.Kletnieks@vt.edu said: > > Oh, I'm waiting for the fun the first time somebody deploys a plugin > > that has similar semantics to 'chmod g+s dirname/' ;) (You *did* notice it was set-GID of a *directory* not an executable file, right?) > Reiser4 plugins have to be compiled into the kernel. (They're not > plugins in the sense that most people use that word.) And any admin who > would compile that kind of plugin into the kernel needs to have his head Oh? You saying that it *wont* be permitted for a user to say: mkdir $HOME/zipped chattr "files under here are ZIP files" $HOME/zipped and instead you have to do that chattr by hand for *every* *single* zip file? Or "files on this filesystem are encrypted by default"? I suspect that this sort of thing is going to be one of the *first* things that will get created, and any admin who tries to sell this idea to the users *without* that sort of functionality will be handed their head. Or, if "that type of plugin.. needs to have their head examimed", I suggest that you go to your kernel source tree, find fs/ext3/ialloc.c, and this code in ext3_new_inode(): if (test_opt (sb, GRPID)) inode->i_gid = dir->i_gid; else if (dir->i_mode & S_ISGID) { inode->i_gid = dir->i_gid; if (S_ISDIR(mode)) mode |= S_ISGID; } else inode->i_gid = current->fsgid; and #ifdef out all but the last line, and see if anything breaks. ;) > examined. Not to mention that plugins must first go through Hans and/or > Linus before they can get included into the kernel. > > The kernel defines the set of plugins available to the user. The user > selects (to a certain degree) which plugins to use. The point you missed was that plugins *will* have interactions, and as the guys who are working on a stacker for LSM modules have found out the hard way, trying to deal with the composition of functions is fiendishly difficult. And notice that it doesn't *have* to be quite so obvious - how about if a user creates a directory $HOME/zipped/ and flags it as "anything under here is a zipped file". Now throw in multiple users and CPU limits. User A enters that directory and references everything, causing the buffer cache to get filled up. While there, A makes changes, so the pages are dirty - "for i in */*; do echo " " >> $i; done" would do the job... User B now does something that causes a writeback of one of those buffer cache pages. A) What process currently gets ticked for the CPU and I/O for the writeback? B) In your model, who will get ticked for the resources? C) Will the users riot? (Note that you can't win here - currently, the "price" of writing back A's and B's pages are about equal. However, if A gets dinked for an expensive writeback due to B's process, A will get miffed. If B gets charge for an expensive writeback of A's, B will get miffed. If you say "screw it" and bill it to a kernel thread, the bean counters will get miffed... ;) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 4:59 ` Valdis.Kletnieks @ 2005-06-27 5:54 ` David Masover 2005-06-27 6:24 ` Valdis.Kletnieks 2005-06-27 14:58 ` Hubert Chan 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-27 5:54 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Hubert Chan, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Sun, 26 Jun 2005 23:10:43 EDT, Hubert Chan said: > >>On Sun, 26 Jun 2005 20:40:29 -0400, Valdis.Kletnieks@vt.edu said: > > >>Reiser4 plugins have to be compiled into the kernel. (They're not >>plugins in the sense that most people use that word.) And any admin who >>would compile that kind of plugin into the kernel needs to have his head > > > Oh? You saying that it *wont* be permitted for a user to say: > > mkdir $HOME/zipped > chattr "files under here are ZIP files" $HOME/zipped > > and instead you have to do that chattr by hand for *every* *single* zip file? chown works this way. At least for username, there's no default user possible per directory. But scroll down a bit... > Or "files on this filesystem are encrypted by default"? > > I suspect that this sort of thing is going to be one of the *first* things > that will get created, and any admin who tries to sell this idea to the users > *without* that sort of functionality will be handed their head. I think it may be possible to set defaults to a certain extent, but I think that it's still actually a per-file setting whether the file is encrypted. Obviously, the keys have to be shared somehow, so there's probably more grouping than just defaults... There has been some mention of inheritance, but I've forgotten how that's supposed to work. If there's some sort of inheritance where children inherit properties of their parent directory, and also inherit changes to those properties, than Hans probably wants that to be the prefered way of doing things? I don't think there's that kind of inheritance, though. > And notice that it doesn't *have* to be quite so obvious - how about if a > user creates a directory $HOME/zipped/ and flags it as "anything under here > is a zipped file". > > Now throw in multiple users and CPU limits. User A enters that directory and > references everything, causing the buffer cache to get filled up. While there, > A makes changes, so the pages are dirty - "for i in */*; do echo " " >> $i; done" > would do the job... User B now does something that causes a writeback of one > of those buffer cache pages. > > A) What process currently gets ticked for the CPU and I/O for the writeback? > > B) In your model, who will get ticked for the resources? > > C) Will the users riot? (Note that you can't win here - currently, the "price" > of writing back A's and B's pages are about equal. However, if A gets dinked > for an expensive writeback due to B's process, A will get miffed. If B gets > charge for an expensive writeback of A's, B will get miffed. If you say "screw it" > and bill it to a kernel thread, the bean counters will get miffed... ;) If I understand this correctly, this is somewhat like if user A creates a 50 meg file on a system with 100 megs free RAM, and user B runs "sync". Also similar to if B were to suddenly fill up 75 megs of RAM, forcing A's file to be flushed -- last I checked, in Reiser4, only a sync or memory pressure causes writes to flush. Right? This is tempting to comment on, but I want to make sure I grok it first... -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr+UiHgHNmZLgCUhAQK+KBAAm6BSQuoFNeY5+cPQQaVK2BACqwJZabMH Ze440Hjf9Pgn6Is/xWbCKKv6Mx4Vfx5P4+E4dkZJOyFBaVym6v5wy7ovPhFZVD8f oW8IOUnCngnQ/Ea7fhxwr+hst2L4jEbMF0deG3zg328zYKwHoY0NA7hQZg2xLhF6 +J5jQWNR+CWyhFCBMD/NG+XwtSd4pxzOjb42e7zlEuKoCgGiTB92qPGDcOYMw/Te 3ez1Z+iJ6gIMUgwrzO4J6TzsgR9d9W0rJKMpm6ulto0AQtg1Joln3Vj5pxBX6ULe 5hkV6zeNOW8Uisz6+tdUX6PC+hjfSPvJhzkPMccTm8cJGyiF+j+PI5nUj37hyAz6 iL8kBPMrsulrGphuJzeb81yLzmgknX4+tc6WrVqMPcCpP4iIYOi0RMpWxSxQuRH9 ooSUnN/JRuhHaz4JeJ6VOvkwwl4nw1dcChBnBiws4IlFQS+mgKTAzZHhHm/+/E6q nHY/uiFo1Tr95wyxrWyNcdGA/axriSIAaCXc1bEpQpzpOOcXr5d437TwEhBTIeXr UlXwM6CdSrA3XGy7ksHovRghdm/7MOmNxL4SagVvY98Dlc+UlWi755rpin7JRget 9tFGtH0IESmPwSXfsBJYNIZCk4bG8X9d2W1/6e96yVtvKdn16MgX6n3Hdt3aVgzv Ec9kEA42Ixs= =qALY -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 5:54 ` David Masover @ 2005-06-27 6:24 ` Valdis.Kletnieks 2005-06-27 7:07 ` David Masover 2005-06-27 7:24 ` Artem B. Bityuckiy 0 siblings, 2 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 6:24 UTC (permalink / raw) To: David Masover Cc: Hubert Chan, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 2675 bytes --] On Mon, 27 Jun 2005 00:54:17 CDT, David Masover said: > There has been some mention of inheritance, but I've forgotten how > that's supposed to work. If there's some sort of inheritance where > children inherit properties of their parent directory, and also inherit > changes to those properties, than Hans probably wants that to be the > prefered way of doing things? Well, the 'chmod g+s dirname/' example *is* just "children inherit the group of the directory", and somebody didn't like that.. ;) > > Now throw in multiple users and CPU limits. User A enters that directory and > > references everything, causing the buffer cache to get filled up. While there, > > A makes changes, so the pages are dirty - "for i in */*; do echo " " >> $i; done" > > would do the job... User B now does something that causes a writeback of one > > of those buffer cache pages. > > > > A) What process currently gets ticked for the CPU and I/O for the writeback? > > > > B) In your model, who will get ticked for the resources? > > > > C) Will the users riot? (Note that you can't win here - currently, the "price" > > of writing back A's and B's pages are about equal. However, if A gets dinked > > for an expensive writeback due to B's process, A will get miffed. If B gets > > charge for an expensive writeback of A's, B will get miffed. If you say "screw it" > > and bill it to a kernel thread, the bean counters will get miffed... ;) > > If I understand this correctly, this is somewhat like if user A creates > a 50 meg file on a system with 100 megs free RAM, and user B runs > "sync". Also similar to if B were to suddenly fill up 75 megs of RAM, > forcing A's file to be flushed -- last I checked, in Reiser4, only a > sync or memory pressure causes writes to flush. Exactly the same sort of thing - traditionally it's been more or less ignored in the system accounting, because A would usually average out to causing as many I/Os as B did, and they were roughly equal in cost so it was a wash. However, if one user has a much higher per-page cost than the other, the imbalance can start to matter *very* quickly.... > Right? This is tempting to comment on, but I want to make sure I grok > it first... For more fun, consider how you can write 1 megabyte of data to a file, lseek to the beginning and start writing again - and you go over quota on the *second* write even though you're over-writing already existing data. Can happen if you're compressing, and the second write doesn't compress as well as the first. (To be fair, we already have similar issues with sparse files - but at least 'tar --sparse' has an easy way to deal with it compared to this. ;) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 6:24 ` Valdis.Kletnieks @ 2005-06-27 7:07 ` David Masover 2005-06-27 16:53 ` Valdis.Kletnieks 2005-06-27 7:24 ` Artem B. Bityuckiy 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-27 7:07 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Hubert Chan, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Mon, 27 Jun 2005 00:54:17 CDT, David Masover said: > > >>There has been some mention of inheritance, but I've forgotten how >>that's supposed to work. If there's some sort of inheritance where >>children inherit properties of their parent directory, and also inherit >>changes to those properties, than Hans probably wants that to be the >>prefered way of doing things? > > > Well, the 'chmod g+s dirname/' example *is* just "children inherit the > group of the directory", and somebody didn't like that.. ;) Respectfully ignoring this until he can comment. I suspect he made a mistake, but I'm trying to avoid putting words in his mouth... >>>Now throw in multiple users and CPU limits. User A enters that directory and >>>references everything, causing the buffer cache to get filled up. While there, >>>A makes changes, so the pages are dirty - "for i in */*; do echo " " >> $i; done" >>>would do the job... User B now does something that causes a writeback of one >>>of those buffer cache pages. >>> >>>A) What process currently gets ticked for the CPU and I/O for the writeback? >>> >>>B) In your model, who will get ticked for the resources? >>> >>>C) Will the users riot? (Note that you can't win here - currently, the "price" >>>of writing back A's and B's pages are about equal. However, if A gets dinked >>>for an expensive writeback due to B's process, A will get miffed. If B gets >>>charge for an expensive writeback of A's, B will get miffed. If you say "screw it" >>>and bill it to a kernel thread, the bean counters will get miffed... ;) >> >>If I understand this correctly, this is somewhat like if user A creates >>a 50 meg file on a system with 100 megs free RAM, and user B runs >>"sync". Also similar to if B were to suddenly fill up 75 megs of RAM, >>forcing A's file to be flushed -- last I checked, in Reiser4, only a >>sync or memory pressure causes writes to flush. > > > Exactly the same sort of thing - traditionally it's been more or less ignored > in the system accounting, because A would usually average out to causing as > many I/Os as B did, and they were roughly equal in cost so it was a wash. Even if A is doing A/V work and B is programming? > However, if one user has a much higher per-page cost than the other, the > imbalance can start to matter *very* quickly.... I see. My instinct is to charge A if B just caused a sync, but change B if B was automagically forcing A to do something... But I still don't understand the layers involved well enough. >>Right? This is tempting to comment on, but I want to make sure I grok >>it first... > > > For more fun, consider how you can write 1 megabyte of data to a file, > lseek to the beginning and start writing again - and you go over quota > on the *second* write even though you're over-writing already existing > data. Can happen if you're compressing, and the second write doesn't > compress as well as the first. (To be fair, we already have similar > issues with sparse files - but at least 'tar --sparse' has an easy way > to deal with it compared to this. ;) To be even more fair, but possibly launch another flame, Reiser4 reserves 5% of disk space by default to avoid this kind of thing. But that's talking about total disk, not quotas. Also doesn't answer my question, but it seems I'm starting to get it. How do we get over quota errors, btw? Can we get them from write() calls? If so, I don't see a Problem(TM), just an annoyance. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr+lwngHNmZLgCUhAQKYUw/9HmIZRqL7+1qcji/y4EhgUMftbdaKHUaJ D+gyssdIgkfNwAR5vEvE6Lb0Fdj4zWU12nnx/ppL0nU1n6pWOBqyOMMtbHs4dbwe iTJmoe1OCBM5uOS8bSdYUfVFUWNRF5+MPc8cWcpBa1LPCBmpzUnr5hS8iuNknWl2 7r6Hs64GSJ1JI7djE0P8fZVTou95azc/1pRU4kRtlsavXMmmAL3sefA9HS4b85VB haOuNPmDIhJhPXhAMY6ZYKq3xnVyGWuiU6Z1Clv8JyP0Y7jaqGjF+V16zVSKvJH8 d0BrN05zGL1CIYDQrLtC0CQHuuLek34NDYimgVSLhkSB73GJ/MHxfO++68Pnhxhz NOF86tPhoDQHhdEkUw0Kq5ZeI43ETejbMRIMoPAhuWo1vCQNFsfsSLljWLatBGNr Lzgb0Q74Iqnrw9fPwrQttA/aPdlnVmOGv1qVYz/LQ8C+UHPVLXkjm5NVzHFDesPS XXRZxSxPcOcMK9zPxLNer3zPlGPNOjQbylfL6+RwcJDmHq3IrDJDLY4TB4cZzJpc gJyXrpSjprYCbSi+cc1d1R4cLcbdMHjR/rEszibLyeMuINYyx6vJklKy9xvpmyEB woGrADFDnmKyv4WOzswSgIM6s8U2dLY4GFM/GW7KLx+rMF98UDhGyLjvSyvIpigs BJz4T03qCtA= =GoJz -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 7:07 ` David Masover @ 2005-06-27 16:53 ` Valdis.Kletnieks 0 siblings, 0 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 16:53 UTC (permalink / raw) To: David Masover Cc: Hubert Chan, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1471 bytes --] On Mon, 27 Jun 2005 02:07:46 CDT, David Masover said: > > Exactly the same sort of thing - traditionally it's been more or less ignored > > in the system accounting, because A would usually average out to causing as > > many I/Os as B did, and they were roughly equal in cost so it was a wash. > > Even if A is doing A/V work and B is programming? I said "traditionally" - it's been a "oh well, we can't do much about it" problem for a *long* time (for instance, time spent in an interrupt handler has usually been charged off against whoever's timeslide the interrupt handler took a chunk out of). It's only been tolerated so far because (a) the costs for both users are about equal and (b) you rarely have a heavy I/O DB and a number cruncher on the same box, or a user doing A/V work and a user doing programming - if it's not a single-use machine, there's *multiple* number crunchers, DBs, or programmers, and they tend to balance out. Said tendency can dissapear quite easily here.... > How do we get over quota errors, btw? Can we get them from write() > calls? If so, I don't see a Problem(TM), just an annoyance. One gotcha here is that it means that you can't do delayed allocation on writes - you *have* to allocate disk space at each write and then update the quotas. (And yes, I know that 'man 2 close' says that bad stuff can happen to your data even after your program exits - that doesn't mean we should go out of our way to make things worse.. ;) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 6:24 ` Valdis.Kletnieks 2005-06-27 7:07 ` David Masover @ 2005-06-27 7:24 ` Artem B. Bityuckiy 1 sibling, 0 replies; 620+ messages in thread From: Artem B. Bityuckiy @ 2005-06-27 7:24 UTC (permalink / raw) To: Valdis.Kletnieks Cc: David Masover, Hubert Chan, Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Valdis.Kletnieks@vt.edu wrote: > For more fun, consider how you can write 1 megabyte of data to a file, > lseek to the beginning and start writing again - and you go over quota > on the *second* write even though you're over-writing already existing > data. Can happen if you're compressing, and the second write doesn't > compress as well as the first. (To be fair, we already have similar > issues with sparse files - but at least 'tar --sparse' has an easy way > to deal with it compared to this. ;) Sorry, may be I'm out of the context, but here is my view. In case of compression in the kernel space you may take into account the size of file in the _uncompressed_ form and how much it takes in the compressed form - doesn't matter. So, no problems with rewriting. Moreover, are you sure that the current quota model is enough for FSes with on-fligh compression? The model should probably be extended/changed. Problems with quota and accounting is not the reason to forbid the on-flight compression. And they don't have to co-exist well. -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-27 4:59 ` Valdis.Kletnieks 2005-06-27 5:54 ` David Masover @ 2005-06-27 14:58 ` Hubert Chan 1 sibling, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-27 14:58 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Mon, 27 Jun 2005 00:59:39 -0400, Valdis.Kletnieks@vt.edu said: > On Sun, 26 Jun 2005 23:10:43 EDT, Hubert Chan said: >> On Sun, 26 Jun 2005 20:40:29 -0400, Valdis.Kletnieks@vt.edu said: >> Oh, I'm waiting for the fun the first time somebody deploys a >> plugin that has similar semantics to 'chmod g+s dirname/' ;) > (You *did* notice it was set-GID of a *directory* not an executable > file, right?) >> Reiser4 plugins have to be compiled into the kernel. (They're not >> plugins in the sense that most people use that word.) And any admin >> who would compile that kind of plugin into the kernel needs to have >> his head > Oh? You saying that it *wont* be permitted for a user to say: [...] Sorry. I completely misunderstood what you were getting at. I thought that you were implying that with the right kind of plugin, users would be able to get around security measures. Of course, this is probably true, which means that plugins need to be coded carefully (as with everything else in the kernel). My point that I was trying to make was that users can't install arbitrary plugins, and the set of plugins available is controlled by the administrator. Since that wasn't what you were talking about, I retract my previous email. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 21:05 ` Valdis.Kletnieks 2005-06-26 22:35 ` David Masover @ 2005-06-26 22:54 ` Hans Reiser 2005-06-27 0:59 ` Valdis.Kletnieks 1 sibling, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-26 22:54 UTC (permalink / raw) To: Valdis.Kletnieks Cc: David Masover, Lincoln Dale, Gregory Maxwell, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Valdis.Kletnieks@vt.edu wrote: > (Hint - work out how long a kernel 'make' would take >if you were doing it inside a .tar.bz2). > > After the first time, not very long, if you had enough ram.... the plugin would keep the data uncompressed until it flushed it to disk. Performance might even improve since less would be written to disk. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 22:54 ` Hans Reiser @ 2005-06-27 0:59 ` Valdis.Kletnieks 0 siblings, 0 replies; 620+ messages in thread From: Valdis.Kletnieks @ 2005-06-27 0:59 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Lincoln Dale, Gregory Maxwell, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1007 bytes --] On Sun, 26 Jun 2005 15:54:25 PDT, Hans Reiser said: > Valdis.Kletnieks@vt.edu wrote: > > > (Hint - work out how long a kernel 'make' would take > >if you were doing it inside a .tar.bz2). > > > > > After the first time, not very long, if you had enough ram.... the > plugin would keep the data uncompressed until it flushed it to disk. You're not allowed to use current existing stuff like the disk buffer cache to weasel your way out on this one. "if you had enough ram" has been true for decades. The trouble is that quite often you *don't* have enough ram.... > Performance might even improve since less would be written to disk. I've worked with filesystems where performance improves due to compression (AIX's JFS). It's a lot harder to provide an improvement if you're writing 37 more bytes in between bytes 399457 and 399458.... (I suppose by aligning byte 399458 so it actually is on the start of a 4K block you can do that, but then you're losing the advantages of the compression.. ;) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 7:48 ` David Masover 2005-06-26 8:26 ` Lincoln Dale 2005-06-26 18:16 ` Valdis.Kletnieks @ 2005-06-29 19:32 ` Thomas Rösner 2 siblings, 0 replies; 620+ messages in thread From: Thomas Rösner @ 2005-06-29 19:32 UTC (permalink / raw) To: David Masover Cc: Lincoln Dale, Gregory Maxwell, Hans Reiser, Valdis.Kletnieks, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hi, I hereby throw myself into the neverending flames called "reiser4 plugins" - may it please the gods. David Masover (who really could use pgp mime attatchments): > So, the API becomes something like: > > cat crypto/inflated/foo # transparently decompressed > cat crypto/raw/foo.gz # raw, gzip-compressed Wee, I've seen this before. Wait, its AVFS! But didn't AFVS work with any underlying fs? Even nfs? cd foo.zip#/bar/ cat foo.tgz#/bar/plonk.txt There is no reason why there shouldn't be a gpg AVFS-plugin, too. OK, AVFS worked via coda's redir.o and IMHO we need smth like this in 2.6. But, apart from more efficient compression on disk (if that really ever works) you didn't show any convincing use cases. Or am I fundamentally wrong? regards, Thomas ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 4:59 ` Lincoln Dale 2005-06-26 5:07 ` Gregory Maxwell @ 2005-06-26 5:18 ` David Masover 2005-06-26 7:09 ` Lincoln Dale 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-26 5:18 UTC (permalink / raw) To: Lincoln Dale Cc: Hans Reiser, Valdis.Kletnieks, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Lincoln Dale wrote: > Hans Reiser wrote: > >> There has been no response to the technical email asking for what >> exactly it is that is duplicative, and what exactly it is that ought to >> be changed in how the code works, as opposed to what file the code is >> placed in, or who is considered its maintainer. There has been no >> response to the questions about whether the difference between class and >> instance makes our layer non-duplicative. >> >> Perhaps no response was possible? >> >> > Hans, > > the l-k community have asked YOU may times. any lack of response isn't > because of the kernel cabal .. its because YOU are refusing to entertain > any notion that what Reiser4 has implemented is unpalatable to the > kernel community. > > you can threaten all you want to take your code and move it to BSD. or > fork the kernel. whatever. > but if you want to get your work into the mainline kernel, you need to > provide answers to the question that keeps being asked of you - and > which you patently keep ignoring time & time again. > > in short, as per Message-ID: <42BBC710.8010906@cisco.com>: > posting to l-k on "why Reiser4 cannot use VFS infrastructure for > [crypto,compression,blahblah] plugins" - ideally, for each plugin. > > or again, in Message-ID: <42BBB1FA.7070400@cisco.com>: > [..] but instead just understand that this is the framework that you > have to > work in to get it into the mainline kernel. > if you don't want to go down that path, you're free to do so. > its open source, you can keep your own linux-kernel fork. > but if you want to get your code into mainline, i don't think its > so much a case of l-k folks telling you how to make your code > work under VFS. the onus is on you to say WHY your code > and plugins could never be put into VFS. > > or further back in Message-ID: <42BB7083.2070107@cisco.com> > you know that VFS is the mechanism in Linux. you know > (i hope..) how it works. it isn't so hard to see how many > of the Reiser4 "plug-ins" could be tied into VFS calls. > OR, if they cannot TODAY, propose how VFS _COULD_ be > made to do this. > > > NB. it doesn't matter what David thinks. this is linux-kernel, not > linux-users. Gee, thanks. Great attitude. I'm sure your users love you. By the way, Groklaw is run by a user. Ok, I'll bite. Hans put it best a moment ago: "Because our code is 90% library routines (aka plugins), eliminating the plugin layer is like eliminating main() in a user space program." We want to get a feeling of exactly which plugins you are talking about. Because if we pick the wrong ones and spend a few weeks implementing them, and come back and you think we're still too "layered" or maybe we've "bloated" VFS too much, and maybe the cycle repeats for another two years... What Hans seems to be worried about is having food and rent for two years, and the fact that in those two years, the XFS guys or the ext3 guys may duplicate enough of the "plugin" architecture that he gets no credit anymore, AND has to rewrite his stuff to work with theirs. I have to be neutral on that, as I realize that many of the linux-kernel people are sick of hearing it and honestly don't care. And to tell the truth, I'm a little sick of hearing it, and more than a little sick of the personal attacks from both sides. No, as a user, I just want a working plugin architecture to play with (I'm not *just* a user), and a working Reiser4 'cause it's fast, and I am eager to see new improvements coming out of Namesys, instead of two years spent trying to keep up with the vanilla kernel *and* adapt to some unspecified, possibly unneccessary, decree of a benevolent dictator. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr46kHgHNmZLgCUhAQKA+g//U9RiQpbaRQTBcYT4WCh+SPKJGheZCQ/S 1Xz1gI/M0phGVjfuLenpbz7CpIdfuZE670xccMXaqlYauW3D8NAkZopwLQQtuX9v MlctbWDX86Xbq0Ng3Zi6UPgKrY3kuWLP+NIjyq0geu5rnXFCgZLn2qOpZzWn1Uyf O0PNwKloBFoGeFcCs2F3HmzQ4sw2pVL645UxyatCmB/omNZIFTChkyMR4A7Ybvfc nUDtH9AMqJ/EROb3LkCQIK79I4yoDJraD64glkp/iIuADUCioVlAbyzTHuGIbFYY U3QqdOFSoCXJHC8PVSXCN1LBv4YWS2EWsYYiPiKisCHtRQi5jpZEgFdcAGbmiNaA PNIP6zfcAC8bxJ9aeH9LK+QbfzBU9LFjDIfn4TgrZdkDp+q6rHaS4EO3KWaVubn/ YDbDRd+QCDLgnzjNvQZdTXHrtotFXk+xWkKfN5e4fP5Z56EWXY1SUkv+oRC3vdJI yE0D+a8qg0XJKuFsEzhOa4Pxhu27eRCVPQ4b+s3ivmJmep3Og6v4MaG/SLTeCGMX ESHRpYUZfG81mwpI5GmkIm8E6dnZp6YmaQx9ZI9+J1B6mJ/1payL78TBIckq79Tx mKudpapkH3cZ93Br54f5C+pY/XjaHhepYZbkytl0BNb75R4T3r93s81M2q5N3ghN 3+ruvJY2Kto= =Skae -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 5:18 ` David Masover @ 2005-06-26 7:09 ` Lincoln Dale 2005-06-26 8:00 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Lincoln Dale @ 2005-06-26 7:09 UTC (permalink / raw) To: David Masover Cc: Hans Reiser, Valdis.Kletnieks, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover wrote: >Ok, I'll bite. Hans put it best a moment ago: > > [..] you seem to have some misconceived notion that this is somehow a "ReiserFS versus XFS" or "ReiserFS versus ext3" case. l-k could do without those conspiracy theories. lets just stick to the facts. >No, as a user, I just want a working plugin architecture to play with >(I'm not *just* a user), and a working Reiser4 'cause it's fast, and I >am eager to see new improvements coming out of Namesys, instead of two >years spent trying to keep up with the vanilla kernel *and* adapt to >some unspecified, possibly unneccessary, decree of a benevolent dictator. > > one of the nice things about linux is the freedom open-source gives you. there is no reason why Hans cannot keep whatever proprietary mechanisms to do whatever fancy plugins he wants and you as a user can use until the cows come home. but just don't have the false expectation that just because something is 'kool' that its going to get into the kernel without rigorous peer review & approval cheers, lincoln. > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 7:09 ` Lincoln Dale @ 2005-06-26 8:00 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-26 8:00 UTC (permalink / raw) To: Lincoln Dale Cc: Hans Reiser, Valdis.Kletnieks, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Lincoln Dale wrote: > David Masover wrote: > >> Ok, I'll bite. Hans put it best a moment ago: >> >> > [..] > > you seem to have some misconceived notion that this is somehow a > "ReiserFS versus XFS" or "ReiserFS versus ext3" case. > l-k could do without those conspiracy theories. lets just stick to the > facts. erm, let me take a peek at what you just chopped out. Yeah. Um. I mention XFS and ext3 *once* in the whole message. As an example of filesystems in general, because the only other two I'd consider (or ever use) for a desktop FS at this point is XFS or ext3. And it was a hypothetical opinion (I don't really know if / how much Hans feels that way), and I personally find it far easier to believe that there are stubborn people or that there isn't enough communication of facts than that there is actually some anti-reiser conspiracy. So, my mistake for trying to speak for other people. >> No, as a user, I just want a working plugin architecture to play with >> (I'm not *just* a user), and a working Reiser4 'cause it's fast, and I >> am eager to see new improvements coming out of Namesys, instead of two >> years spent trying to keep up with the vanilla kernel *and* adapt to >> some unspecified, possibly unneccessary, decree of a benevolent dictator. [...] > but just don't have the false expectation that just because something is > 'kool' that its going to get into the kernel without rigorous peer > review & approval I don't, actually. I did expect the peer review to be a little more efficient, less confrontational, and (maybe) more fair. But I'm actually a bit content right now. After all, I *am* running reiser4 on two of the three machines I maintain at home. And, if you scroll up a bit, at this point I just want this resolved. I don't particularly care if Hans decides to just keep maintaining a patchset, although I would be happier if it got into the mainstream kernel -- I wouldn't have to wait as long to install each new 2.6.11 or 2.6.12 + nvidia module. But this is a lot of time and energy from a lot of people on both sides who I would rather see working on something else. I appologize for the tone of that, too. I know you don't work for me... -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr5gq3gHNmZLgCUhAQLgixAAgC1WTVa8+6wP3CbCJafz4V5+f/hcJkuG xtXeh/CPvN4FzRRu+UjDVgji6yrLQ9AxFOa9kg9iJzZLIDDUNKu6UvFe+j22Mmpv F/24aLD8NAtF4JNGOJv6xFZtwk03N8Q92+CU0b5jPEViom2h55OkKfSIzoGz47Ee 45XGDx0v2LCHVG+HhVuG3EVQNjI4oBiwQteErHjmoNcvh7npkbdYGvEHRULgX3rO eCals0WPCQ+A10xDoTll6NvqEU59aHeheDw+FBkCZw4GhGaSCdZn0q8EHqqTdufL iU5z/Q6J98KvjeMdhlCW8QRWA+hSIwQJcn+09IzI2lT4QnpPDRTeX3NxsdCrVbLu fg37+d46cfWNrXpIrm3SoaTMl5GGvGGTekD46deTtotbJ40fSXGv3FbB6KCFK04s U6kjfnqO8fFG/iKWExCqts6HUPiboI/zpz8w/oL7XudJO/jxKavKQGk+POCFquyT U2e7KvZig5Ct0aunlIec7NrrRbutfYU6TFlYYRlV6XmIDP29ZDtqN9DsIBEvSeP5 7RFs4r0nTaIj17mpWmX5XCaLGNSUqzsQ1bzIoAl/D8NzbRTD9dsukvZyz5lW7tiZ zsWe4mHtJrZ1/mQf/oCj36FXID+bd3xN0WGIjli5gzMjCK8uCkS09MQ+JeLGQHES M/wMDcMcrwE= =3y4o -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 20:31 ` Valdis.Kletnieks 2005-06-25 20:52 ` Hans Reiser @ 2005-06-26 0:24 ` Hubert Chan 2005-06-26 2:46 ` David Masover 2005-06-26 3:14 ` David Masover 2 siblings, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-06-26 0:24 UTC (permalink / raw) To: Valdis.Kletnieks Cc: David Masover, Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Sat, 25 Jun 2005 16:31:37 -0400, Valdis.Kletnieks@vt.edu said: [...] > Meanwhile, PGP was designed to be used in an environment where you > could do this: "Today's secret plans are AES256 encrypted. The key is > the next key in your one-time-pad book, XOR'ed with your 128-bit > secret key - do it in your head". (And yes, you can easily memorize a > 16-digit hex number and learn to do an XOR with another 16-digit > number, if failing to do so means you could end up dead). Huh? I have no idea what this has to do with PGP vs. encryption being handled by the filesystem/loopback/dm-crypt. What's the difference between that scenario and "Today's secret plans are stored in the loopback file foo.iso on an AES256 encrypted ISO. The key is ... "? Or "Today's secret plans are stored AES256 encrypted in foo.txt. To access it, you'll need to do {magic command to enter the key into the filesystem} [1]. The key is ..."? [1] I have no idea what kind of interface the crypto plugin in Reiser4 will have. I'm assuming that there will be some commands for adding and removing keys from the plugin. If such commands don't exist, then we have a seriously broken system. > This is inconvenient for the user, but intractable for an attacker to > create a scenario where they can just 'vi /each/decrypted/file' ;) If you can memorize a 16-digit hex number and do XOR in your head, you can learn to unmount the loopback/remove the key from the filesystem too. Which is definitely easier than remembering to create a temporary RAMFS, mount it (with the right permissions!), decrypt the secret plans to that FS, edit the plans, re-encrypt the plans, blow away the RAMFS... >> won't do it. If security is transparent, just enter a password and >> go, then more people would do it. > "Just enter a password and go, then more people would do it". > Two words: "phishing e-mail". Social engineering works in meatspace too. Most people have locks on their doors, even though they're easy to get around if you can pretend to be from the electrical/gas/water company. But having locks on your doors is better than not having locks, because it prevents other types of attacks. I don't care if phishing gets around encryption. It would work the same if people didn't have their stuff encrypted. But users with encrypted stuff are safer from other types of attacks, and no less safe from phishing. I'd rather have people encrypting all their stuff and still be vulnerable to phishing but secure from someone stealing their computer and fetching all their personal data, than having people not encrypt their stuff and have all their personal data harvested when they lose their computers and still be vulnerable to phishing. P.S. Is the custom on the linux-kernel list really to Cc: everyone and their dog? I'm seeing a lot of long Cc: lists here... -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 0:24 ` Hubert Chan @ 2005-06-26 2:46 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-26 2:46 UTC (permalink / raw) To: Hubert Chan Cc: Valdis.Kletnieks, Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hubert Chan wrote: > On Sat, 25 Jun 2005 16:31:37 -0400, Valdis.Kletnieks@vt.edu said: > > [...] > > >>Meanwhile, PGP was designed to be used in an environment where you >>could do this: "Today's secret plans are AES256 encrypted. The key is >>the next key in your one-time-pad book, XOR'ed with your 128-bit >>secret key - do it in your head". (And yes, you can easily memorize a >>16-digit hex number and learn to do an XOR with another 16-digit >>number, if failing to do so means you could end up dead). [...] > [1] I have no idea what kind of interface the crypto plugin in Reiser4 > will have. I'm assuming that there will be some commands for adding and > removing keys from the plugin. If such commands don't exist, then we > have a seriously broken system. If we do meta-files as was originally intended, the command will be a shell script could look something like this: #!/bin/bash read -sp 'passphrase: ' key echo "$key" > "$0"/.../plugins/cryptocompress/key The syntax of that echo command may change, but this is what we like about metas -- less namespace fragmentation, less random interfaces. >>Two words: "phishing e-mail". [...] > I'd rather have people encrypting all their stuff and still be > vulnerable to phishing but secure from someone stealing their computer > and fetching all their personal data, than having people not encrypt > their stuff and have all their personal data harvested when they lose > their computers and still be vulnerable to phishing. Thank you, I was just about to say that. There's a quote about this. Someone once asked Mohammed, "Should we tie up the horses or trust in Allah?" Mohammed said, "Trust in Allah. ...But, tie up the horses." > P.S. Is the custom on the linux-kernel list really to Cc: everyone and > their dog? I'm seeing a lot of long Cc: lists here... It seems to be the custom of any list to just hit "reply-to-all". That way, even if someone posted a reply after reading the archives or from a forwarded mail, or even if they unsubscribe from the list, or even if someone simply opened up their client and started a thread by mailing linux-kernel@vger.kernel.org directly, or if someone just randomly adds someone who might be interested to the CC list directly -- no matter what, someone who's posted on a topic will see replies to that post until the topic dies. Of course, this isn't true of all lists. Some lists munge headers, meaning that either "reply" or "reply-to-all" goes to the same place, meaning no automatic CC's. I don't like that, because then it's harder to take something off-list, and easier to accidently send something supposedly private to the entire list. If you take something off-list when you don't mean to, you can just re-send it, but if you put something on-list when it was supposed to be private, there isn't much you can do... Of course, there's the annoying side-effect that if you are subscribed to the list, you'll get lots of dupes, but so what? Bandwidth is cheap. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr4XAngHNmZLgCUhAQK+JRAAgg4YlZ9cb0S5hp9JzGdZHm0FeJDoKIok voT5LvqgooQpJZk3mwyagYqvP5uvY2UgFA74seMqzTHRnynCDp4orCPGgADofaFU KfjGcVUS6SuY3VgeF7WBEjFj1NHSwDp3h0LfeRocEglbFxoPGJvw3gWSnX1QDZ68 nmuSRGVSB7nb1Os6c2zsM0uXvJA43HNJ+W/UrEPOFjiRI1bOOioxzphgVwCAQH3j 8Vb2/HWmPLTCqXoOYESZ5OBQOR6iZViegh/x/Rn+O99UfiENdacoX5xwGM68SAXM CR3JjRA3JEA1iScz9I2byD2dZyb2596Q09Xh/5/5PQxK8zGm0FtWWEbOvDeF7Re7 cQiXkZB9uLQDJel+jlwatKGCPRVlx9wDJ8puIMf47QOsWhx0TY8lAxCebu4zorjz K2vQiF/ZMOYXFB5nBCvgHL7XG24FRpuaU0wWo7+0cY2o4WBhvfBO5o+93Klwa7Na ltPKRFPv6B6KBDD71quSwZ9M1YkjfR0vaPZWV9T5TFBklfRv26N1DhNmW1o2KjRI wX3yrsIbAAG9dK3Vs71oxWIw0hqmfpo4UZTZGRoi8GL+drHBNvHxzNtaeSBF4AeO fxYnQHXO1+Us3zbb/oBR4Hm3ugvsRYXMyArm1hHHFlmcXdggjz+K36IiCK7wKpfp 2gVec7JaEkY= =3Mkk -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 20:31 ` Valdis.Kletnieks 2005-06-25 20:52 ` Hans Reiser 2005-06-26 0:24 ` Hubert Chan @ 2005-06-26 3:14 ` David Masover 2005-06-26 4:32 ` Hans Reiser 2 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-26 3:14 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Valdis.Kletnieks@vt.edu wrote: > On Sat, 25 Jun 2005 13:33:27 CDT, David Masover said: > >>>Now *think* for a moment - how does a hypothetical Reiser4 using ext3 format >>>gain any speed advantage with small files, when the speed advantage is based >>>on using a format other than ext3? > > >>happen in RAM. If you do a ton of work with a dataset that stays in >>RAM, Reiser probably performs as well or better than a ramdisk, because >>changes that get overwritten while still in RAM never actually touch the >>disk. > > > At that point, since the actual buffer management is being done at the VFS > level (see fs/buffer.c and friends) what you're really comparing is the speed > of journalling metadata - at which point you need to be *very* careful to No, just metadata. > specify exactly what configuration you're talking about. If you don't believe > me, investigate why mounting a filesystem with 'noatime,nodiratime' can make a > dramatic difference totally independent of the underlying filesystem, but the > actual amount of gain is dependent on format (hint - how far do the heads have > to move to record 3 atime updates against 3 random inodes on an ext2, an ext3, > and a VFAT filesystem, assuming no other disk activity), and why > ext3 has 3 different modes data=ordered/writeback/journal. I'm not saying format doesn't matter for _all_ optimizations, but there are some common ones for which it does matter. >> Reiser also doesn't fragment as quickly as ext3, and I don't >>think that has anything to do with its format. > > > Care to explain why it's not format-dependent? Reiser4 and XFS both do lazy allocation. They both avoid allocating blocks until they run out of buffer RAM. When they have to, they allocate and flush everything. This may have changed a bit -- Hans was talking about a more stack-like system, to avoid the system locking up for tens of seconds at a time while ALL ram is flushed -- but the principle is the same. That principle is that if you have a large chunk of data to allocate all at once, you are more likely to know how it should be arranged on disk. If you allocate every write(2) or every 5 seconds, how do you know if they are writing 2K or 2 gigs? You might try to pack it into 1 meg of space, then 5 megs, and so on, but you end up with quite a fragmented file. With allocate-on-flush, if you've got more like 100 megs to allocate at once, you can find space for that 100 megs. In fact, with the Reiser format, you can pack lots of tiny (much less than a block) files together to save space and speed things up. But the lack of fragmentation is not format-dependent. >>The FS that gets merged ahead of time without plugins would no longer be >>Reiser4. Go read the whitepaper, or tell me why I'm wrong, but even >>symlinks are implemented as plugins. > > > Which is another way of saying Reiser4 can't be merged in its present form. Actually, I'll have to go back on this a bit. There are different kinds of plugins, and I'm not sure exactly which people want in the VFS. This may be because nobody's said that, though. Still, while plugins may not depend on Reiser, Reiser depends on plugins. >>Actually, plugins are just as easy or easier than crypto-loop or >>dm-crypt. And why shouldn't my crypto be easy? Most users are insecure >>in all kinds of ways because of that attitude -- security is HARD, so I > > There's a vast distinction between "easy for implementors" and "easy for > users". Jaari Russo's loop-aes stuff does a wonderful job of being "easy for > users" - just say "mount", answer the passphrase, and you're good to go. The Not as easy as a plugin, though. Per-file granularity is nice. Especially on a USB key, where you're changing the files all the time. Sometimes you want just a few (encrypted) SSH/GPG keys and a bunch of (unencrypted) mpegs, and sometimes you want just a few (unencrypted) HTML files and a bunch of (encrypted) top-secret blueprints in extremely high-res JPEG/PNG format. A loop file on a keychain is only slightly better than partitions. [...] > Meanwhile, PGP was designed to be used in an environment where you could do > this: "Today's secret plans are AES256 encrypted. The key is the next key in > your one-time-pad book, XOR'ed with your 128-bit secret key - do it in your head". > (And yes, you can easily memorize a 16-digit hex number and learn to do an XOR > with another 16-digit number, if failing to do so means you could end up dead). > > This is inconvenient for the user, but intractable for an attacker to create a > scenario where they can just 'vi /each/decrypted/file' ;) Just as intractable the plugin way. Not to mention, setting up a ramdisk properly and decrypting to said ramdisk is a step you might forget, which might put your files on the disk unencrypted. Much more unforgivable than the heinous crime of making top-secret government work easier! >>won't do it. If security is transparent, just enter a password and go, >>then more people would do it. > > > "Just enter a password and go, then more people would do it". > > Two words: "phishing e-mail". If someone's going to learn about crypto, they are going to learn about how to avoid phishing scams. Both can be easy. After all, private/public keys can be made user-friendly and secure. At the same time! -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr4dmngHNmZLgCUhAQLJVA//WE20P/91cVuDclUT0W6bOX8bJvycOilB v55mUI9eoTdWink80RNo8QZ4aM1sidOrbtKglNmWdku1Fr80JW8HEPeQWRWR8PUL midO7ulTfRldBdM0mU+Ojl+Dj8vln6Qr+80Rveo5XDUlcWVwKHH7d87ONlFRlwIO NZMQXbB90huexJVoiITIGkFcDeoSdth/Em3cKjGhoEpreGkw6DuQzvS3aTOV+Fw4 c8neGsw2Tnx05+LuwE1VKxL7aV6U/Z0JDXwMqJtvINvkxnu6mKR++hRkWKW374LY 71idZzBu92oTNlEl02+y/EUGGU7+TMtduTo98ww4eN9r8bpB/WbMuOm+w05EKHKg H1OT1CX+G4sP/0GZaslPzh8z7cGI6318FN3Twk922J8t3dWcmNer4ULt6rp+c/38 stjbammdaBTWNTYg/BjZ+oedhMa4BHVTzwU2R0HpEzI3EIj/5rjek/D1cWzuBu/C NQ4QfVKJPHLgjKK4DV3iLQTkNpXC2skcluJ4FC1c+VYZTx8wvIrVUmJfPJajLAVg en42jbCYzdldtzR/yYal3b7KCeJoi1x1PUvStgL6iWSe7MjEJoF4jV/TcmE2gLf3 RzQUQ0UuMMisdQ2qbKAA3BrbZV239P/HUtyZdnIj8yWX4hF5Hhu7G1Ncz545VCdk +e0D/XVpgbc= =EBr/ -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 3:14 ` David Masover @ 2005-06-26 4:32 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-26 4:32 UTC (permalink / raw) To: David Masover Cc: Valdis.Kletnieks, Horst von Brand, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Think of reiser4 as being designed to be 90% library routines, and 10% program. Now, can these libraries be used by other filesystems? Why yes, some can. How many of them should be used by other filesystems? Reality: few people are going to rewrite their existing filesytems to mostly use our code. However, people writing new filesystems, if we have done our job right, will decide that our code is the easiest code base to use for implementing whatever differentiates them while avoiding reinventing what they don't seek to be better at. Detail: Regarding our allocation and balance and compress and encrypt at flush time code, unless your filesystem is also based on balanced trees it might be the least reusable code in the filesystem. It is the hardest code in the filesystem, because the algorithms we implemented were simply hard. Number one task for me, after we go in and I can stop dealing with prerequisites to inclusion, is to review the vm interaction from beginning to end one more time (and kill the emergency flush code). Good part: if you want to implement new filesystem semantics, our storage layer is more suitable for your innovating on top of than any other. Less work to code it, more functionality and flexibility, plugins are great for hacking on top of. The storage layer is very very high performance, and if semantics are your focus, using our storage layer is likely to be a good choice because it is well abstracted and works without understanding what it works on. For example, you can invent new items for the tree to balance (e.g. new directory entry formats), and all you have to do is write an item handler for the item and you don't have to touch the balancing code. In general, if a new filesystem can do some narrow aspect better than our existing library routine, it should do its aspect using its innovative new code, and where it is not trying to be better than us, it can reuse our code more easily than any alternative. Many people who would write a new filesystem from scratch, can, if they use our code base, accomplish their objectives with just writing some new plugins and contributing them to the collection. Others can define a new filesystem to consist of a particular configuration of plugins and glue that are what you get when you mount fubarfs. We would be happy to accomodate that by creating subdirectories for different flavorings of reiser4 to live in. Because our code is 90% library routines (aka plugins), eliminating the plugin layer is like eliminating main() in a user space program. Hans David Masover wrote: > Actually, I'll have to go back on this a bit. There are different kinds > of plugins, and I'm not sure exactly which people want in the VFS. This > may be because nobody's said that, though. > > Still, while plugins may not depend on Reiser, Reiser depends on plugins. > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-25 4:10 ` David Masover 2005-06-25 14:20 ` Valdis.Kletnieks @ 2005-06-26 4:01 ` Horst von Brand 2005-06-26 4:53 ` David Masover 2005-06-26 5:45 ` Hubert Chan 1 sibling, 2 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-26 4:01 UTC (permalink / raw) To: David Masover Cc: Horst von Brand, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List David Masover <ninja@slaphack.com> wrote: > Horst von Brand wrote: > > David Masover <ninja@slaphack.com> wrote: > >>Horst von Brand wrote: > >>>David Masover <ninja@slaphack.com> wrote: > >>>>Hans Reiser wrote: > >>>>>Jeff Garzik wrote: [...] > >>"Ain't broke" is the battle cry of stagnation. > > I see it as the battle cry of those that are looking for /real/ > > problems to solve. > I'll refer you to my other rant about stagnation and oil. Read it. Makes no sense, wind and solar power have their /own/ problems, environmentally speaking. Besides, as things stand today, they are *extremely* expnsive and hard to use, so next to useless (for now). > And, listen to yourself. "Good riddance, no, I don't use DOS" but > "Ain't broke is the battle cry of those looking for real problems to solve." > You can solve real problems in DOS, Sure. But not all (or even most) problems I'd like to solve with a computer as of today. So... > it's usable, it ain't broke, there > are even some decent games (doom) and windowing systems (win3.1 and > others) for it. Sure. But that's not enough. > But Linux is better. DOS ain't broke, For me, it is. > but Linux is better. So maybe > VFS ain't broke, I think it is doing fine. > but plugins would be better. Maybe. It's up to /you/ to convince the head kernel hackers (and me on the way). > I guess we'll only know > if we let Reiser4 merge... No. Just like devfs was "the obvious right way", it /had/ to be merged ASAP, "wrinkles will be ironed out later". Turned out it wasn't wrinkles, but fundamental brokenness, and it was soon abandoned by the people who promised to maintain it forever. And now we have the flamewar about its removal... > >>But, there are some things Reiser does better and faster than ext3, > > Yes, I've heard it is supposed to be faster on huge directories, and > > doesn't run out of inodes. And it is more efficient spacewise on small > > files. But then again, space is extremely cheap today... > Speed isn't. CPU is, but not disk speed. And packing stuff more > efficiently, without actually compressing it, will give you some of that > speed. For my current uses, ext3 is plenty fast enough. No pressing need to change. > Also, space is not so cheap that I won't take 25% more. It is cheap enough that I can't realistically fill the disks I have with /usefull/ stuff. So... > > And again, on a list around here I've seen several cries for help with > > completely hosed filesystems, all ReiserFS. No solution has ever come > > forth. > I haven't been counting, but I've seen a number of solutions fly around > reiserfs-list for people with reiser4 problems. It was ReiserFS 3. Maybe the problems are fixed now, but as they say about burned children... [...] > A lot of what people like about ext3 is its stability and fairly > universally accepted format. A lot of what people like about XFS is its > stability and speed, mainly with large files. A lot of what people like > about Reiser4 (as it is today) is its speed, with large and especially > with small files. Right. And mushing it all together is way more likely to combine all /bad/ features than to retain some of the /good/ ones. > Those are broad and somewhat uneducated statements, but I doubt most > people care what FS they are using if the stability and performance is > what they want. In that case, why have so much duplicated effort > between different filesystems -- even between ISO and UDF and Reiser and > XFS -- when most of what's really different is the on-disk format and > the optimization? Because they are different on-disk and are optimized for different uses, perhaps? I can't use ext3 for reading my CDs, I need NTFS to access the Windows partition here, ... > So, in this hypothetical situation where ext3 is a reiser4 plugin, > suddenly all the ext3 developers are trying to improve the speed and > reliability of reiser4, which benefits both ext3 and reiser4, instead of > just ext3. I guess that it won't ever turn out to be that simple. ext3 developers will have to consider not screwing up XFS, etc. And I don't see any real difference there from where we stand today... just a bigger mess: VFS with ReiserFS with plugins for ext3, instead of VFS with ext3. No gain, much pain. [...] > > And that would surely break Windows compatibility (because you have to keep > > the data on what to encrypt one the filesystem itself). Besides, having > Aside from what someone else already said about this, why not just have > support for accessing, say, a .gpg file as transparently decrypted? That should, if anything, be a /user/ decision, not a /kernel/ one. Even sometimes decrypt, sometimes don't. > You > don't even need file-as-directory, just create a file called foo which > is really the decrypted version of foo.gpg. No need to change the > format, just the filesystem. No need to change the filesystem, learn to use the tools at hand. > > pgp, or gpg, or crypt, or my own whacky encryption proggie do the job in > > /userland/, and not shoving into the kernel and so down the next user's > > throat, is in the end a largeish part of what Unix is all about for me. > Do you use ipv6? Sometimes. > I don't. And it's not shoved down my throat, either, > although it's in the kernel. I simply disable it, and you would (I'm > sure) disable the crypto stuff. An Internet standard is a quite different kettle of fish than the pet experimental filesystem for a minority operating system... > Plus, as someone else said, it's much easier to do > $ vim /some/encrypted/file > than > $ gpg --decrypt /some/encrypted/file > /some/decrypted/file > $ vim /some/decrypted/file > $ gpg --encrypt /some/decrypted/file > /some/encrypted/file > $ shred /some/decrypted/file So what? Write your script to do it. Or use emacs, I'm sure it either has now or will soon have a plugin for it... much easier to develop, much more flexible to use, ... > Not to mention, shred doesn't work on modern filesystems, so you need to > either patch vim (and any other program you might want to use), create > an actual ramdisk, or deactivate swap and mount a tmpfs. Right. And all this complex futzing around (and making sure the unencrypted data doesn't remain if the power is cut in the right moment, and...) > On doing stuff in userland: I am all for moving tons of stuff to > userland, but not until someone does a microkernel right, if it's even > possible. And even then, I'd probably still use Linux and argue for > some stuff in the kernel, because Linux has developers. Developers. > Developers. Developers. Sorry, I don't buy this. Linux (the kernel) has tons of developers. But that doesn't mean everybody is interested in everything. Just like there are probably much more userland developers (for one, it is /much/ easier to work with...), and there not everybody is interested in everything. Putting stuff in the kernel that doesn't belong there (== can't be done elsewhere) is useless bloat. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 4:01 ` Horst von Brand @ 2005-06-26 4:53 ` David Masover 2005-06-26 5:45 ` Hubert Chan 1 sibling, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-26 4:53 UTC (permalink / raw) To: Horst von Brand Cc: Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Horst von Brand wrote: > David Masover <ninja@slaphack.com> wrote: > >>Horst von Brand wrote: >> >>>David Masover <ninja@slaphack.com> wrote: >>> >>>>Horst von Brand wrote: >>>> >>>>>David Masover <ninja@slaphack.com> wrote: >>>>> >>>>>>Hans Reiser wrote: >>>>>> >>>>>>>Jeff Garzik wrote: > > > [...] > > >>>>"Ain't broke" is the battle cry of stagnation. > > >>>I see it as the battle cry of those that are looking for /real/ >>>problems to solve. > > >>I'll refer you to my other rant about stagnation and oil. > > > Read it. Makes no sense, wind and solar power have their /own/ problems, > environmentally speaking. Besides, as things stand today, they are *extremely* > expnsive and hard to use, so next to useless (for now). Have you bothered to research them? The main problem is in the construction. After that... hard to use? Not really. Expensive? They pay for themselves, eventually. Plus, they get better as money goes into them. Oil *can't* get much better, but all the money goes there, instead of into alternatives. Thank you, Republicat administration... > Maybe. It's up to /you/ to convince the head kernel hackers (and me on the > way). > > >> I guess we'll only know >>if we let Reiser4 merge... > > > No. Just like devfs was "the obvious right way", it /had/ to be merged > ASAP, "wrinkles will be ironed out later". Turned out it wasn't wrinkles, > but fundamental brokenness, and it was soon abandoned by the people who > promised to maintain it forever. And now we have the flamewar about its > removal... Should it have been kept out in the beginning, before we knew we needed udev? Would we have udev at all, had devfs not been merged? >>>>But, there are some things Reiser does better and faster than ext3, > > >>>Yes, I've heard it is supposed to be faster on huge directories, and >>>doesn't run out of inodes. And it is more efficient spacewise on small >>>files. But then again, space is extremely cheap today... > > >>Speed isn't. CPU is, but not disk speed. And packing stuff more >>efficiently, without actually compressing it, will give you some of that >>speed. > > > For my current uses, ext3 is plenty fast enough. No pressing need to change. That is the problem I attempted to illustrate with the oil rant. No pressing need to change doesn't mean that something astronomically better shouldn't be adopted. Amish people live just fine. There's no pressing need for them to change. But I bet that many of them would be happier if they had a car. >>>And again, on a list around here I've seen several cries for help with >>>completely hosed filesystems, all ReiserFS. No solution has ever come >>>forth. > > >>I haven't been counting, but I've seen a number of solutions fly around >>reiserfs-list for people with reiser4 problems. > > > It was ReiserFS 3. Maybe the problems are fixed now, but as they say about > burned children... speaking for yourself? >>A lot of what people like about ext3 is its stability and fairly >>universally accepted format. A lot of what people like about XFS is its >>stability and speed, mainly with large files. A lot of what people like >>about Reiser4 (as it is today) is its speed, with large and especially >>with small files. > > > Right. And mushing it all together is way more likely to combine all /bad/ > features than to retain some of the /good/ ones. Actually, it wasn't "mushing it all together". It ended up throwing out a fair bit of it in the example I was talking about. But I really shouldn't be arguing that. It's not what people care about. >>Those are broad and somewhat uneducated statements, but I doubt most >>people care what FS they are using if the stability and performance is >>what they want. In that case, why have so much duplicated effort >>between different filesystems -- even between ISO and UDF and Reiser and >>XFS -- when most of what's really different is the on-disk format and >>the optimization? > > > Because they are different on-disk and are optimized for different uses, > perhaps? I can't use ext3 for reading my CDs, I need NTFS to access the > Windows partition here, ... As things stand. And it would be pointless to change some of those, like CDs. But then, transparent decompression as a plugin might be better. >>So, in this hypothetical situation where ext3 is a reiser4 plugin, >>suddenly all the ext3 developers are trying to improve the speed and >>reliability of reiser4, which benefits both ext3 and reiser4, instead of >>just ext3. > > > I guess that it won't ever turn out to be that simple. ext3 developers will > have to consider not screwing up XFS, etc. And I don't see any real > difference there from where we stand today... just a bigger mess: VFS with > ReiserFS with plugins for ext3, instead of VFS with ext3. No gain, much > pain. That you don't see the gain doesn't mean it doesn't exist. I don't deny the pain, though I don't think it will be as bad as you think. For one thing, storage plugins are fairly self-contained, or should be. >>>And that would surely break Windows compatibility (because you have to keep >>>the data on what to encrypt one the filesystem itself). Besides, having > > >>Aside from what someone else already said about this, why not just have >>support for accessing, say, a .gpg file as transparently decrypted? > > > That should, if anything, be a /user/ decision, not a /kernel/ one. Even > sometimes decrypt, sometimes don't. Sometimes decrypt, sometimes don't, would certainly be allowed. What's not allowed now is an easy way to vim an encrypted file without patching/wrapping vim or dealing with ramdisks. And that's assuming vim is the only program I want to use. Transparency is a good thing. >> You >>don't even need file-as-directory, just create a file called foo which >>is really the decrypted version of foo.gpg. No need to change the >>format, just the filesystem. > > > No need to change the filesystem, learn to use the tools at hand. Read above. >>>pgp, or gpg, or crypt, or my own whacky encryption proggie do the job in >>>/userland/, and not shoving into the kernel and so down the next user's >>>throat, is in the end a largeish part of what Unix is all about for me. > > >>Do you use ipv6? > > > Sometimes. > > >> I don't. And it's not shoved down my throat, either, >>although it's in the kernel. I simply disable it, and you would (I'm >>sure) disable the crypto stuff. > > > An Internet standard is a quite different kettle of fish than the pet > experimental filesystem for a minority operating system... "standard"? How many Linux people use ipv6? How many use reiser4? And what's your point, anyway? Disable it if you don't like it. >>Plus, as someone else said, it's much easier to do >>$ vim /some/encrypted/file >>than >>$ gpg --decrypt /some/encrypted/file > /some/decrypted/file >>$ vim /some/decrypted/file >>$ gpg --encrypt /some/decrypted/file > /some/encrypted/file >>$ shred /some/decrypted/file > > > So what? Write your script to do it. Or use emacs, I'm sure it either has > now or will soon have a plugin for it... much easier to develop, much more > flexible to use, ... more flexible? You don't seem to understand plugins. That makes sense; you don't want to. Plugins are far more flexible than trying to patch every single program I want to deal with. And the script described doesn't work because shred doesn't work, meaning you have to deal with ramdisks, which is even worse and less likely to work out-of-the-box. >>Not to mention, shred doesn't work on modern filesystems, so you need to >>either patch vim (and any other program you might want to use), create >>an actual ramdisk, or deactivate swap and mount a tmpfs. > > > Right. And all this complex futzing around (and making sure the unencrypted > data doesn't remain if the power is cut in the right moment, and...) Unencrypted data stays in RAM... with plugins. >>On doing stuff in userland: I am all for moving tons of stuff to >>userland, but not until someone does a microkernel right, if it's even >>possible. And even then, I'd probably still use Linux and argue for >>some stuff in the kernel, because Linux has developers. Developers. >>Developers. Developers. > > > Sorry, I don't buy this. Linux (the kernel) has tons of developers. But > that doesn't mean everybody is interested in everything. Just like there > are probably much more userland developers (for one, it is /much/ easier to > work with...), and there not everybody is interested in everything. Putting > stuff in the kernel that doesn't belong there (== can't be done elsewhere) > is useless bloat. 90% of what's currently in the kernel can be done elsewhere. Witness HURD. Doesn't mean it should. Witness HURD. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr4043gHNmZLgCUhAQI4/A/9E6kVoYyVgQWFrrw7SO6e6vVZzQufsTFe h2prUuaITxSDYc6oO6gR4UorkQAGtHjFFKCzdyH+/Eo2cmYAGtom14y8aNjWqQOj iz4hqyH4RZJ3eSP0/YsMdopoDpkdtrlD6QCWL49C7A1h1Mrq0gL+SrRNzpLYc+QW XfyqQppMgDRIQfCHKG+ZgS9cloW1HnYHppxYakvhkS4ZiFktEi1UHXj+2GQ22Mm5 EN56EHRthS8v2k7vOdsrkQE02lrgDtWWOt8bkrkHFCFB6lU9oXbqlKHWRrU/dd27 zYgvajgVZWjGT3iAXgyhcojCuIEpyOB2+Cegdga2D/H+kxh8tk1J1f1MJh9oXosb 4ScrlUIq+ghVx2+/YMvTmA8geKMqqi9OBvB1npfFXklXJHa/QJzTTo3/MJGxJ6Ko fKyG0sWWNwbMdCmLoCQp1T+WycxMeOILw8jheVfQbF+JAXm/ialDXJ/rH5RYHeG4 vhlZHxKqihgiuQ0ZGvj8niR5vlei9tKZ8fBFiVgDxQhoUhrutiVQKIzxT6YXMQqM PpXefWdDNi4vnNFn6eDVcXTs14kvBod8SM/5WxKUnnlCJNvjFUwo5x4lQ9iXnlYT LHbJ06lGbLU6y8RfaY2+eBYziyMSStql77mwW9y7WssXI0cV7VzgEYSTd+S1eBN5 Yr+8chQqWOI= =/wpm -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 4:01 ` Horst von Brand 2005-06-26 4:53 ` David Masover @ 2005-06-26 5:45 ` Hubert Chan 1 sibling, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-26 5:45 UTC (permalink / raw) To: Horst von Brand Cc: Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Sun, 26 Jun 2005 00:01:39 -0400, Horst von Brand <vonbrand@inf.utfsm.cl> said: > David Masover <ninja@slaphack.com> wrote: [...] >> Also, space is not so cheap that I won't take 25% more. > It is cheap enough that I can't realistically fill the disks I have > with /usefull/ stuff. So... And since when did you become the spokesperson for all Linux users? Just because *you* can't think of a use for 25% more space doesn't mean that the rest of us don't want it. Do you really think that there isn't a significant number of people who would love to have more space? [...] > So what? Write your script to do it. Or use emacs, I'm sure it either > has now or will soon have a plugin for it... much easier to develop, > much more flexible to use, ... Oh yes. Please. Let's make everyone use emacs. Even if they want to edit an OpenOffice document or a picture. -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* cryptocompress [was Re: reiser4 plugins] 2005-06-23 19:24 ` Horst von Brand 2005-06-23 20:12 ` Adrian Ulrich 2005-06-23 22:04 ` David Masover @ 2005-06-28 13:54 ` Pavel Machek 2 siblings, 0 replies; 620+ messages in thread From: Pavel Machek @ 2005-06-28 13:54 UTC (permalink / raw) To: Horst von Brand Cc: David Masover, Hans Reiser, Jeff Garzik, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List Hi! > > and vfat people may decide they want cryptocompress, > > I'm sure they don't, because it is mostly for Windows and assorted devices > (pendrives, digital cameras, ...) compatibility. Actually cryptocompress for vfat *would* be nice; pen drives are small, slow, and likely to be lost/stolen. That makes them very nice candidates for cryptocompress ;-). Pavel -- teflon -- maybe it is a trademark, but it should not be. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 5:58 ` Hans Reiser 2005-06-23 6:26 ` Pekka Enberg 2005-06-23 14:11 ` David Masover @ 2005-06-23 21:41 ` Alan Cox 2005-06-24 1:23 ` reiser4 plugins (back to flames, oh well) Hans Reiser 2 siblings, 1 reply; 620+ messages in thread From: Alan Cox @ 2005-06-23 21:41 UTC (permalink / raw) To: Hans Reiser Cc: Jeff Garzik, David Masover, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List On Iau, 2005-06-23 at 06:58, Hans Reiser wrote: > Jeff Garzik wrote: > > > We have to maintain said ugly code for decades. > > No you don't, I do. Really. I must be mis-remembering some of the comments made about reiserfs 3 during the 2.5 to 2.6 period. > I am entitled to get some advantage from being the first on the block You were not first on the block. Linus was, thats why it's Linux (well not directly his fault about the name) and why he sets policy. I've never heard Linus espousing such an idea so I wonder why you think that way. Rather I've heard Linus tell groups of people with differing ideas about interfaces to go figure it out first and build the right interface then come back with a consensus (for example with SELinux) > It is odd how Hellwig no longer describes himself as XFS associated Why would that be "odd". What are you trying to imply and why not just say it if you actually have something to say. Alan ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins (back to flames, oh well) 2005-06-23 21:41 ` reiser4 plugins Alan Cox @ 2005-06-24 1:23 ` Hans Reiser 2005-06-24 3:26 ` reiser4/VFS plugins David Masover 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-24 1:23 UTC (permalink / raw) To: Alan Cox Cc: Jeff Garzik, David Masover, Christoph Hellwig, Andrew Morton, Linux Kernel Mailing List, ReiserFS List Alan Cox wrote: >>I am entitled to get some advantage from being the first on the block >> >> > >You were not first on the block. Linus was, > >thats why it's Linux (well >not directly his fault about the name) and why he sets policy. I've >never heard Linus espousing such an idea so I wonder why you think that >way. Rather I've heard Linus tell groups of people with differing ideas >about interfaces to go figure it out first and build the right interface >then come back with a consensus (for example with SELinux) > > We aren't people with differing ideas about interfaces, we are one filesystem team with functionality that no other filesystem has yet expressed a concrete desire to use, and some guy who complains about it after it has been the basis for coding for 4 years without expressing any desire to use it himself, merely expressing a desire that we not use it because he never paid any attention to our website or mailing list for the last 4 years as we developed it. Oh, and he says it is duplicative without explaining where in VFS plugins and pluginids are currently defined. Oh, and rather than saying "clever guys, thanks for coming up with this, I am going to go write a patch that makes your ideas available to other filesystems to do this", he says, stay out of the tree because your code is so crappy that it has to be used by everyone else before you can go in. There is absolutely nothing preventing him from generalizing it to other filesystems after we go into the tree. If you take the existing code as a base, the generic part of the work required is I hope small. The fs specific part is more though, as each filesystem has to implement methods for storing and retrieving plugins, and then a set of plugins (and their methods). It is all pure turf war. Some guy suddenly realized that VFS was not doing something, and got upset that someone else was writing code that did the job. Since it was someone else's code, and ready to go, and he had nothing himself written yet, it had to be crap and kept out. In fairness, it might be possible to eliminate a function indirection or two if the effort was made. There is no need to do it this week as merging our code will not make it the least bit more difficult to change things later, but hey, if someone wants it, happy to discuss the right way to do it. This is why Linux doesn't have the contributors its market share would suggest it should expect. Contributing to Linux is such a misery. If you guys would just say "plugins, interesting, can we make it more general so other filesystems can use it?" I would be happy to have Zam do it for you. ^ permalink raw reply [flat|nested] 620+ messages in thread
* reiser4/VFS plugins 2005-06-24 1:23 ` reiser4 plugins (back to flames, oh well) Hans Reiser @ 2005-06-24 3:26 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-24 3:26 UTC (permalink / raw) To: ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ok, how long will it actually take to implement the suggested changes? And, can we maintain both versions, VFS and internal? It might be a good idea to be able to present both versions at once and let the benevolent dictators take their pick. They didn't like it when we changed some VFS things and were trying to merge, and they don't like it now when we duplicate some VFS things and try to merge. It seems they just don't want it merged. So, we can give them the choice, very bluntly and simply, with code to back it up. "Choose this patchset or this one, or tell us what you really want changed." (Please excuse the "we" pronoun. I can't say I've been helping much.) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrt9dHgHNmZLgCUhAQL9vQ/9GUvW5qw+5mJUniMly+IpCXbEur8CvPU0 1B+cIulUjMPX6JHQ9DA4o1tCRPp0rE8423jXpmIfFsEJ5hs7coOEaRTctxNI67pz wzG4N8HVY8h6obvLXqYPU1u93unWRi1bzriGKGCi/hlMCdFss8J19O6aPtKlQq/p HB1Q4zMhr+nHbI7iMjJk3K0ZwQwA7UjG3bwnaJNl7CNUcnTsoyb+qVuALUJgIdtK 262OWUzOdrLF00zkQdy1rzoBya5LJsZ+Wn2f1Jzbq0Jsc1dDYjs0Tr9aOrgLNTqZ S7W5rIPJm5uKYWQZppxuLMKDGr7/sPki3If8OSJUa2g+0uRnpyACsnsejSYfCiYi 1k1pp04i9WtVssdYbEzLxYrJE+1edk34X0SfSf2i/kNTZkr71yx9R2V8S0yuXCJ4 V2UEY4Se+kLDxHphWlrOqP3SE/9AjAU/tzxeBdOLZr/z329F9Yljh2IpzJTInOux PmPzhZfxLL6yVG9O5FhCfPhIVDqlOFi5KyrE1apgVIdwr9rS3xZOWkGepX4qyvC6 b+LKfbOoz/X2hheXsmLMu4txXd+5lRs1DEDBGUVYAoWJ98SSE7SGGAVoch0F7O9s +S1KFke9a9JS4Nd0YQNyTbZ1/nlTnDuWXF/P3sOUJONxNXAQYliwJwYTZobSf22y Fsf375GHEPI= =d7sL -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 4:25 ` David Masover 2005-06-22 5:11 ` Bedros Hanounik 2005-06-22 5:18 ` Jeff Garzik @ 2005-06-22 5:36 ` Christoph Hellwig 2005-06-22 7:46 ` David Masover 2005-06-22 9:56 ` Stefan Smietanowski 2005-06-22 14:00 ` Rik Van Riel 4 siblings, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-06-22 5:36 UTC (permalink / raw) To: David Masover Cc: Jeff Garzik, Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Tue, Jun 21, 2005 at 11:25:24PM -0500, David Masover wrote: > > You're basically implementing another VFS layer inside of reiser4, which > > is a big layering violation. > > There's been sloppy code in the kernel before. I remember one bit in > particular which was commented "Fuck me gently with a chainsaw." If I > remember correctly, this had all of the PCI ids and the names and > manufacturers of the corresponding devices -- in a data structure -- in > C source code. It was something like one massive array definition, or > maybe it was a structure. I don't remember exactly, but... Every device driver has a big array of corresponing device ids as an array in C code - oh my god we're doomed .. not. > I agree there, too. In fact, some people have suggested that all > "legacy" (read: non-reiser) filesystems should be implemented as Reiser4 > plugins, effectively killing VFS.* > > So, Reiser4 may eventually take over VFS and be the only Linux > filesystem, but if so, it will have to do it much more slowly. Take the > good ideas -- things like plugins -- and make them at least look like > incremental updates to the current VFS, and make them available to all > filesystems. And why exactly would we replace a stable, working abstraction with an unpoven mess? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 5:36 ` reiser4 plugins Christoph Hellwig @ 2005-06-22 7:46 ` David Masover 2005-06-26 16:52 ` Christoph Hellwig 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-22 7:46 UTC (permalink / raw) To: Christoph Hellwig Cc: Jeff Garzik, Hans Reiser, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Christoph Hellwig wrote: > On Tue, Jun 21, 2005 at 11:25:24PM -0500, David Masover wrote: > >>>You're basically implementing another VFS layer inside of reiser4, which >>>is a big layering violation. >> >>There's been sloppy code in the kernel before. I remember one bit in >>particular which was commented "Fuck me gently with a chainsaw." If I >>remember correctly, this had all of the PCI ids and the names and >>manufacturers of the corresponding devices -- in a data structure -- in >>C source code. It was something like one massive array definition, or >>maybe it was a structure. I don't remember exactly, but... > > > Every device driver has a big array of corresponing device ids as an > array in C code - oh my god we're doomed .. not. I could throw the same sarcasm back at you. We must be doomed because Reiser does some stuff that VFS already does! Or am I misunderstanding the complaint? >>I agree there, too. In fact, some people have suggested that all >>"legacy" (read: non-reiser) filesystems should be implemented as Reiser4 >>plugins, effectively killing VFS.* >> >>So, Reiser4 may eventually take over VFS and be the only Linux >>filesystem, but if so, it will have to do it much more slowly. Take the >>good ideas -- things like plugins -- and make them at least look like >>incremental updates to the current VFS, and make them available to all >>filesystems. > > > And why exactly would we replace a stable, working abstraction with an unpoven > mess? How does it get proven if you won't give it a chance as a *separate* unproven mess, with a big fat EXPERIMENTAL flag, for users to play with? I know, it exists as a separate patch. But it works now, and I think the best way to "prove" it would be to package it with the kernel. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrkXY3gHNmZLgCUhAQLQUw//ZFN1KS+2wS/yDMa+/oWXVemZ690sMCLx ZlKGg82bnv2XxqMXQwuPG9V02oN/D+1bkPmZzr8rD/tm5WshxpAHroIhnp3ZVpRi lbMwULFQ8Z8fcsY3+YUag4XAUYGK+tmIeZc47FJGL0avsRa3RFJsFm+Kb6E/fi2f H4wda43rt2CJYD5GqCtMsqyxzHzPclKHq25betIcPWBOqvE5NzQbc2tFTo0n3KMb vmyZc4B34kiKhrrW/7pZCxDpiGjoxr87F19Tk8IIltM9kAuSVLXgtY/T+2DA2vJE 2N/Offr1rZh9zSq8PGkGoI+K41AaY3CkeYGjUF2eiZd4qwE624/1jUSEg685Puse 091EuIMzdndJYM0H+OsaFtvH9Rc67Hv6yR7aucNF5j8sIam37y7Fl+MToRgJK1+E 7YSpm/Ld61RaPqbJ4mqv4f0fHLTa2SpbFI8vmA1ARuiA+/YtUz9jBjLrPtMo4ppj VvNTVMmftUgRr1NlQ+MKJO4Kxt4kKQnt1OtUe2y4bjCqO7ldUvPWLKGhsY0EsS0k 9yymlBbhsjTFrY9CsyrThshyHe9ikBVSLY7i16W+KhjLF/FKaq9k93nHd4B5Shni Km9zyd0DlCUr3Y20SpBDITCWtM0CL0YQzeEW0JJTxVpHIDjh6s65XcBfrlWwEUiw j/GJZA5h+bw= =fBov -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 7:46 ` David Masover @ 2005-06-26 16:52 ` Christoph Hellwig 2005-06-26 17:21 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-06-26 16:52 UTC (permalink / raw) To: David Masover Cc: Jeff Garzik, Hans Reiser, Andrew Morton, linux-kernel, ReiserFS List On Wed, Jun 22, 2005 at 02:46:44AM -0500, David Masover wrote: > >>There's been sloppy code in the kernel before. I remember one bit in > >>particular which was commented "Fuck me gently with a chainsaw." If I > >>remember correctly, this had all of the PCI ids and the names and > >>manufacturers of the corresponding devices -- in a data structure -- in > >>C source code. It was something like one massive array definition, or > >>maybe it was a structure. I don't remember exactly, but... > > > > > > Every device driver has a big array of corresponing device ids as an > > array in C code - oh my god we're doomed .. not. > > I could throw the same sarcasm back at you. We must be doomed because > Reiser does some stuff that VFS already does! Or am I misunderstanding > the complaint? I rather wanted to say I absolutely don't see any correlation of your PCI driver example to what we're discussing here. PCI driver hardcode ID tables because they are supposed to do that. And if a PCI driver works around hardware bugs for a specific subset of hardware it needs to use an ID table for that one aswell. And adding a strong comment about broken hardware is considered to be just fine in Linux kernel land aswell. Now to your reply. We're not doomed if a driver re-implements "something" we already have in common code. We would be doomed if every driver reimplements lots of things, that's why we push hard to avoid drivers doing that. It gets even more important if that "something" duplicated is not some simple piece code but complex abstractions. > How does it get proven if you won't give it a chance as a *separate* > unproven mess, with a big fat EXPERIMENTAL flag, for users to play with? > > I know, it exists as a separate patch. But it works now, and I think > the best way to "prove" it would be to package it with the kernel. You're free to test it as much outside the kernel tree. This might make it more proven one day, but not automatically less of a mess. The real point here is that we already have a useful abstraction in that area, and we spent a lot of work to make it proven and not a mess (or just a little bit of a mess..), and thus we'd rather see people extending it reasonably for their needs instead of duplicating lots of infastructure in less than ideal ways. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 16:52 ` Christoph Hellwig @ 2005-06-26 17:21 ` David Masover 2005-06-26 17:28 ` Christoph Hellwig 0 siblings, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-26 17:21 UTC (permalink / raw) To: Christoph Hellwig Cc: Jeff Garzik, Hans Reiser, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Christoph Hellwig wrote: > On Wed, Jun 22, 2005 at 02:46:44AM -0500, David Masover wrote: > >>>>There's been sloppy code in the kernel before. I remember one bit in >>>>particular which was commented "Fuck me gently with a chainsaw." If I >>>>remember correctly, this had all of the PCI ids and the names and >>>>manufacturers of the corresponding devices -- in a data structure -- in >>>>C source code. It was something like one massive array definition, or >>>>maybe it was a structure. I don't remember exactly, but... >>> >>> >>>Every device driver has a big array of corresponing device ids as an >>>array in C code - oh my god we're doomed .. not. >> >>I could throw the same sarcasm back at you. We must be doomed because >>Reiser does some stuff that VFS already does! Or am I misunderstanding >>the complaint? > > > I rather wanted to say I absolutely don't see any correlation of your > PCI driver example to what we're discussing here. PCI driver hardcode > ID tables because they are supposed to do that. And if a PCI driver works > around hardware bugs for a specific subset of hardware it needs to use > an ID table for that one aswell. And adding a strong comment about broken > hardware is considered to be just fine in Linux kernel land aswell. I seem to remember the comment was more about hardcoded ID tables. And this was the generic ID code database, which is now maintained out of the kernel: /usr/share/misc/pci.ids A list of all known PCI ID's (vendors, devices, classes and sub-classes). That is what I was referring to, that used to be in the kernel. What this has to do with is whether you believe that it's better to keep code out until it's perfectly clean, or to let in code that has some things about it that you don't like, but works, is useful, and the maintainers seem very willing to fix it later on. I believe it's better to let stuff in so long as it works and doesn't break anything, so long as it is being improved to the *real* standards along the way. But, this opinion obviously isn't shared by most people here, so I'm going to stop arguing it now. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr7kMHgHNmZLgCUhAQIHdg/+M0ExRndz9Wym0rTcnSjlg4dc3cM5ZW7N LZ1lcLM+4Vtwsj16dc9ezNj7VLAx7Vmj/3afW3TxmjQKn50J53ZlTfd6HgpBvkAi i3V7syjYJg/WaqOlEWCDwO3i/HzYdd9QAgJBbL30g56/ZtJj6SNFlKvb6UizYjIf dHgY/ZG3BuUKLBTQFUaFuBmkb/eHlhZAq7qbwn6om3bR9UmjDr41l6nP/Ry9k438 gKvwNUQZX9EkQK/F34uDM8S1bbqt8YBcULbUBYp6A12kTL8dLS3Ax8TMbHK0Zxk1 OAMpel7e9kVUbC+NAGv2PRgJQEL4jDzwWS8kP12oBbeacxRnwj1WAKixBM+3v+uq ThGwcN8CXCfPW8DBokzqYw32vVA+PVpz3tCmnVyobPPPQZdZ5wlKTgZ+Yg4NCp+M WstSf/LE6OjVIH7xjVLBeZGuynmXHshuEubwRwaiHJKp9kUli+WEpJconvR0W3ph 4WQ6/7px64XxOPnxfR7jjSCNVKjzEZjeXOf1LbJF0a8yh6eVd5g1lsxYS3Ht04w3 yEup3PKWkhvIlGZw8w7IxR4fDI80/t4F9EugILJMLDwMwvJ1k5rRWrEpzF1/I6OM wdq99cj380B600peLWZ1PIQpqb0iDj+FBqio/MIQSz4Pqlg7qaME21kI8PHxIPib kJUIRwlarKA= =TNGY -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 17:21 ` David Masover @ 2005-06-26 17:28 ` Christoph Hellwig 2005-06-26 17:51 ` David Masover 0 siblings, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-06-26 17:28 UTC (permalink / raw) To: David Masover Cc: Christoph Hellwig, Jeff Garzik, Hans Reiser, Andrew Morton, linux-kernel, ReiserFS List [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain; charset=unknown-8bit, Size: 1047 bytes --] On Sun, Jun 26, 2005 at 12:21:52PM -0500, David Masover wrote: > I seem to remember the comment was more about hardcoded ID tables. > > And this was the generic ID code database, which is now maintained out > of the kernel: > > /usr/share/misc/pci.ids > A list of all known PCI ID's (vendors, devices, classes and sub-classes). > > That is what I was referring to, that used to be in the kernel. And you once again showed that you don't understand what you're talking about. Said database is a pci id to name mapping, which is completely irrelevant for any driver. For things like your example there's very little thing you can do but hardcoding a set of pci ids in one way or another. > What this has to do with is whether you believe that it's better to keep > code out until it's perfectly clean, or to let in code that has some I doesn't need to be perfect, we just need it in a reasonable state and have a buying that it's going to continue to evolve in the rigþt direction. And we're are very far from both of them in this ccase. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 17:28 ` Christoph Hellwig @ 2005-06-26 17:51 ` David Masover 0 siblings, 0 replies; 620+ messages in thread From: David Masover @ 2005-06-26 17:51 UTC (permalink / raw) To: Christoph Hellwig Cc: Jeff Garzik, Hans Reiser, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Christoph Hellwig wrote: > On Sun, Jun 26, 2005 at 12:21:52PM -0500, David Masover wrote: > >>I seem to remember the comment was more about hardcoded ID tables. >> >>And this was the generic ID code database, which is now maintained out >>of the kernel: >> >>/usr/share/misc/pci.ids >> A list of all known PCI ID's (vendors, devices, classes and sub-classes). >> >>That is what I was referring to, that used to be in the kernel. > > > And you once again showed that you don't understand what you're talking > about. Said database is a pci id to name mapping, which is completely > irrelevant for any driver. For things like your example there's very > little thing you can do but hardcoding a set of pci ids in one way or > another. I hope I admitted ahead of time that I don't understand what I'm talking about. For the record -- in this particular example, I don't. But, the distinction is irrelevant. PCI id to name mapping used to be (I think?) in the kernel, it was ugly. It may not have been where the chainsaw comment was, though. >>What this has to do with is whether you believe that it's better to keep >>code out until it's perfectly clean, or to let in code that has some > > > I doesn't need to be perfect, we just need it in a reasonable state and > have a buying that it's going to continue to evolve in the rigýt direction. > > And we're are very far from both of them in this ccase. I believe that we are closer to both of those than much of what's already in the kernel. Sorry I picked such a bad example, though. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQr7rLngHNmZLgCUhAQIl7A//QTNclAuPe0Q+qzZjT7uMXeWXVgxlKGbR cif4g55qtafpCA0m4/SkLYUmXpAL9Il384tCSK3vvnK7w8bQjGMCGk+xBeWQrDOC qvuyu+GOaZh+jCeo25IMm5rQRyxrsFb9d+0r+mclN2MDBmzn3l/lMiAIFq6NnlxZ gP0dOrCGpG1LXohfRhxLTphcmG1UMX/q7WbSSOCNAIMOxoqH2ez0ahdkJ44K+L45 hyMDtDugKCMeJw5r0No8RFl37h6bES/Q3DRv6Q1OQjbk4NYUKo9xCt69ypIvYRLJ SSNE63CRIjOiOn2sgii1q5kwNj+nShnMrl7bBTjtslMLoariWRPJwswwxMPJh6Nv xlovTxJKQnmg++KkJSZ6eDc7oCQapcGjeVxRCryHTIphuJ0OgRpw7xM4fsmOSj5R ruE1XrJZADage92NaozNvCASDh9wLdnkJG9cepJVMsZTb/emDQ54UbOv3yqwtHEm IbKnDNdSUVs0sgnLErWjRiQjfTh2xn0jof60mquVLufcJ0Ev9n7vWDTUBgKsFLVh xU4n9y+GWDqT31nql+si+pEJlBeYQCt4Sz9aci7Sni+sVDv929HnRbf4T9GTi99J 5RumBvrBRdfbo/SJos4ttxEQDxUFbIc92UJtYESyLzwH1SIl8lihqfN+fP/oPDdf unJVGXcgnMQ= =BJ6I -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 4:25 ` David Masover ` (2 preceding siblings ...) 2005-06-22 5:36 ` reiser4 plugins Christoph Hellwig @ 2005-06-22 9:56 ` Stefan Smietanowski 2005-06-22 14:00 ` Rik Van Riel 4 siblings, 0 replies; 620+ messages in thread From: Stefan Smietanowski @ 2005-06-22 9:56 UTC (permalink / raw) To: David Masover Cc: Jeff Garzik, Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi David > And here is the crucial point. Reiser4 is usable and useful NOW, not > after it has undergone massive surgery, and Namesys is bankrupt, and > users have given up and moved on to XFS. But the massive surgery should > happen eventually, partly to make all filesystems better (see below), > and partly to make the transition easier and more palatable for those > who don't work for Namesys. Sometimes someone comes with "I have this code NOW and if we just merge it I'll fix it up properly". It's rejected, stating "come back when you've fixed it up". We don't even have that here. We have "I have this code NOW and when it's merged I have no intention of fixing it up properly". In my eyes I'd rather take the first one than the second, there at least there's the INTENTION of fixing it up. Oh and btw, I'm not pro- or against- reiserfs in any way. I haven't tried reiserfs4 but I hear it's good so don't take this is a statement of my like or dislike of it, I'm just stating something worth thinking about. // Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (MingW32) iD8DBQFCuTXZBrn2kJu9P78RAi0JAJwIp6uyAd3AL4mgSBMAH59h7CrklQCfRW+V 8ZlZqxHwnpqpHa8OURtUEuw= =OD/E -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 4:25 ` David Masover ` (3 preceding siblings ...) 2005-06-22 9:56 ` Stefan Smietanowski @ 2005-06-22 14:00 ` Rik Van Riel 4 siblings, 0 replies; 620+ messages in thread From: Rik Van Riel @ 2005-06-22 14:00 UTC (permalink / raw) To: David Masover Cc: Jeff Garzik, Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel, ReiserFS List On Tue, 21 Jun 2005, David Masover wrote: > The point is, this was in the kernel for quite awhile, and it was so > ugly that someone would rather be fucked with a chainsaw. If something > that bad can make it in the kernel and stay for awhile because it > worked, and no one wanted to replace it I would like to think we could learn from the mistakes made in the past, instead of repeating them. Ugly code often is so ugly people don't *want* to fix it, so merging ugly code is often a big mistake. -- The Theory of Escalating Commitment: "The cost of continuing mistakes is borne by others, while the cost of admitting mistakes is borne by yourself." -- Joseph Stiglitz, Nobel Laureate in Economics ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 1:14 ` Jeff Garzik 2005-06-22 4:25 ` David Masover @ 2005-06-22 14:24 ` Alexander Zarochentsev 2005-06-22 14:29 ` Christoph Hellwig 1 sibling, 1 reply; 620+ messages in thread From: Alexander Zarochentsev @ 2005-06-22 14:24 UTC (permalink / raw) To: Jeff Garzik Cc: reiserfs-list, Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel On Wednesday 22 June 2005 05:14, Jeff Garzik wrote: > Hans Reiser wrote: > > Christoph, > > > > Reiser4 users love the plugin concept, and all audiences which have > > listened to a presentation on plugins have been quite positive about > > it. Many users think it is the best thing about reiser4. Can you > > articulate why you are opposed to plugins in more detail? Perhaps you > > are simply not as familiar with it as the audiences I have presented > > to. Perhaps persons on our mailing list can comment..... > > > > In particular, what is wrong with having a plugin id associated with > > every file, storing the pluginid on disk in permanent storage in the > > stat data, and having that plugin id define the set of methods that > > implement the vfs operations associated with a particular file, rather > > than defining VFS methods only at filesystem granularity? > > You're basically implementing another VFS layer inside of reiser4, which > is a big layering violation. Reiser4 suggests a bit more file types than VFS is aware of. I don't even think VFS should be. It is enough that VFS allows each inode/file/dentry to behave own way. IIRC VFS object's ->foo_ops is for that. Reiser plugins are for the same. Would you agree with reiser4 plugin design if the plugins will not dispatch VFS object methods calls by themselves but set ->foo_ops fileds instead? I guess you don't like to have the two dispatching systems at the same level. > This sort of feature should -not- be done at the low-level filesystem > level. > > What happens if people decide plugins are a good idea, and they want > them in ext3? We need massive surgery to extract the guts from reiser4.if > > Jeff Thanks, Alex ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 14:24 ` Alexander Zarochentsev @ 2005-06-22 14:29 ` Christoph Hellwig 2005-06-23 3:39 ` Hans Reiser ` (2 more replies) 0 siblings, 3 replies; 620+ messages in thread From: Christoph Hellwig @ 2005-06-22 14:29 UTC (permalink / raw) To: Alexander Zarochentsev Cc: Jeff Garzik, reiserfs-list, Hans Reiser, Christoph Hellwig, Andrew Morton, linux-kernel On Wed, Jun 22, 2005 at 06:24:32PM +0400, Alexander Zarochentsev wrote: > Reiser plugins are for the same. Would you agree with reiser4 plugin design > if the plugins will not dispatch VFS object methods calls by themselves but > set ->foo_ops fileds instead? I guess you don't like to have the two > dispatching systems at the same level. That is exactly the point I want to make. I haven't looked at the design in detail for a long time, but schemes to allow different object to have different operation vectors is a good idea. We already have that to varying degrees in all filesystems, and making that more formal is a good thing. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 14:29 ` Christoph Hellwig @ 2005-06-23 3:39 ` Hans Reiser 2005-06-26 16:46 ` Christoph Hellwig 2005-06-23 13:17 ` reiser4 plugins (the technical email in this flame fest) Hans Reiser 2005-06-23 17:31 ` reiser4 plugins Alexander Zarochentsev 2 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-23 3:39 UTC (permalink / raw) To: Christoph Hellwig Cc: Alexander Zarochentsev, Jeff Garzik, reiserfs-list, Andrew Morton, linux-kernel Correct me if I am wrong: What exists currently in VFS are vector instances, not classes. Plugins, selected by pluginids, are vector classes, with each pluginid selecting a vector class. You propose to have the vector class layer (aka plugin layer) in reiser4 export the vector instance to VFS for VFS to handle for each object, rather than having VFS select reiser4 and reiser4 selecting a vector class (aka plugin) which selects a method. If we agree on the above, then I will comment further. Christoph Hellwig wrote: >On Wed, Jun 22, 2005 at 06:24:32PM +0400, Alexander Zarochentsev wrote: > > >>Reiser plugins are for the same. Would you agree with reiser4 plugin design >>if the plugins will not dispatch VFS object methods calls by themselves but >>set ->foo_ops fileds instead? I guess you don't like to have the two >>dispatching systems at the same level. >> >> > >That is exactly the point I want to make. I haven't looked at the design >in detail for a long time, but schemes to allow different object to have >different operation vectors is a good idea. We already have that to >varying degrees in all filesystems, and making that more formal is a good >thing. > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 3:39 ` Hans Reiser @ 2005-06-26 16:46 ` Christoph Hellwig 2005-06-26 17:07 ` Artem B. Bityuckiy ` (2 more replies) 0 siblings, 3 replies; 620+ messages in thread From: Christoph Hellwig @ 2005-06-26 16:46 UTC (permalink / raw) To: Hans Reiser Cc: Alexander Zarochentsev, Jeff Garzik, reiserfs-list, Andrew Morton, linux-kernel On Wed, Jun 22, 2005 at 08:39:01PM -0700, Hans Reiser wrote: > Correct me if I am wrong: > > What exists currently in VFS are vector instances, not classes. Plugins, > selected by pluginids, are vector classes, with each pluginid selecting > a vector class. You propose to have the vector class layer (aka plugin > layer) in reiser4 export the vector instance to VFS for VFS to handle > for each object, rather than having VFS select reiser4 and reiser4 > selecting a vector class (aka plugin) which selects a method. > > If we agree on the above, then I will comment further. I'm a bit confused about what you're saying here. What does the 'vector' in all these expressions mean? In OO terminology our *_operation structures are interfaces. Each particular instance of such a struct that is filled with function pointers is a class. Each instance of an inode/file/dentry/superblock/... that uses these operation vectors is an object. What I propose (or rather demand ;-)) is that you don't duplicate this infrastructure, and add another dispath layer below these objects but instead re-use it for what it is done and only handle things specific to reiser4 in your own code. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 16:46 ` Christoph Hellwig @ 2005-06-26 17:07 ` Artem B. Bityuckiy 2005-06-26 17:25 ` Christoph Hellwig 2005-06-26 18:14 ` randy_dunlap 2005-06-26 23:42 ` Hans Reiser 2 siblings, 1 reply; 620+ messages in thread From: Artem B. Bityuckiy @ 2005-06-26 17:07 UTC (permalink / raw) To: Christoph Hellwig Cc: Hans Reiser, Alexander Zarochentsev, Jeff Garzik, reiserfs-list, Andrew Morton, linux-kernel Christoph Hellwig wrote: > I'm a bit confused about what you're saying here. What does the 'vector' > in all these expressions mean? > > In OO terminology our *_operation structures are interfaces. Each particular > instance of such a struct that is filled with function pointers is a class. > Each instance of an inode/file/dentry/superblock/... that uses these operation > vectors is an object. > > What I propose (or rather demand ;-)) is that you don't duplicate this > infrastructure, and add another dispath layer below these objects but instead > re-use it for what it is done and only handle things specific to reiser4 > in your own code. Just out of curiosity, could you please specify few exact examples with specific file/function names which duplicate the existing infrastructure. What do they duplicate and why? How should these functions be implemented on VFS? Ho should the the other FSes implement/ignore them? Why are you shure they will fit VFS well? etc. Thanks. -- Best regards, Artem B. Bityuckiy Oktet Labs (St. Petersburg), Software Engineer. +78124286709 (office) +79112449030 (mobile) E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 17:07 ` Artem B. Bityuckiy @ 2005-06-26 17:25 ` Christoph Hellwig 0 siblings, 0 replies; 620+ messages in thread From: Christoph Hellwig @ 2005-06-26 17:25 UTC (permalink / raw) To: Artem B. Bityuckiy Cc: Hans Reiser, Alexander Zarochentsev, Jeff Garzik, reiserfs-list, Andrew Morton, linux-kernel On Sun, Jun 26, 2005 at 09:07:00PM +0400, Artem B. Bityuckiy wrote: > Just out of curiosity, could you please specify few exact examples with > specific file/function names which duplicate the existing > infrastructure. What do they duplicate and why? How should these > functions be implemented on VFS? Ho should the the other FSes > implement/ignore them? Why are you shure they will fit VFS well? etc. Right now every file/inode/etc method in reiser4 is just a trivial wrapper around a plugin call. It should instead just set the method table directly in the plugin initialization. Example: static int reiser4_permission(struct inode *inode /* object */ , int mask, /* mode bits to check permissions * for */ struct nameidata *nameidata) { /* reiser4_context creation/destruction removed from here, because permission checks currently don't require this. Permission plugin have to create context itself if necessary. */ assert("nikita-1687", inode != NULL); return perm_chk(inode, mask, inode, mask); } besides a useless assert we just call into perm_chk, which is a macro obsfucation to call generic_permission which would be called if ->permission was zero. A hypothetical reiser4 "plugin" that would need redefine ->permission would just override it in a set inode_operations. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 16:46 ` Christoph Hellwig 2005-06-26 17:07 ` Artem B. Bityuckiy @ 2005-06-26 18:14 ` randy_dunlap 2005-06-26 23:42 ` Hans Reiser 2 siblings, 0 replies; 620+ messages in thread From: randy_dunlap @ 2005-06-26 18:14 UTC (permalink / raw) To: Christoph Hellwig; +Cc: reiser, zam, jgarzik, reiserfs-list, akpm, linux-kernel On Sun, 26 Jun 2005 17:46:06 +0100 Christoph Hellwig wrote: | On Wed, Jun 22, 2005 at 08:39:01PM -0700, Hans Reiser wrote: | > Correct me if I am wrong: | > | > What exists currently in VFS are vector instances, not classes. Plugins, | > selected by pluginids, are vector classes, with each pluginid selecting | > a vector class. You propose to have the vector class layer (aka plugin | > layer) in reiser4 export the vector instance to VFS for VFS to handle | > for each object, rather than having VFS select reiser4 and reiser4 | > selecting a vector class (aka plugin) which selects a method. | > | > If we agree on the above, then I will comment further. | | I'm a bit confused about what you're saying here. What does the 'vector' | in all these expressions mean? | | In OO terminology our *_operation structures are interfaces. Each particular | instance of such a struct that is filled with function pointers is a class. | Each instance of an inode/file/dentry/superblock/... that uses these operation | vectors is an object. | | What I propose (or rather demand ;-)) is that you don't duplicate this | infrastructure, and add another dispath layer below these objects but instead | re-use it for what it is done and only handle things specific to reiser4 | in your own code. I think that what Hans is trying to say is that the functionality that is handled by plugin_operations are a different dimension or vector from the inode/file/dentry etc. operations. Whether he is right or not is part of the discussion. and "plugin" is a bad name :( Is part of the problem that plugins are too extensible? I.e., the plugin_operations methods can do almost anything? I would want to see a list of methods that should be supported and then a reasonable infrastructure for supporting those. Pick one (like metadata) and drill down on how to support that. --- ~Randy ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 16:46 ` Christoph Hellwig 2005-06-26 17:07 ` Artem B. Bityuckiy 2005-06-26 18:14 ` randy_dunlap @ 2005-06-26 23:42 ` Hans Reiser 2005-06-27 8:57 ` Christoph Hellwig 2 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-26 23:42 UTC (permalink / raw) To: Christoph Hellwig, Alexander Zarochentcev Cc: Alexander Zarochentsev, Jeff Garzik, reiserfs-list, Andrew Morton, linux-kernel Christoph Hellwig wrote: >On Wed, Jun 22, 2005 at 08:39:01PM -0700, Hans Reiser wrote: > > >>Correct me if I am wrong: >> >>What exists currently in VFS are vector instances, not classes. Plugins, >>selected by pluginids, are vector classes, with each pluginid selecting >>a vector class. You propose to have the vector class layer (aka plugin >>layer) in reiser4 export the vector instance to VFS for VFS to handle >>for each object, rather than having VFS select reiser4 and reiser4 >>selecting a vector class (aka plugin) which selects a method. >> >>If we agree on the above, then I will comment further. >> >> > >I'm a bit confused about what you're saying here. What does the 'vector' >in all these expressions mean? > > Was your word, I thought, for the *_operation structures. >In OO terminology our *_operation structures are interfaces. Each particular >instance of such a struct that is filled with function pointers is a class. >Each instance of an inode/file/dentry/superblock/... that uses these operation >vectors is an object. > >What I propose (or rather demand ;-)) is that you don't duplicate this >infrastructure, and add another dispath layer below these objects but instead >re-use it for what it is done and only handle things specific to reiser4 >in your own code. > Well, that is very different from saying that we get rid of the plugin layer. So, rather than having a hierarchy, in which we first select filesystem, and then select plugin, VFS should treat each plugin as though it was a filesystem, if I understand you. Plugins and pluginids continue to exist with the exact same functionality, we just eliminate a function dereference for the *_operation structures. Let's see how it codes up in its details. Zam, can you work on this? Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-26 23:42 ` Hans Reiser @ 2005-06-27 8:57 ` Christoph Hellwig 0 siblings, 0 replies; 620+ messages in thread From: Christoph Hellwig @ 2005-06-27 8:57 UTC (permalink / raw) To: Hans Reiser Cc: Alexander Zarochentcev, Jeff Garzik, reiserfs-list, Andrew Morton, linux-kernel On Sun, Jun 26, 2005 at 04:42:35PM -0700, Hans Reiser wrote: > >I'm a bit confused about what you're saying here. What does the 'vector' > >in all these expressions mean? > > > > > Was your word, I thought, for the *_operation structures. We tend to use the term operation vectors, yes. But that's a different terminology that doesn't mix up very well with the OO terminology. > So, rather than having a hierarchy, in which we first select filesystem, > and then select plugin, VFS should treat each plugin as though it was a > filesystem, if I understand you. Kinda. The VFS doesn't have knowledge about what constitutes a file system driver, we register filesystems only to know what routine to call in on a mount with a particular filesystem type - the structure that does describe a filesystem (struct file_system_type) thus is very small. Starting from there the filesystem has always been able to use different methods for different objects (that's something very different from the vnode-based VFSes in the other UNIX variants). > Plugins and pluginids continue to exist > with the exact same functionality, we just eliminate a function > dereference for the *_operation structures. Let's see how it codes up in > its details. For the file and dir plugin that's pretty much that case. What's more problematic is your security plugins. With LSM we alredy have an infrastructure to provide pluggable access control. If there actually was an implementation of "security" (it's actually access control) for it besides the default unix permission one it should move to an LSM. You could even make your lsm part of the reiser4 module and poke into reiser4 directly from a technical point of view. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins (the technical email in this flame fest) 2005-06-22 14:29 ` Christoph Hellwig 2005-06-23 3:39 ` Hans Reiser @ 2005-06-23 13:17 ` Hans Reiser 2005-06-23 17:31 ` reiser4 plugins Alexander Zarochentsev 2 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-23 13:17 UTC (permalink / raw) To: Christoph Hellwig Cc: Alexander Zarochentsev, Jeff Garzik, reiserfs-list, Andrew Morton, linux-kernel Since this did not get an answer, and since it is the technical email in the flamefest, I thought I would resend it appropriately labeled. Correct me if I am wrong: What exists currently in VFS are vector instances, not classes. Plugins, selected by pluginids, are vector classes, with each pluginid selecting a vector class. You propose to have the vector class layer (aka plugin layer) in reiser4 export the vector instance to VFS for VFS to handle for each object, rather than having VFS select reiser4 and reiser4 selecting a vector class (aka plugin) which selects a method. If we agree on the above, then I will comment further. Christoph Hellwig wrote: >On Wed, Jun 22, 2005 at 06:24:32PM +0400, Alexander Zarochentsev wrote: > > >>Reiser plugins are for the same. Would you agree with reiser4 plugin design >>if the plugins will not dispatch VFS object methods calls by themselves but >>set ->foo_ops fileds instead? I guess you don't like to have the two >>dispatching systems at the same level. >> >> > >That is exactly the point I want to make. I haven't looked at the design >in detail for a long time, but schemes to allow different object to have >different operation vectors is a good idea. We already have that to >varying degrees in all filesystems, and making that more formal is a good >thing. > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 14:29 ` Christoph Hellwig 2005-06-23 3:39 ` Hans Reiser 2005-06-23 13:17 ` reiser4 plugins (the technical email in this flame fest) Hans Reiser @ 2005-06-23 17:31 ` Alexander Zarochentsev 2 siblings, 0 replies; 620+ messages in thread From: Alexander Zarochentsev @ 2005-06-23 17:31 UTC (permalink / raw) To: Christoph Hellwig; +Cc: reiserfs-list, Hans Reiser, Andrew Morton, linux-kernel [-- Attachment #1: Type: text/plain, Size: 1387 bytes --] On Wednesday 22 June 2005 18:29, Christoph Hellwig wrote: > On Wed, Jun 22, 2005 at 06:24:32PM +0400, Alexander Zarochentsev wrote: > > Reiser plugins are for the same. Would you agree with reiser4 plugin > > design if the plugins will not dispatch VFS object methods calls by > > themselves but set ->foo_ops fileds instead? I guess you don't like to > > have the two dispatching systems at the same level. > > That is exactly the point I want to make. I haven't looked at the design > in detail for a long time, but schemes to allow different object to have > different operation vectors is a good idea. By same level i meant the vfs-like portion of reiser4 object plugin (read/write/readdir ...). The rest of plugin can't be done by VFS, it is about reiser4-specific things. The way how vfs calls are passed to reiser4 objects can be re-written to use VFS features by 100% but i don't think it woild make the code better. What is done currently is 1) reduces places when VFS enters reiser4 -- better because each VFS method implementation needs at least reiser4 context initialization/destroying; 2) uses existing reiser4 plugin scheme -- it adds no complexity because reiser4 can't live without plugins. Can we leave the plugins as they are? > We already have that to > varying degrees in all filesystems, and making that more formal is a good > thing. Thanks, Alex. [-- Attachment #2: Type: text/html, Size: 1715 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 1:07 ` reiser4 plugins Hans Reiser 2005-06-22 1:14 ` Jeff Garzik @ 2005-06-22 1:18 ` Andrew Morton 2005-06-22 14:56 ` Christophe Saout 2005-06-25 18:55 ` Alexander Zarochentsev 2005-06-22 5:34 ` Christoph Hellwig 2 siblings, 2 replies; 620+ messages in thread From: Andrew Morton @ 2005-06-22 1:18 UTC (permalink / raw) To: Hans Reiser; +Cc: hch, jgarzik, linux-kernel, reiserfs-list Hans Reiser <reiser@namesys.com> wrote: > > What is wrong with having an encryption plugin implemented in this > manner? What is wrong with being able to have some files implemented > using a compression plugin, and others in the same filesystem not. > > What is wrong with having one file in the FS use a write only plugin, in > which the encrypion key is changed with every append in a forward but > not backward computable manner, and in order to read a file you must > either have a key that is stored on another computer or be reading what > was written after the moment of cracking root? > > What is wrong with having a set of critical data files use a CRC > checking file plugin? I think the concern here is that this is implemented at the wrong level. In Linux, a filesystem is some dumb thing which implements address_space_operations, filesystem_operations, etc. Advanced features such as those which you describe are implemented on top of the filesystem, not within it. reiser4 turns it all upside down. Now, some of the features which you envision are not amenable to above-the-fs implementations. But some will be, and that's where we should implement those. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 1:18 ` Andrew Morton @ 2005-06-22 14:56 ` Christophe Saout 2005-06-22 15:10 ` Artem B. Bityuckiy 2005-06-23 19:00 ` Hans Reiser 2005-06-25 18:55 ` Alexander Zarochentsev 1 sibling, 2 replies; 620+ messages in thread From: Christophe Saout @ 2005-06-22 14:56 UTC (permalink / raw) To: Andrew Morton; +Cc: Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 5391 bytes --] Am Dienstag, den 21.06.2005, 18:18 -0700 schrieb Andrew Morton: > > What is wrong with having an encryption plugin implemented in this > > manner? What is wrong with being able to have some files implemented > > using a compression plugin, and others in the same filesystem not. > > > > What is wrong with having one file in the FS use a write only plugin, in > > which the encrypion key is changed with every append in a forward but > > not backward computable manner, and in order to read a file you must > > either have a key that is stored on another computer or be reading what > > was written after the moment of cracking root? > > > > What is wrong with having a set of critical data files use a CRC > > checking file plugin? > > I think the concern here is that this is implemented at the wrong level. > > In Linux, a filesystem is some dumb thing which implements > address_space_operations, filesystem_operations, etc. > > Advanced features such as those which you describe are implemented on top > of the filesystem, not within it. reiser4 turns it all upside down. > > Now, some of the features which you envision are not amenable to > above-the-fs implementations. But some will be, and that's where we should > implement those. Yes, but that would be difficult, probably worse. The name "plugins" is perhaps a bit misleading. These plugins are most of the time some sort client to the reiser4 on-disk database structure. The core code is does on-disk tree handling, journalling and these things. The plugins in turn glue this core database system to the rest of the system in order to make a filesystem of it. The "file plugin" tells the database how to store files. A compression plugins is more tricky. Files should be randomly accessible and if you write in the middle of the file the compressed block might change in size. For reiser4 this is not a problem since it just tells the underlying database "give me some room for 1234 bytes and insert it in the tree instead of the other block". The reiser4 core has totally different semantics than the VFS layer and I don't see how these things could be handled elsewhere in this simple way. The reiser4 core is more some sort of library that abstracts a block device and provides some sort of database API (which is highly optimized for filesystem purposes). The actual filesystem is then another layer on top of this. You could actually implement lots of different filesystems on top of that database core. The actual layer is a bit different though. The database core itself registers with the Linux VFS and then passes the calls down to one of the plugins which then calls back into the reiser4 core to do the actual database modification. I think this was the point that Christoph was criticizing the most. Currently it looks like this: ,--------------. ,--------------. VFS -------> | | ----> | | | /fs/reiser4/ | | .../plugins/ | blockdev <-- | | <---> | | `--------------' `--------------' So the reiser4 code is introducing another abstraction of the Linux VFS layer instead of letting the plugins define the Linux VFS ops directly. Which would look like this: ,--------------. VFS ------------------------------> | | ,--------------, | .../plugins/ | blockdev <-- | /fs/reiser4/ | <---> | | `--------------' `--------------' Which probably would be okay for most of the time except for some details (which could probably be solved otherwise). Actually the flow is not always this simple, usually the path goes back and forth multiple time between the core and the plugins. One of the features Hans is using is that there can be different kinds of files. The on-disk structure tells the core which of the plugins is responsible for the "database object" found on the disk. It could be a directory or a "stat data" (inode) or a file. The file itself could be handled by different plugins like one that stores the data directly or one that compresses it. reiser4 is different than other filesystems in that it uses a lot more abstraction levels. The database aspect and the semantic aspect of a traditional filesystems are strongly separated. To understand the code probably means a lot of work because it is a bit different. Some of the layering concerns may be right, other probably not. The plugins that add additional VFS semantics (that are currently disable) should most definitely not be implemented only inside the filesystem. I think Hans did this because it would have been a lot more work doing this at the proper layer (which means talking to people and a lot of politics...). The last time I looked at the code is a while ago, so if I'm wrong on something, please don't shoot me. The only thing I can say is that reiser4 has very stable for me (though I've gone back to reiser3 for most of my filesystems because I wanted acl/xattr). So here's my advice: Instead of insulting each other or doing pure marketing talk, please try to address each detail and explain why something has been done and if it's good or bad and if it should be changed and how. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 14:56 ` Christophe Saout @ 2005-06-22 15:10 ` Artem B. Bityuckiy 2005-06-22 15:55 ` mjt 2005-06-23 19:00 ` Hans Reiser 1 sibling, 1 reply; 620+ messages in thread From: Artem B. Bityuckiy @ 2005-06-22 15:10 UTC (permalink / raw) To: Christophe Saout Cc: Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list Christophe Saout wrote: > The plugins that add additional VFS semantics (that are currently > disable) should most definitely not be implemented only inside the > filesystem. I think Hans did this because it would have been a lot more > work doing this at the proper layer (which means talking to people and a > lot of politics...). I would even do s/should most definitely not/must not/ More filesystems in future may want to use these semantics. There are cases when one can't use Reiser4 to implement them, but instead, need to implement another file system with the same semantics. -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 15:10 ` Artem B. Bityuckiy @ 2005-06-22 15:55 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-22 15:55 UTC (permalink / raw) To: Artem B. Bityuckiy Cc: Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 892 bytes --] On Wed, Jun 22, 2005 at 07:10:58PM +0400, Artem B. Bityuckiy wrote: > >More filesystems in future may want to use these semantics. There are >cases when one can't use Reiser4 to implement them, but instead, need to >implement another file system with the same semantics. So merge it as it is and move the stuff to the VFS as needed or deemed necessary. And enable the pseudo interface, or at least set it in menuconfig and enable by default, it needs testing too. Then someone says "we can't implement this in the fs" and someone else says "we can't implement this in the vfs" and someone else says "this is a good thing which we want but you won't let us" and we stagnate again... Isn't this bickering getting a bit old? After all, it seems the code is merged in -mm, the big issues are fixed and all should be ready for more real-life testing and such? -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-22 15:55 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-22 15:55 UTC (permalink / raw) To: Artem B. Bityuckiy Cc: Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 892 bytes --] On Wed, Jun 22, 2005 at 07:10:58PM +0400, Artem B. Bityuckiy wrote: > >More filesystems in future may want to use these semantics. There are >cases when one can't use Reiser4 to implement them, but instead, need to >implement another file system with the same semantics. So merge it as it is and move the stuff to the VFS as needed or deemed necessary. And enable the pseudo interface, or at least set it in menuconfig and enable by default, it needs testing too. Then someone says "we can't implement this in the fs" and someone else says "we can't implement this in the vfs" and someone else says "this is a good thing which we want but you won't let us" and we stagnate again... Isn't this bickering getting a bit old? After all, it seems the code is merged in -mm, the big issues are fixed and all should be ready for more real-life testing and such? -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 15:55 ` mjt @ 2005-06-22 16:20 ` Artem B. Bityuckiy -1 siblings, 0 replies; 620+ messages in thread From: Artem B. Bityuckiy @ 2005-06-22 16:20 UTC (permalink / raw) To: Markus TЖrnqvist Cc: Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list Markus TЖrnqvist wrote: > So merge it as it is and move the stuff to the VFS as needed or > deemed necessary. And enable the pseudo interface, or at least > set it in menuconfig and enable by default, it needs testing too. Reiser4 has a number of great (IMO) things like file as directory, atomic operations, different kinds of stat data, fibretions, etc, etc. Some thing is not yet ready - doesn't matter. Some of this is of general interest, some is Reiser4-dedicated. New interfaces are needed to allow users to utilize that all. My point is that the things that are of general interest must not be Reiser4-only. For example, I should have a possibility to implement files-like-dir in _another_ FS using the same interfaces that Reiser4 uses. That's all I wanted to say. The other question that it may be difficult to foresee everything and it may make sense to move some things upper in future. -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-22 16:20 ` Artem B. Bityuckiy 0 siblings, 0 replies; 620+ messages in thread From: Artem B. Bityuckiy @ 2005-06-22 16:20 UTC (permalink / raw) To: Markus TЖrnqvist Cc: Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list Markus Törnqvist wrote: > So merge it as it is and move the stuff to the VFS as needed or > deemed necessary. And enable the pseudo interface, or at least > set it in menuconfig and enable by default, it needs testing too. Reiser4 has a number of great (IMO) things like file as directory, atomic operations, different kinds of stat data, fibretions, etc, etc. Some thing is not yet ready - doesn't matter. Some of this is of general interest, some is Reiser4-dedicated. New interfaces are needed to allow users to utilize that all. My point is that the things that are of general interest must not be Reiser4-only. For example, I should have a possibility to implement files-like-dir in _another_ FS using the same interfaces that Reiser4 uses. That's all I wanted to say. The other question that it may be difficult to foresee everything and it may make sense to move some things upper in future. -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 16:20 ` Artem B. Bityuckiy (?) @ 2005-06-22 16:46 ` M. 2005-06-22 16:54 ` mjt -1 siblings, 1 reply; 620+ messages in thread From: M. @ 2005-06-22 16:46 UTC (permalink / raw) To: Artem B. Bityuckiy Cc: Markus TЖrnqvist, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list Hi, Is it not simpler to ask the reiserfs guys for a detailed explanation of why and where this plugins' layer differs from using VFS for plugins and let others comment on that ? If something cant be done using VFS this layer is needed by reiser4 and has to be merged. Michele ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 16:46 ` M. @ 2005-06-22 16:54 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-22 16:54 UTC (permalink / raw) To: M. Cc: Artem B. Bityuckiy, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 863 bytes --] On Wed, Jun 22, 2005 at 06:46:50PM +0200, M. wrote: > >Is it not simpler to ask the reiserfs guys for a detailed explanation >of why and where this plugins' layer differs from using VFS for >plugins and let others comment on that ? I hope this is not FUD or something like that, but it seems to me the VFS guys are not too willing to implement Reiser4-HiFi stuff anywhere else, like the VFS, and don't want Reiser4 in, because it duplicates. Think of the chicken and the egg. I may be wrong, though. >If something cant be done using VFS this layer is needed by reiser4 >and has to be merged. But it's still possible to let Reiser4 in and take them, say, a feature at a time to the VFS. Unless it's going to be a political wankfest/pissing contest about if things like this should even be implemented, anywhere, ever :P -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-22 16:54 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-22 16:54 UTC (permalink / raw) To: M. Cc: Artem B. Bityuckiy, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 863 bytes --] On Wed, Jun 22, 2005 at 06:46:50PM +0200, M. wrote: > >Is it not simpler to ask the reiserfs guys for a detailed explanation >of why and where this plugins' layer differs from using VFS for >plugins and let others comment on that ? I hope this is not FUD or something like that, but it seems to me the VFS guys are not too willing to implement Reiser4-HiFi stuff anywhere else, like the VFS, and don't want Reiser4 in, because it duplicates. Think of the chicken and the egg. I may be wrong, though. >If something cant be done using VFS this layer is needed by reiser4 >and has to be merged. But it's still possible to let Reiser4 in and take them, say, a feature at a time to the VFS. Unless it's going to be a political wankfest/pissing contest about if things like this should even be implemented, anywhere, ever :P -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 16:20 ` Artem B. Bityuckiy @ 2005-06-22 17:33 ` Horst von Brand -1 siblings, 0 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-22 17:33 UTC (permalink / raw) To: Artem B. Bityuckiy Cc: Markus TЖrnqvist, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain, Size: 2883 bytes --] Artem B. Bityuckiy <dedekind@yandex.ru> wrote: > Markus TЖrnqvist wrote: > > So merge it as it is Fix it first. The "merge as it stands" just gives rise to stuff that is /never/ fixed properly. > > and move the stuff to the VFS as needed or > > deemed necessary. And enable the pseudo interface, or at least > > set it in menuconfig and enable by default, it needs testing too. Then test it out of the standard tree... > Reiser4 has a number of great (IMO) things like file as directory, Urgh. > atomic operations, What is atomic that isn't in the standard filesystems? How do you guarantee it doesn't stop the system dead in its tracks waiting for some very long transaction to finish? > different kinds of stat data, Usefulness? Sounds like kernel bloat leading to userspace bloat and applications/users wondering what the heck goes on when they don't grok the particular stat format. > fibretions, etc, ??? > etc. Some thing is not yet ready - doesn't matter. Some of this is of > general interest, some is Reiser4-dedicated. I don't see anything that would interest me at least, so you can safely scratch the "general interest" part. > New interfaces are needed to allow users to utilize that all. That is a quite strong argument /against/ it all in my book. It means bloat in /every/ filesystem, and they have shown to be able to do without for some 30 years now. I'd need /very/ strong reasons for adding something. > My point > is that the things that are of general interest must not be > Reiser4-only. Reiser4-only stuff is of very limited use, if it isn't just internal stuff. And that doesn't need any changes. > For example, I should have a possibility to implement > files-like-dir in _another_ FS using the same interfaces that Reiser4 > uses. That's all I wanted to say. It has been argued over and over that that particular feature /can't/ be implemented sanely anyway, so it has to stay out. Besides not making any sense. "You've got files and directories" is a bit asymetrical and so not quite nice; "all you have is directories" is symmetrical, estetic, and completely useless; "some files are directories, some aren't; files in file-directories are different than regular files in directory-directories" is just a mindless jumble. > The other question that it may be difficult to foresee everything and > it may make sense to move some things upper in future. Another good reason to keep it out ;-) -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-22 17:33 ` Horst von Brand 0 siblings, 0 replies; 620+ messages in thread From: Horst von Brand @ 2005-06-22 17:33 UTC (permalink / raw) To: Artem B. Bityuckiy Cc: Markus TЖrnqvist, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list Artem B. Bityuckiy <dedekind@yandex.ru> wrote: > Markus TЖrnqvist wrote: > > So merge it as it is Fix it first. The "merge as it stands" just gives rise to stuff that is /never/ fixed properly. > > and move the stuff to the VFS as needed or > > deemed necessary. And enable the pseudo interface, or at least > > set it in menuconfig and enable by default, it needs testing too. Then test it out of the standard tree... > Reiser4 has a number of great (IMO) things like file as directory, Urgh. > atomic operations, What is atomic that isn't in the standard filesystems? How do you guarantee it doesn't stop the system dead in its tracks waiting for some very long transaction to finish? > different kinds of stat data, Usefulness? Sounds like kernel bloat leading to userspace bloat and applications/users wondering what the heck goes on when they don't grok the particular stat format. > fibretions, etc, ??? > etc. Some thing is not yet ready - doesn't matter. Some of this is of > general interest, some is Reiser4-dedicated. I don't see anything that would interest me at least, so you can safely scratch the "general interest" part. > New interfaces are needed to allow users to utilize that all. That is a quite strong argument /against/ it all in my book. It means bloat in /every/ filesystem, and they have shown to be able to do without for some 30 years now. I'd need /very/ strong reasons for adding something. > My point > is that the things that are of general interest must not be > Reiser4-only. Reiser4-only stuff is of very limited use, if it isn't just internal stuff. And that doesn't need any changes. > For example, I should have a possibility to implement > files-like-dir in _another_ FS using the same interfaces that Reiser4 > uses. That's all I wanted to say. It has been argued over and over that that particular feature /can't/ be implemented sanely anyway, so it has to stay out. Besides not making any sense. "You've got files and directories" is a bit asymetrical and so not quite nice; "all you have is directories" is symmetrical, estetic, and completely useless; "some files are directories, some aren't; files in file-directories are different than regular files in directory-directories" is just a mindless jumble. > The other question that it may be difficult to foresee everything and > it may make sense to move some things upper in future. Another good reason to keep it out ;-) -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 17:33 ` Horst von Brand (?) @ 2005-06-22 21:51 ` David Masover 2005-06-22 22:23 ` Nikita Danilov 2005-06-22 22:27 ` Roland Dreier -1 siblings, 2 replies; 620+ messages in thread From: David Masover @ 2005-06-22 21:51 UTC (permalink / raw) To: Horst von Brand Cc: Artem B. Bityuckiy, Markus TЖrnqvist, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Horst von Brand wrote: > Artem B. Bityuckiy <dedekind@yandex.ru> wrote: > >>Markus TЖrnqvist wrote: [...] >>> and move the stuff to the VFS as needed or >>>deemed necessary. And enable the pseudo interface, or at least >>>set it in menuconfig and enable by default, it needs testing too. > > > Then test it out of the standard tree... Standard tree is actually a good place to test things. That's why there are so many things there that say "EXPERIMENTAL", and so many of those don't work. Some even refuse to compile. >>atomic operations, > > > What is atomic that isn't in the standard filesystems? How do you guarantee We've been discussing this for quite awhile. In standard filesystems, the only things that are atomic are metadata. If I rename a file and crash, I'm guarenteed to either have renamed the file or not, and not be caught halfway. This does not apply to files. In fact, the only way to perform an atomic write on a file, using the filesystem's atomicity, is to write a new file, nuke the old one, and rename the new one on top of the old one. What we want is to have programs that can write small changes to one file or to many files, lump all those changes into a transaction, and have the transaction either succeed or fail. > it doesn't stop the system dead in its tracks waiting for some very long > transaction to finish? We've also discussed this. For one thing, if we can have transactions in databases which don't stop the database dead in its tracks, why can't we do it with filesystems? But anyway, if you really want to know, ask someone else or read the archives. I wasn't really paying attention except to remember that this issue was resolved. >> different kinds of stat data, > > > Usefulness? Sounds like kernel bloat leading to userspace bloat and > applications/users wondering what the heck goes on when they don't grok the > particular stat format. So you allow multiple stat formats. Bloat is not as big an issue here as the bloat of existing systems which run on top of the FS and don't cooperate. Gnome and KDE each have their own VFS, for instance. >> fibretions, etc, > > > ??? Low-level tweaking. I think the word is from some sort of calculus. >>etc. Some thing is not yet ready - doesn't matter. Some of this is of >>general interest, some is Reiser4-dedicated. > > > I don't see anything that would interest me at least, so you can safely > scratch the "general interest" part. You're the sole general public? >>New interfaces are needed to allow users to utilize that all. > > > That is a quite strong argument /against/ it all in my book. It means bloat > in /every/ filesystem, and they have shown to be able to do without for > some 30 years now. I'd need /very/ strong reasons for adding something. Spotlight on the Mac. Users love it. We can do it. But not without changing something in the filesystem. Actually, I think we came up with several ways to do this, all of which required Reiser4 interfaces. It's "bloat" until you need it. >> My point >>is that the things that are of general interest must not be >>Reiser4-only. > > > Reiser4-only stuff is of very limited use, if it isn't just internal > stuff. And that doesn't need any changes. Only it does. Reiser4 can't get into the kernel because it duplicates the VFS in order to extend it. It couldn't get in before because it extended the VFS directly. Maybe you just don't want this system to EVER get in, no matter what they do to it? >> For example, I should have a possibility to implement >>files-like-dir in _another_ FS using the same interfaces that Reiser4 >>uses. That's all I wanted to say. > > > It has been argued over and over that that particular feature /can't/ be > implemented sanely anyway, so it has to stay out. Besides not making any It's been dropped for the moment, but it's been argued just as many times that it can be done sanely. > sense. "You've got files and directories" is a bit asymetrical and so not > quite nice; "all you have is directories" is symmetrical, estetic, and > completely useless; "some files are directories, some aren't; files in > file-directories are different than regular files in directory-directories" > is just a mindless jumble. Or you can just say "There are no files. There are no directories. There are only objects, which contain a chunk of linear data and other objects." If you actually go read the whitepaper, you will discover that this is actually a cleaner, more esthetic, and more useful model than the current one. It's just a little harder to do. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrndSHgHNmZLgCUhAQKOCxAAlhBiu234fmMaUv7kYp0Xt1V2yjP67c8U ggUbB6sSac1CDo3XQm3cWcR9jqVTTxDCZx012XVEM2MQ7n69YF6+xTxFK/wS69rO cFh7ChBDe71YOB3aBQ1ojG5D/cM8G4VPVF3pt7ywqoEK67VNwynLvdjjWYYD5w9y mZd+eRo6kpCmjakNwOpU4LOA8u3C+OK3o2bduzFSMquV9dckV4vt8rxTi1R1l12s aTBuGc1yvye19vo6DQv28hVcqbv7GmyA5+oBg+TdsFF6Oy/+oR5h4bsfDlFR6Ycs u23K0yS1b+Iivszqb/wM4CewmkiTOKlxjlR4t93isfsRM1alVeFTfsVP3OYBVRwO 42upk/yzVvliP32n5smZVssf99UK0LuXpb+NTIXABm7WDOtcGV1gC1zz93g0NcBw Huh82+OPH1l8JXn+4iuZsHBx2VtG5Q7ZENGqXRcEZMb+bYECzFhPNVYBWlPBK+LC rk/byrkO6zKBkQ5mh7BeK/JrcKVR/IEH+OjpSPmwuam5oPSaoHt9ep5zV+oFYCU6 KIRh8/p7br9i6A2WKUMs+v8j+LcA+Lg+8fbW7RC5LXjG6+QpwvBTuZGU5ykRbj2i KmrQPCg3x3ZXDGkMLEUySPFL0P3JHpLxsf6tfVEZtsyXCHn1MOeo5sS13NLU719P 6wYcB+VAbSE= =0NX7 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 21:51 ` David Masover @ 2005-06-22 22:23 ` Nikita Danilov 2005-06-23 14:25 ` David Masover 2005-06-22 22:27 ` Roland Dreier 1 sibling, 1 reply; 620+ messages in thread From: Nikita Danilov @ 2005-06-22 22:23 UTC (permalink / raw) To: David Masover Cc: Artem B. Bityuckiy, Markus TЖrnqvist, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list David Masover writes: [...] > > What we want is to have programs that can write small changes to one > file or to many files, lump all those changes into a transaction, and > have the transaction either succeed or fail. No existing file system guarantees such behavior. Even atomicity of single system call is not guaranteed. > > > it doesn't stop the system dead in its tracks waiting for some very long > > transaction to finish? > > We've also discussed this. For one thing, if we can have transactions > in databases which don't stop the database dead in its tracks, why can't > we do it with filesystems? Because to have such transactions databases pay huge price in both resource consumption and available concurrency (isolation, commit-time locks, etc.), and yet mechanism they use to deal with stuck transactions (which is simply to abort it) is not very suitable for the file system. > > But anyway, if you really want to know, ask someone else or read the > archives. I wasn't really paying attention except to remember that this > issue was resolved. That would be real breakthrough. [...] > > >> fibretions, etc, > > > > > > ??? > > Low-level tweaking. I think the word is from some sort of calculus. Fibration. http://marc.theaimsgroup.com/?l=linux-kernel&m=108032604606183&w=2 Nikita. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 22:23 ` Nikita Danilov @ 2005-06-23 14:25 ` David Masover 2005-06-23 15:12 ` Hans Reiser 2005-06-23 22:31 ` Nikita Danilov 0 siblings, 2 replies; 620+ messages in thread From: David Masover @ 2005-06-23 14:25 UTC (permalink / raw) To: Nikita Danilov Cc: Artem B. Bityuckiy, Markus TЖrnqvist, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Nikita Danilov wrote: > David Masover writes: > > [...] > > > > > What we want is to have programs that can write small changes to one > > file or to many files, lump all those changes into a transaction, and > > have the transaction either succeed or fail. > > No existing file system guarantees such behavior. Even atomicity of > single system call is not guaranteed. No _existing_ filesystem. But I seem to recall that this was one of the design decisions of Reiser4, and that the system call itself was pushed off to 4.1? Maybe I'm just wrong about how big a transaction can be. Maybe it was limited to a single file. I don't think so, though. From the whitepaper: "Stuffing a transaction into a single file just because you need the transaction to be atomic is hardly what one would call flexible semantics." I also seem to recall that the rolling back of the transaction, should it fail, was supposed to be handled by the application. This doesn't quite click with the whitepaper, but it could work. More whitepaper goodness: "A new system call sys_reiser4() will be implemented to support applications that don't have to be fooled into thinking that they are using POSIX. Through this entry point a richer set of semantics will access the same files that are also accessible using POSIX calls. Reiser4() will not implement more than hierarchical names. A full set theoretic naming system as described on our future vision page will not be implemented before Reiser6() is implemented (Reiser5 is our distributed filesystem, Reiser6 is our enhanced semantics, whether we implement Reiser5 or Reiser6 first depends on which sponsors we find ;-) ). Reiser4() will implement all features necessary to access ACLs as files/directories rather than as something neither file nor directory. These include opening and closing transactions, performing a sequence of I/Os in one system call, and accessing files without use of file descriptors (necessary for efficient small I/O). Reiser4 will use a syntax suitable for evolving into Reiser5() syntax with its set theoretic naming." So, some sort of transaction is planned. But, as I said, I wasn't paying enough attention. Maybe there is a technical reason why this can't be done in Linux? > > > it doesn't stop the system dead in its tracks waiting for some very long > > > transaction to finish? > > > > We've also discussed this. For one thing, if we can have transactions > > in databases which don't stop the database dead in its tracks, why can't > > we do it with filesystems? > > Because to have such transactions databases pay huge price in both > resource consumption and available concurrency (isolation, commit-time > locks, etc.), and yet mechanism they use to deal with stuck transactions > (which is simply to abort it) is not very suitable for the file system. Oh, really? If we've got application support through sys_reiser4? The application should be ready to deal with a transaction abort. I'm still not convinced of any of that paragraph. I don't know enough to argue the point, but it intuitively feels wrong. After all, if the metadata is atomic, and we are allowed to make our own system calls, why can't we make the data atomic? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQrrGOngHNmZLgCUhAQJZhw//dmJ6S2GlGT6J5YI9DTCyoTDIPUYNb8o0 M1me6KDTElzzQ3yUak/eUd0sbGBAQcf0vn/iVscfq2DoAwnUWxHjht+PaOA98axR 0pnofqE291QLTQJ36epW0kKqFjavVVsrpD80llcaCFz9Rq48W40DoI5CWuX1RQqK pCnr9vYe8cAsRY+PzV9/KUaSQ+eZJ9daLsAmMwA3Gcxo4XYqILlZm90X3QQTdc8W gnKSabG3zIjEozfgG/nvtV/09mktHINGq3ud8W1XubBOXs4z+ECsLyvi7QNW83Bq b/wTDUX3PkrjDHnfcmFkFZJqRrCBD9Ko36f9NThxuaba5eV7kb6h+qx+kS5ZM6Lm bh90TjJrIpJ4aQr2qrPRAE85GSnvSlyi3E01gk/+UnkBFMoTqTvw2dPb0GhvMINM EhSUhEyeaopWXIdv3IszOOpbHJLwczixLDBtZ8OFDS26bnJGj7YlnTjdf+TZ9CGf ZXn7GaG16CiSTOt0YkKk2UGZz+AOubPAUHc6v8Wg587qXWKD3cXVQeZVqYwJG/8B G5qq51LB6jypjAoP4uSeuTs4DfANGi2H2mHjZVAyaGcwzhxf3ffGRFkfjElAj8RA RdmB6bq10nt32mL7YP8+n7xa38iCP+ks9wsoOY2KBBDlOpHu07xb/c2DS9yfTCpj FvMShQw6EBI= =S7ds -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 14:25 ` David Masover @ 2005-06-23 15:12 ` Hans Reiser 2005-06-23 22:31 ` Nikita Danilov 1 sibling, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-23 15:12 UTC (permalink / raw) To: David Masover Cc: Nikita Danilov, Artem B. Bityuckiy, Markus TЖrnqvist, Christophe Saout, Andrew Morton, hch, jgarzik, linux-kernel, reiserfs-list David Masover wrote: > Nikita Danilov wrote: > > >David Masover writes: > > >[...] > > > > > > > What we want is to have programs that can write small changes to one > > > file or to many files, lump all those changes into a transaction, and > > > have the transaction either succeed or fail. > > >No existing file system guarantees such behavior. Even atomicity of > >single system call is not guaranteed. > > > No _existing_ filesystem. But I seem to recall that this was one of the > design decisions of Reiser4, and that the system call itself was pushed > off to 4.1? Thats right. > > Maybe I'm just wrong about how big a transaction can be. No, you are not. Isolation is not easy for an fs, but fusing multiple atoms from multiple file modifications into one "guaranteed to commit or fail all together" atom is easy. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 14:25 ` David Masover 2005-06-23 15:12 ` Hans Reiser @ 2005-06-23 22:31 ` Nikita Danilov 2005-06-24 6:37 ` Hans Reiser 1 sibling, 1 reply; 620+ messages in thread From: Nikita Danilov @ 2005-06-23 22:31 UTC (permalink / raw) To: David Masover Cc: Artem B. Bityuckiy, Markus TЖrnqvist, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list David Masover writes: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Nikita Danilov wrote: > > David Masover writes: > > > > [...] > > > > > > > > What we want is to have programs that can write small changes to one > > > file or to many files, lump all those changes into a transaction, and > > > have the transaction either succeed or fail. > > > > No existing file system guarantees such behavior. Even atomicity of > > single system call is not guaranteed. > > No _existing_ filesystem. But I seem to recall that this was one of the > design decisions of Reiser4, and that the system call itself was pushed > off to 4.1? First off, my comment was in response to the message http://marc.theaimsgroup.com/?l=linux-kernel&m=111945793205480&w=2 claiming that reiser4 was "atomic". As it stands, currently reiser4 cannot guarantee that single write(2) call is atomic. Think of system with X bytes of physical memory and 9*X bytes of swap. User does buf = malloc(10 * X); memset(buf, 42, 10 * X); write(reiser4_fd, buf, 10 * X); As far as I know, current reiser4 code cannot guarantee that such write is always handled as a single transaction. It is _possible_ to implement such a guarantee, I believe, but this requires a lot of work. > > Maybe I'm just wrong about how big a transaction can be. Maybe it was > limited to a single file. I don't think so, though. From the > whitepaper: "Stuffing a transaction into a single file just because you > need the transaction to be atomic is hardly what one would call flexible > semantics." I don't quite understand what do you mean. Whitepaper describes how things work according to some grand visionary scheme. It doesn't go to the level of technical details, it doesn't discuss possible obstacles. There is a huge gap between saying "it's nice to have user-visible transactions" and having ones. So huge, in my opinion, that compared with it, difference in reiser4 and, say, ext3 as possible starting points is immaterial. > > I also seem to recall that the rolling back of the transaction, should > it fail, was supposed to be handled by the application. This doesn't > quite click with the whitepaper, but it could work. What if user level application err... crashed (does this happen?) or stuck in the infinite loop (hmm... user level programmers never do this, I hope)? [...] > > > > Because to have such transactions databases pay huge price in both > > resource consumption and available concurrency (isolation, commit-time > > locks, etc.), and yet mechanism they use to deal with stuck transactions > > (which is simply to abort it) is not very suitable for the file system. > > Oh, really? If we've got application support through sys_reiser4? The > application should be ready to deal with a transaction abort. Transactional systems have to deal with so-called external actions. E.g., in the ATM external action is to hand cash out to the person doing withdraw. After external action was done as part of transaction, that transaction can no longer be aborted (because one cannot reverse external action). Typical example of file system transactions is mail server doing something roughly following: (0) begin work; (1) take message from the queue; (2) send message to the recipient; (3) wait for confirmation; (4) remove message; (5) commit work; Here (2), and (3) are external actions, they cannot be aborted or reversed. This means that should transaction abort at the step (4) system would be left in the inconsistent state. > > I'm still not convinced of any of that paragraph. I don't know enough > to argue the point, but it intuitively feels wrong. After all, if the > metadata is atomic, and we are allowed to make our own system calls, why > can't we make the data atomic? Because meta-data are under tight kernel control. There is a difference between implementing transaction engine for use by a small body of trusted code (kernel) and a general purpose transaction manager. Nikita. > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.1 (GNU/Linux) > Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org > > iQIVAwUBQrrGOngHNmZLgCUhAQJZhw//dmJ6S2GlGT6J5YI9DTCyoTDIPUYNb8o0 > M1me6KDTElzzQ3yUak/eUd0sbGBAQcf0vn/iVscfq2DoAwnUWxHjht+PaOA98axR > 0pnofqE291QLTQJ36epW0kKqFjavVVsrpD80llcaCFz9Rq48W40DoI5CWuX1RQqK > pCnr9vYe8cAsRY+PzV9/KUaSQ+eZJ9daLsAmMwA3Gcxo4XYqILlZm90X3QQTdc8W > gnKSabG3zIjEozfgG/nvtV/09mktHINGq3ud8W1XubBOXs4z+ECsLyvi7QNW83Bq > b/wTDUX3PkrjDHnfcmFkFZJqRrCBD9Ko36f9NThxuaba5eV7kb6h+qx+kS5ZM6Lm > bh90TjJrIpJ4aQr2qrPRAE85GSnvSlyi3E01gk/+UnkBFMoTqTvw2dPb0GhvMINM > EhSUhEyeaopWXIdv3IszOOpbHJLwczixLDBtZ8OFDS26bnJGj7YlnTjdf+TZ9CGf > ZXn7GaG16CiSTOt0YkKk2UGZz+AOubPAUHc6v8Wg587qXWKD3cXVQeZVqYwJG/8B > G5qq51LB6jypjAoP4uSeuTs4DfANGi2H2mHjZVAyaGcwzhxf3ffGRFkfjElAj8RA > RdmB6bq10nt32mL7YP8+n7xa38iCP+ks9wsoOY2KBBDlOpHu07xb/c2DS9yfTCpj > FvMShQw6EBI= > =S7ds > -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-23 22:31 ` Nikita Danilov @ 2005-06-24 6:37 ` Hans Reiser 2005-06-24 7:21 ` David Masover 2005-06-24 10:31 ` Nikita Danilov 0 siblings, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-24 6:37 UTC (permalink / raw) To: Nikita Danilov Cc: David Masover, Artem B. Bityuckiy, Markus TЖrnqvist, Christophe Saout, Andrew Morton, hch, jgarzik, linux-kernel, reiserfs-list Nikita, I respectfully disagree with what you say about the state of our atomicity code. It is not so far away as you describe, and probably 6 man weeks work could polish it off. You don't see the value in what I define as useful, namely atomicity without isolation. Since you don't see that, it is harder for you to see that something is close to working and just needs 6 weeks of someone who groks what I am asking for. Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 6:37 ` Hans Reiser @ 2005-06-24 7:21 ` David Masover 2005-06-24 11:13 ` Bernd Eckenfels 2005-06-24 10:31 ` Nikita Danilov 1 sibling, 1 reply; 620+ messages in thread From: David Masover @ 2005-06-24 7:21 UTC (permalink / raw) To: Hans Reiser Cc: Nikita Danilov, Artem B. Bityuckiy, Markus TЖrnqvist, Christophe Saout, Andrew Morton, hch, jgarzik, linux-kernel, reiserfs-list -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hans Reiser wrote: > Nikita, I respectfully disagree with what you say about the state of our > atomicity code. It is not so far away as you describe, and probably 6 > man weeks work could polish it off. You don't see the value in what I > define as useful, namely atomicity without isolation. Since you don't > see that, it is harder for you to see that something is close to working > and just needs 6 weeks of someone who groks what I am asking for. How about a poor-man's isolation -- any process other than that responsible for the transaction sees a consistent state, never a transaction-in-progress. I'm sure there's a name for that. So, until the transaction commits, all apps can still see the original state, except the one actually writing the transaction. After it's done, all apps see the new consistent state. Isolation can then be done easily in userspace, via voluntary locking. If the isolation needs to be global, the user can take a performance hit and do mandatory locking. Either way, locking and unlocking could be seen as part of the transaction. Details follow, most of them pure speculation on my part... I'm not sure how this ties into the bigger kind of transaction, as you've decided to let userspace handle the rollback in the large, multi-file transactions, but for a single file, it seems obvious (famous last words). You've got the wandering stuff, meaning that the new file (or changes to a file) are written to a new space on disk, after which you update the metadata (blocklist?) -- presumably atomically. So, it already seems inherent in the system, but it should be defined. For some transactions, it doesn't make sense to do this, especially huge ones (where storing changes separately may well fill up the disk) which make more sense to have an application handle the rollback. Here's a bad example of such a situation: We're doing a brand-new install, and we start by unpacking a tarball from the installation media. After the tarball is unpacked, we start installing packages. But, if the installation fails, maybe we want to roll back to the initial tarball, change some installation parameters, and try again. In such a situation, it makes more sense to just unpack the tarball again than to try to roll back each change individually. This is a bad example because you shouldn't have to start over from the beginning, and even if you did, exit codes should be enough for this kind of installer. Anyway, I'm not sure quite how this works when dealing with more than one file, but we already have atomic metadata operations, and here we have an example of a somewhat atomic file operation, so I'm sure the two can be stringed together... somehow... Conceptually, it seems you'd just be building a COW subtree, and the actual "atomic" part consists of simply changing the pointer from the original tree to the modified one. The cleanup of the original tree need not be atomic. We do have COW support, don't we? It was discussed, and we should. Main point is, I've just described two kinds of transactions: entirely FS-managed and userland-rolled-back. I think they are both useful. Also, neither looks incredibly hard to implement. I'm sure I'm wrong about that, though. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQru0fHgHNmZLgCUhAQLX9hAAjSO4WkIaqyzeTYyz3l9PW5mWvvUKhxrv eK3glrQvrEf2HHy8gBfdcqv5RV8BAHfYIh6V/4S9VqmMzpBcUfBfEGRCJg0vcTsj oVSpzV0/ASdxitMjsHqIz3q3kNRhRuhZoD/ppsVVlH4zZtefioYecyD2asEzTz6w vAMN08chHR/vQiSDMBPDwkJ5Ov1tyPqdpSA4vBuRreWhDpD32ChBd3epEVXjt090 P0HxddqGR6FXCo9m4z2nTgSFHhAPkrOJE5bNS72hpk94nTfLRMI1DDTKw7J4AO10 U2mdjaXD5AUyahwE3diMcsxrH+94orE5bmK4yvgLUquBZ+OeRFoH0u8FG7sD3coP 2YkJewbhSrCQD69hrrR3pluwds3aoe+awj3x2WOk+6NV9xajw0czPs1P3cEw2Dwt Fl2+92M6545UUbysL51Dvk+DWp/PRniAL1nBoZ1q9nBlhYIiL7ccr48sXP2PxoZw nSQ1BDASCZalS6AFlMhI6A6U/OxIMzEZQr/cq1J9/dwRdrST2VLqiQdaCmsfgmL5 EVxA1C0tFjOPWwAlzVHB7MisxDp491w6WrxzV3r8x9vCwJZ747+2RNF3z2ah0A56 bVV6ZTjOK/pAsC0p+cfZS3NOTtvmP8x9QXQFll9WZvVEgFMYaOYkPlL6qwtpdNNR r0yOvDy1TLI= =FeCh -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 7:21 ` David Masover @ 2005-06-24 11:13 ` Bernd Eckenfels 0 siblings, 0 replies; 620+ messages in thread From: Bernd Eckenfels @ 2005-06-24 11:13 UTC (permalink / raw) To: linux-kernel In article <42BBB47C.9010002@slaphack.com> you wrote: > How about a poor-man's isolation -- any process other than that > responsible for the transaction sees a consistent state, never a > transaction-in-progress. I'm sure there's a name for that. It is Isolation Level Serializeable. It is the less performant isolation level and still can generate deadlocks if you have two process doing transactions. A more simpler solution would be that process without transactions never see infligt tx and a second process simple returnes a retry error if touching a locked ressource. Bernd ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-24 6:37 ` Hans Reiser @ 2005-06-24 10:31 ` Nikita Danilov 2005-06-24 10:31 ` Nikita Danilov 1 sibling, 0 replies; 620+ messages in thread From: Nikita Danilov @ 2005-06-24 10:31 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Artem B. Bityuckiy, Markus "T\x16rnqvist", Christophe Saout, Andrew Morton, hch, jgarzik, linux-kernel, reiserfs-list Hans Reiser <reiser@namesys.com> writes: > Nikita, I respectfully disagree with what you say about the state of our > atomicity code. It is not so far away as you describe, and probably 6 > man weeks work could polish it off. You don't see the value in what I > define as useful, namely atomicity without isolation. Since you don't > see that, it is harder for you to see that something is close to working > and just needs 6 weeks of someone who groks what I am asking for. No, I see the value of "atomicity", and think it is very useful. What I don't see the value of is the making of premature claims. _When_ reiser4 has atomic write(2), you have full right to call it atomic, not before. _When_ reiser4 is tested through objective benchmark-set exercising various workloads, you can refer to these benchmarks as the proof of reiser4 technical superiority, not before. On a more personal note, I invested large amount of my time and effort into developing reiser4, and I feel attached to it and to the great ideas embodied in it. For reiser4 to rot on the forgotten shelf in obscurity is the thing I want least. I want it to be included into mainline kernel, but for this to happen, you have to take more realistic stance towards err... reality. > > Hans Nikita. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-24 10:31 ` Nikita Danilov 0 siblings, 0 replies; 620+ messages in thread From: Nikita Danilov @ 2005-06-24 10:31 UTC (permalink / raw) To: Hans Reiser; +Cc: David Masover, Artem B. Bityuckiy, Markus Hans Reiser <reiser@namesys.com> writes: > Nikita, I respectfully disagree with what you say about the state of our > atomicity code. It is not so far away as you describe, and probably 6 > man weeks work could polish it off. You don't see the value in what I > define as useful, namely atomicity without isolation. Since you don't > see that, it is harder for you to see that something is close to working > and just needs 6 weeks of someone who groks what I am asking for. No, I see the value of "atomicity", and think it is very useful. What I don't see the value of is the making of premature claims. _When_ reiser4 has atomic write(2), you have full right to call it atomic, not before. _When_ reiser4 is tested through objective benchmark-set exercising various workloads, you can refer to these benchmarks as the proof of reiser4 technical superiority, not before. On a more personal note, I invested large amount of my time and effort into developing reiser4, and I feel attached to it and to the great ideas embodied in it. For reiser4 to rot on the forgotten shelf in obscurity is the thing I want least. I want it to be included into mainline kernel, but for this to happen, you have to take more realistic stance towards err... reality. > > Hans Nikita. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 21:51 ` David Masover 2005-06-22 22:23 ` Nikita Danilov @ 2005-06-22 22:27 ` Roland Dreier 1 sibling, 0 replies; 620+ messages in thread From: Roland Dreier @ 2005-06-22 22:27 UTC (permalink / raw) To: David Masover Cc: Horst von Brand, Artem B. Bityuckiy, Markus TЖrnqvist, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list David> Spotlight on the Mac. Users love it. We can do it. But David> not without changing something in the filesystem. David> Actually, I think we came up with several ways to do this, David> all of which required Reiser4 interfaces. It seems the existing Beagle project is a counterexample to this. In fact, to make things even more perfect, Beagle doesn't work on top of Reiser4. - R. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 17:33 ` Horst von Brand (?) (?) @ 2005-06-23 7:44 ` Artem B. Bityuckiy -1 siblings, 0 replies; 620+ messages in thread From: Artem B. Bityuckiy @ 2005-06-23 7:44 UTC (permalink / raw) To: Horst von Brand Cc: Artem B. Bityuckiy, Markus TЖrnqvist, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list Horst von Brand wrote: > I don't see anything that would interest me at least, so you can safely > scratch the "general interest" part. For me that's all very interesting. I believe people must take a chance Reiser4 to evolve as a part of the kernel, albeit there are many issues. Rape me, but I even think it is a kind of privilege for Linux to have Reiser4 in it (it could have gone to another OS). -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 17:33 ` Horst von Brand @ 2005-06-23 8:08 ` mjt -1 siblings, 0 replies; 620+ messages in thread From: Markus Törnqvist @ 2005-06-23 8:08 UTC (permalink / raw) To: Horst von Brand Cc: Artem B. Bityuckiy, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 3496 bytes --] On Wed, Jun 22, 2005 at 01:33:14PM -0400, Horst von Brand wrote: >> > So merge it as it is >Fix it first. The "merge as it stands" just gives rise to stuff that is >/never/ fixed properly. It's in -mm already and it was said the big grudges are fixed. What do you still want? >> > and move the stuff to the VFS as needed or >> > deemed necessary. And enable the pseudo interface, or at least >> > set it in menuconfig and enable by default, it needs testing too. > >Then test it out of the standard tree... Huh? You mean -mm? That's what people do, backport to mainline. Or did I miss something? >> Reiser4 has a number of great (IMO) things like file as directory, >Urgh. Your opinion :) >> different kinds of stat data, >Usefulness? Sounds like kernel bloat leading to userspace bloat and >applications/users wondering what the heck goes on when they don't grok the >particular stat format. Oh no! NOOOOOOOO! New features come at a cost?! Shit, I better unpack my C64. >> etc. Some thing is not yet ready - doesn't matter. Some of this is of >> general interest, some is Reiser4-dedicated. > >I don't see anything that would interest me at least, so you can safely >scratch the "general interest" part. These are things people will start using when available. They're given a framework which enables them to do things other people can't imagine. Too bad you're convinced everything everyone does is on the level of "Urgh" >> New interfaces are needed to allow users to utilize that all. > >That is a quite strong argument /against/ it all in my book. It means bloat >in /every/ filesystem, and they have shown to be able to do without for >some 30 years now. I'd need /very/ strong reasons for adding something. But the competitor's filesystems are evolving past that 30-year-old paradigm, why shouldn't Linux's? And if Reiser4 is merged and its VFS-doubling things merged into the VFS, how much will it bloat Ext2 if Ext2 doesn't go near files-as-dirs or extended stat data? >It has been argued over and over that that particular feature /can't/ be >implemented sanely anyway, so it has to stay out. Besides not making any Arguments left and right for and against this. I'd rather see someone write the code that does this, or, even better, you can write the code that proves this definitively wrong and someone else will get it to work. (Ok, I hate the "DIY" counter-argument as much as the next man, but this seemed almost an appropriate place to use it) >sense. "You've got files and directories" is a bit asymetrical and so not >quite nice; "all you have is directories" is symmetrical, estetic, and >completely useless; "some files are directories, some aren't; files in >file-directories are different than regular files in directory-directories" >is just a mindless jumble. "You've got data objects" All we need is the extra bit that's MAY_LOOKUP so we don't need +x to access metadata. But apparently this is too much to ask for. Why? /Probably/ standards compliance. So why be compliant, if it's backwards compatible? SILENCE! No really, why not extend something without breaking? ...we don't wanna... >> The other question that it may be difficult to foresee everything and >> it may make sense to move some things upper in future. >Another good reason to keep it out ;-) Sure, extinguish innovation. Got it. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins @ 2005-06-23 8:08 ` mjt 0 siblings, 0 replies; 620+ messages in thread From: mjt @ 2005-06-23 8:08 UTC (permalink / raw) To: Horst von Brand Cc: Artem B. Bityuckiy, Christophe Saout, Andrew Morton, Hans Reiser, hch, jgarzik, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 3496 bytes --] On Wed, Jun 22, 2005 at 01:33:14PM -0400, Horst von Brand wrote: >> > So merge it as it is >Fix it first. The "merge as it stands" just gives rise to stuff that is >/never/ fixed properly. It's in -mm already and it was said the big grudges are fixed. What do you still want? >> > and move the stuff to the VFS as needed or >> > deemed necessary. And enable the pseudo interface, or at least >> > set it in menuconfig and enable by default, it needs testing too. > >Then test it out of the standard tree... Huh? You mean -mm? That's what people do, backport to mainline. Or did I miss something? >> Reiser4 has a number of great (IMO) things like file as directory, >Urgh. Your opinion :) >> different kinds of stat data, >Usefulness? Sounds like kernel bloat leading to userspace bloat and >applications/users wondering what the heck goes on when they don't grok the >particular stat format. Oh no! NOOOOOOOO! New features come at a cost?! Shit, I better unpack my C64. >> etc. Some thing is not yet ready - doesn't matter. Some of this is of >> general interest, some is Reiser4-dedicated. > >I don't see anything that would interest me at least, so you can safely >scratch the "general interest" part. These are things people will start using when available. They're given a framework which enables them to do things other people can't imagine. Too bad you're convinced everything everyone does is on the level of "Urgh" >> New interfaces are needed to allow users to utilize that all. > >That is a quite strong argument /against/ it all in my book. It means bloat >in /every/ filesystem, and they have shown to be able to do without for >some 30 years now. I'd need /very/ strong reasons for adding something. But the competitor's filesystems are evolving past that 30-year-old paradigm, why shouldn't Linux's? And if Reiser4 is merged and its VFS-doubling things merged into the VFS, how much will it bloat Ext2 if Ext2 doesn't go near files-as-dirs or extended stat data? >It has been argued over and over that that particular feature /can't/ be >implemented sanely anyway, so it has to stay out. Besides not making any Arguments left and right for and against this. I'd rather see someone write the code that does this, or, even better, you can write the code that proves this definitively wrong and someone else will get it to work. (Ok, I hate the "DIY" counter-argument as much as the next man, but this seemed almost an appropriate place to use it) >sense. "You've got files and directories" is a bit asymetrical and so not >quite nice; "all you have is directories" is symmetrical, estetic, and >completely useless; "some files are directories, some aren't; files in >file-directories are different than regular files in directory-directories" >is just a mindless jumble. "You've got data objects" All we need is the extra bit that's MAY_LOOKUP so we don't need +x to access metadata. But apparently this is too much to ask for. Why? /Probably/ standards compliance. So why be compliant, if it's backwards compatible? SILENCE! No really, why not extend something without breaking? ...we don't wanna... >> The other question that it may be difficult to foresee everything and >> it may make sense to move some things upper in future. >Another good reason to keep it out ;-) Sure, extinguish innovation. Got it. -- mjt [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 14:56 ` Christophe Saout 2005-06-22 15:10 ` Artem B. Bityuckiy @ 2005-06-23 19:00 ` Hans Reiser 1 sibling, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-23 19:00 UTC (permalink / raw) To: Christophe Saout; +Cc: Andrew Morton, hch, jgarzik, linux-kernel, reiserfs-list This is a very nice description, thank you Christophe. My comments are below. Christophe Saout wrote: >Am Dienstag, den 21.06.2005, 18:18 -0700 schrieb Andrew Morton: > > > >>>What is wrong with having an encryption plugin implemented in this >>> manner? What is wrong with being able to have some files implemented >>> using a compression plugin, and others in the same filesystem not. >>> >>> What is wrong with having one file in the FS use a write only plugin, in >>> which the encrypion key is changed with every append in a forward but >>> not backward computable manner, and in order to read a file you must >>> either have a key that is stored on another computer or be reading what >>> was written after the moment of cracking root? >>> >>> What is wrong with having a set of critical data files use a CRC >>> checking file plugin? >>> >>> >>I think the concern here is that this is implemented at the wrong level. >> >>In Linux, a filesystem is some dumb thing which implements >>address_space_operations, filesystem_operations, etc. >> >>Advanced features such as those which you describe are implemented on top >>of the filesystem, not within it. reiser4 turns it all upside down. >> >>Now, some of the features which you envision are not amenable to >>above-the-fs implementations. But some will be, and that's where we should >>implement those. >> >> > >Yes, but that would be difficult, probably worse. The name "plugins" is >perhaps a bit misleading. These plugins are most of the time some sort >client to the reiser4 on-disk database structure. The core code is does >on-disk tree handling, journalling and these things. The plugins in turn >glue this core database system to the rest of the system in order to >make a filesystem of it. The "file plugin" tells the database how to >store files. > >A compression plugins is more tricky. Files should be randomly >accessible and if you write in the middle of the file the compressed >block might change in size. For reiser4 this is not a problem since it >just tells the underlying database "give me some room for 1234 bytes and >insert it in the tree instead of the other block". The reiser4 core has >totally different semantics than the VFS layer and I don't see how these >things could be handled elsewhere in this simple way. > >The reiser4 core is more some sort of library that abstracts a block >device and provides some sort of database API (which is highly optimized >for filesystem purposes). The actual filesystem is then another layer on >top of this. You could actually implement lots of different filesystems >on top of that database core. > >The actual layer is a bit different though. The database core itself >registers with the Linux VFS and then passes the calls down to one of >the plugins which then calls back into the reiser4 core to do the actual >database modification. I think this was the point that Christoph was >criticizing the most. > >Currently it looks like this: > > ,--------------. ,--------------. >VFS -------> | | ----> | | > | /fs/reiser4/ | | .../plugins/ | >blockdev <-- | | <---> | | > `--------------' `--------------' > >So the reiser4 code is introducing another abstraction of the Linux VFS >layer instead of letting the plugins define the Linux VFS ops directly. >Which would look like this: > > ,--------------. >VFS ------------------------------> | | > ,--------------, | .../plugins/ | >blockdev <-- | /fs/reiser4/ | <---> | | > `--------------' `--------------' > > Note that the proposed change (as I understand it) creates a need to load plugin definitions (classes) into VFS vectors (instances), which requires additional code and operations. Plugins need to be FS specific by their nature (unless someone wants the nightmare of allocating pluginids to each of the different filesystems and dealing with the inevitable collisions and violations). Note that with the proposed change there are now two places the contents of the plugin definition must reside: once in each VFS instance of that plugin, and once in the plugin definition. As Codd taught us, putting data in two places that must be kept synchronous is strongly undesirable, and it always causes more bugs over time than people think it will. By loading the instance from the plugin layer into VFS, we are creating a need to instantiate something that could instead be shared among all the instances. It seems an additional step that is unneeded. The advantage though is that it avoids a function dereference. Perhaps I miss something here? >Which probably would be okay for most of the time except for some >details (which could probably be solved otherwise). > >Actually the flow is not always this simple, usually the path goes back >and forth multiple time between the core and the plugins. > >One of the features Hans is using is that there can be different kinds >of files. The on-disk structure tells the core which of the plugins is >responsible for the "database object" found on the disk. It could be a >directory or a "stat data" (inode) or a file. The file itself could be >handled by different plugins like one that stores the data directly or >one that compresses it. > >reiser4 is different than other filesystems in that it uses a lot more >abstraction levels. The database aspect and the semantic aspect of a >traditional filesystems are strongly separated. > >To understand the code probably means a lot of work because it is a bit >different. Some of the layering concerns may be right, other probably >not. > >The plugins that add additional VFS semantics (that are currently >disable) should most definitely not be implemented only inside the >filesystem. I think Hans did this because it would have been a lot more >work doing this at the proper layer (which means talking to people and a >lot of politics...). > >The last time I looked at the code is a while ago, so if I'm wrong on >something, please don't shoot me. The only thing I can say is that >reiser4 has very stable for me (though I've gone back to reiser3 for >most of my filesystems because I wanted acl/xattr). > >So here's my advice: Instead of insulting each other or doing pure >marketing talk, please try to address each detail and explain why >something has been done and if it's good or bad and if it should be >changed and how. > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 1:18 ` Andrew Morton 2005-06-22 14:56 ` Christophe Saout @ 2005-06-25 18:55 ` Alexander Zarochentsev 1 sibling, 0 replies; 620+ messages in thread From: Alexander Zarochentsev @ 2005-06-25 18:55 UTC (permalink / raw) To: Andrew Morton; +Cc: reiserfs-list, Hans Reiser, linux-kernel On Wednesday 22 June 2005 05:18, Andrew Morton wrote: > Hans Reiser <reiser@namesys.com> wrote: > > What is wrong with having an encryption plugin implemented in this > > manner? What is wrong with being able to have some files implemented > > using a compression plugin, and others in the same filesystem not. > > > > What is wrong with having one file in the FS use a write only plugin, in > > which the encrypion key is changed with every append in a forward but > > not backward computable manner, and in order to read a file you must > > either have a key that is stored on another computer or be reading what > > was written after the moment of cracking root? > > > > What is wrong with having a set of critical data files use a CRC > > checking file plugin? > > I think the concern here is that this is implemented at the wrong level. > In Linux, a filesystem is some dumb thing which implements > address_space_operations, filesystem_operations, etc. > It is not so already. XFS, FUSE, network fs are not of that type. Thanks, Alex. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 1:07 ` reiser4 plugins Hans Reiser 2005-06-22 1:14 ` Jeff Garzik 2005-06-22 1:18 ` Andrew Morton @ 2005-06-22 5:34 ` Christoph Hellwig 2005-06-23 5:14 ` Hans Reiser 2 siblings, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-06-22 5:34 UTC (permalink / raw) To: Hans Reiser Cc: Christoph Hellwig, Jeff Garzik, Andrew Morton, linux-kernel, ReiserFS List On Tue, Jun 21, 2005 at 06:07:58PM -0700, Hans Reiser wrote: > Christoph, > > Reiser4 users love the plugin concept, and all audiences which have > listened to a presentation on plugins have been quite positive about > it. Many users think it is the best thing about reiser4. Can you > articulate why you are opposed to plugins in more detail? Perhaps you > are simply not as familiar with it as the audiences I have presented > to. Perhaps persons on our mailing list can comment..... You're duplicating the VFS infrastructure with your pluging system. > In particular, what is wrong with having a plugin id associated with > every file, storing the pluginid on disk in permanent storage in the > stat data, and having that plugin id define the set of methods that > implement the vfs operations associated with a particular file, rather > than defining VFS methods only at filesystem granularity? Hans, this only shows you didn't listen to be the last few times I explained it to you. There's nothing wrong with defining plug in IDs associated with every file, and the linux file_operations, inode_operations, etc.. are _not_ filesystem granularity but inode granularity (except in reiser4 where you throw everything together just to add your own de-multiplexer below). So the only thing your plugin might need to is to define it's own file or inode operations (in fact it might need a few more things speaking from experience with kinda similar things, but it certainly doesn't need to duplicate what's there) > What is wrong with having an encryption plugin implemented in this > manner? What is wrong with being able to have some files implemented > using a compression plugin, and others in the same filesystem not. There's nothing wrong with that, although such things might be better as a stackable filesystem. Maybe they're not, and we'll find out once people are prototyping these things and playing with them. But you don't need your additional layer of abstraction for those anyway > What is wrong with having one file in the FS use a write only plugin, in > which the encrypion key is changed with every append in a forward but > not backward computable manner, and in order to read a file you must > either have a key that is stored on another computer or be reading what > was written after the moment of cracking root? Because root can read kernel memory this is completely useless :) But if you want it as your private patch no one forbits you to do it, just don't expect such security by obscurity to go into mainline. > What we have hurts no one but us. I have never seen an audience for one > of my talks that thought it hurt us..... most audiences think it is > great. most of your audience doesn't understand the fine points of filesystem implementation. they want your feature but don't care how it's implemented. here it's the other way around - we don't care too much about what not so important features we have but more about how sanely they're implemented. > Let us tinker with our FS, and you tinker with yours, and so long as > what we do does not affect your FS, let the users choose. Now we're getting personal again, right? My filesystems according to MAINTAINERS are freevxfs and sysfs, but if you look at commit logs I've done work on pretty much any filesystem in the tree, and often doing tree-wide changes. That's why I care about every new filesystem in the tree and not a single one. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: reiser4 plugins 2005-06-22 5:34 ` Christoph Hellwig @ 2005-06-23 5:14 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-23 5:14 UTC (permalink / raw) To: Christoph Hellwig; +Cc: Jeff Garzik, Andrew Morton, linux-kernel, ReiserFS List Christoph Hellwig wrote: > > > > >>What is wrong with having one file in the FS use a write only plugin, in >>which the encrypion key is changed with every append in a forward but >>not backward computable manner, and in order to read a file you must >>either have a key that is stored on another computer or be reading what >>was written after the moment of cracking root? >> >> > >Because root can read kernel memory this is completely useless :) > > You missed the point of it rather nicely. If root can read kernel memory, that only gets it the appends made after the point in time of cracking root. It is not my idea, and it is not yet present in our code, let me not seem to take credit for it though I think it a good idea. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 15:26 ` -mm -> 2.6.13 merge status Jeff Garzik ` (3 preceding siblings ...) 2005-06-21 20:05 ` Hans Reiser @ 2005-06-21 20:22 ` Andrew Morton 2005-06-21 20:56 ` Gerrit Huizenga ` (3 more replies) 4 siblings, 4 replies; 620+ messages in thread From: Andrew Morton @ 2005-06-21 20:22 UTC (permalink / raw) To: Jeff Garzik; +Cc: linux-kernel Jeff Garzik <jgarzik@pobox.com> wrote: > > > sparsemem > > > > OK by me for a merge. Need to poke arch maintainers first, check that > > they've looked at it sufficiently closely. > > seems sane, though there are some whitespace niggles that should be > cleaned up > There are? I thought I fixed most of them. *general sigh*. I wish people would absorb CodingStyle. It's not hard, and fixing the style post-facto creates a real mess. I now have a great string of kexec patches followed by a "kexec-code-cleanup.patch" which totally buggers up the patch sequencing and really needs to be split into 18 parts and sprinkled back over the entire series. > > rapidio-* > > > > Will merge. > > send through netdev, as is proper > OK. But then the master version vanishes into the jgarzik git forest and I won't know how to get it ;) > > connector.patch > > > > Nice idea IMO, but there are still questions around the > > implementation. More dialogue needed ;) > > > > connector-add-a-fork-connector.patch > > > > OK, but needs connector. > > I don't like connector > How come? > > > pcmcia-*.patch > > > > Makes the pcmcia layer generate hotplug events and deprecates cardmgr. > > Will merge. > > Testing? The goal behind the patch is certainly good, but I worry about > exposure. > Yes, there will be a few problems I guess. But people are testing it - we know, because we've had lots of bug reports which were actually due to greg-pci breakage... > > > cachefs > > > > This is a ton of code which knows rather a lot about pagecache > > internals. It allows the AFS client to cache file contents on a local > > blockdev. > > > > I don't think it's a justified addition for only AFS and I'd prefer to > > see it proven for NFS as well. > > > > Issues around add-page-becoming-writable-notification.patch need to > > be resolved. > > > > cachefs-for-nfs > > > > A recent addition. Needs review from NFS developers and considerably > > more testing. > > > > These things aren't looking likely for 2.6.13. > > If I could vote more than once, I would! I really like cachefs, and > have been pushing for its inclusion for a while. > You've been using it? > > kexec and kdump > > > > I guess we should merge these. > > > > I'm still concerned that the various device shutdown problems will > > mean that the success rate for crashing kernels is not high enough for > > kdump to be considered a success. In which case in six months time we'll > > hear rumours about vendors shipping wholly different crashdump > > implementations, which would be quite bad. > > > > But I think this has gone as far as it can go in -mm, so it's a bit of > > a punt. > > I'm not particularly pleased with these, How come? > and indeed vendors ARE shipping > other crashdump methods. Which ones? > > > reiser4 > > > > Merge it, I guess. > > > > The patches still contain all the reiser4-specific namespace > > enhancements, only it is disabled, so it is effectively dead code. Maybe > > we should ask that it actually be removed? > > The plugin stuff is crap. This is not a filesystem but a filesystem + > new layer. IMO considered in that light, it duplicates functionality > elsewhere. > hm. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:22 ` -mm -> 2.6.13 merge status Andrew Morton @ 2005-06-21 20:56 ` Gerrit Huizenga 2005-06-21 21:04 ` Andrew Morton 2005-06-21 21:28 ` -mm -> 2.6.13 merge status Carsten Otte ` (2 subsequent siblings) 3 siblings, 1 reply; 620+ messages in thread From: Gerrit Huizenga @ 2005-06-21 20:56 UTC (permalink / raw) To: Andrew Morton; +Cc: Jeff Garzik, linux-kernel On Tue, 21 Jun 2005 13:22:04 PDT, Andrew Morton wrote: > Jeff Garzik <jgarzik@pobox.com> wrote: > > > kexec and kdump > > > > > > I guess we should merge these. > > > > > > I'm still concerned that the various device shutdown problems will > > > mean that the success rate for crashing kernels is not high enough for > > > kdump to be considered a success. In which case in six months time we'll > > > hear rumours about vendors shipping wholly different crashdump > > > implementations, which would be quite bad. > > > > > > But I think this has gone as far as it can go in -mm, so it's a bit of > > > a punt. > > > > I'm not particularly pleased with these, > > How come? > > > and indeed vendors ARE shipping > > other crashdump methods. > > Which ones? And which ones that __WORK__? We have a few customers out there from both distros and all the crash dump methods that I've seen fail either ALWAYS or ALMOST ALWAYS on customer sites. And yes, we hear about them and I believe that our partners understand the pain that this causes us and our customers. Kexec/kdump has a chance of working reliably. The others are complete crap. gerrit ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:56 ` Gerrit Huizenga @ 2005-06-21 21:04 ` Andrew Morton 2005-06-21 21:23 ` Gerrit Huizenga ` (3 more replies) 0 siblings, 4 replies; 620+ messages in thread From: Andrew Morton @ 2005-06-21 21:04 UTC (permalink / raw) To: Gerrit Huizenga; +Cc: jgarzik, linux-kernel Gerrit Huizenga <gh@us.ibm.com> wrote: > > Kexec/kdump has a chance of working reliably. IOW: Kexec/kdump has a chance of not working reliably. Worried. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 21:04 ` Andrew Morton @ 2005-06-21 21:23 ` Gerrit Huizenga 2005-06-21 21:38 ` Arjan van de Ven ` (2 subsequent siblings) 3 siblings, 0 replies; 620+ messages in thread From: Gerrit Huizenga @ 2005-06-21 21:23 UTC (permalink / raw) To: Andrew Morton; +Cc: jgarzik, linux-kernel On Tue, 21 Jun 2005 14:04:41 PDT, Andrew Morton wrote: > Gerrit Huizenga <gh@us.ibm.com> wrote: > > > > Kexec/kdump has a chance of working reliably. > > IOW: Kexec/kdump has a chance of not working reliably. > > Worried. No worries. Machine locks up hard, hardware failures, etc., there is a possibility that nothing but a hard reset can unlock a machine. But that is rare and outside the scope of the simple locking problems that today prevent crash dumps. There are still some rough edges in PCI shutdown code, reinitialization, etc. that will need to be shaken out over time with more experience, but those at least can be addressed in the fundamental architecture of kexec/kdump. About the only possible solution that *might* be fail proof (and even that case I doubt) would be an external dump program under control of the firmware (assuming the firmware can still run) which does a copy of memory off to some device without any assistance from the kernel. Kexec/kdump needs much wider exposure at this point and there will a few bumps along the way. They should be isolated to cases where the machine is already crashing and the only thing that doesn't work is the crash dump/restart. And those we will fix like any other bugs. gerrit ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 21:04 ` Andrew Morton 2005-06-21 21:23 ` Gerrit Huizenga @ 2005-06-21 21:38 ` Arjan van de Ven 2005-06-22 6:33 ` Martin J. Bligh 2005-06-23 0:58 ` -mm -> 2.6.13 merge status (kexec/kdump) Vara Prasad 3 siblings, 0 replies; 620+ messages in thread From: Arjan van de Ven @ 2005-06-21 21:38 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel, jgarzik, Gerrit Huizenga [-- Attachment #1: Type: text/plain, Size: 852 bytes --] On Tue, 2005-06-21 at 14:04 -0700, Andrew Morton wrote: > Gerrit Huizenga <gh@us.ibm.com> wrote: > > > > Kexec/kdump has a chance of working reliably. > > IOW: Kexec/kdump has a chance of not working reliably. > > Worried. it's about rates... you can hose your system so bad that nothing can fix it. the "distro" stuff probably succeeds 10% of the time in non-artificial cases, the kexec one has the potential at least to go 90% (so yes I admit that I pull these numbers out of my backside but I'm with Gerrit on this one). The stuff the distros ship is also not so nice in that you can't dump to LVM or SAN or ... etc; kexec gets all that right. It's also far more an all-or-nothing thing; if you make the transition you know you're going to get a good dump; most of the other dumps die halfway in the process at some point. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 21:04 ` Andrew Morton 2005-06-21 21:23 ` Gerrit Huizenga 2005-06-21 21:38 ` Arjan van de Ven @ 2005-06-22 6:33 ` Martin J. Bligh 2005-06-23 0:58 ` -mm -> 2.6.13 merge status (kexec/kdump) Vara Prasad 3 siblings, 0 replies; 620+ messages in thread From: Martin J. Bligh @ 2005-06-22 6:33 UTC (permalink / raw) To: Andrew Morton, Gerrit Huizenga; +Cc: jgarzik, linux-kernel --Andrew Morton <akpm@osdl.org> wrote (on Tuesday, June 21, 2005 14:04:41 -0700): > Gerrit Huizenga <gh@us.ibm.com> wrote: >> >> Kexec/kdump has a chance of working reliably. > > IOW: Kexec/kdump has a chance of not working reliably. > > Worried. Personally I'm more concerned about the design issues - I can't see how any of the other options are sustainable / workable. Things that require maintaining their own driver base are just insane. Things that dump from the panicing kernel are just broken. People want to be able to dump to disk / network / flash-ram card / god-knows-what, so we need something that's flexible. I don't think kdump is perfect and bug-free yet, but at least it has a design that looks like it'll be workable and sustainable through the future. Plus it's a small patch on top of kexec, which is useful in it's own right (for fast reboot, etc) so we get to reuse a lot of code. We could go into how crashdump itself is important (eg. first time failure capture is critical for customers, less downtime, I can ship you better data on bugs I find in test, etc, etc) but I kind of assumed most people were convinced of that by now. Even Linus seemed to think kdump was the sensible way forward (at KS last year), and he seems to be one of the most ardent sceptics of crashdump I've ever met ;-) M. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (kexec/kdump) 2005-06-21 21:04 ` Andrew Morton ` (2 preceding siblings ...) 2005-06-22 6:33 ` Martin J. Bligh @ 2005-06-23 0:58 ` Vara Prasad 2005-06-23 1:08 ` Andrew Morton 3 siblings, 1 reply; 620+ messages in thread From: Vara Prasad @ 2005-06-23 0:58 UTC (permalink / raw) To: Andrew Morton; +Cc: Gerrit Huizenga, jgarzik, linux-kernel Andrew Morton wrote: >Gerrit Huizenga <gh@us.ibm.com> wrote: > > >>Kexec/kdump has a chance of working reliably. >> >> > >IOW: Kexec/kdump has a chance of not working reliably. > >Worried. >- >To unsubscribe from this list: send the line "unsubscribe linux-kernel" in >the body of a message to majordomo@vger.kernel.org >More majordomo info at http://vger.kernel.org/majordomo-info.html >Please read the FAQ at http://www.tux.org/lkml/ > > > I understand your concerns based on the device shutdown issues but these are not fundamental design problems that we can not solve. We should be able to fix them either through generic solutions to a class of devices or one of kind for special devices. As you know we are engaging in those discussions and providing fixes. I think all the alternatives out there are less reliable than Kdump based on the design. Vendors are currently shipping other solutions since they didn't have any better alternatives until now. The existing solutions in the two major distro's doesn't work lot of times. I don't know what percentage of times they work as i only get involved when they don't work, but i can certainly tell you they don't work many a times. It is very embarrassing to tell the customer sorry we couldn't get dump can you try reproducing the problem again. At least two major distros expressed interest in replacing their current solutions with kdump once it matures. As you are well aware we are doing testing with as many configurations as we can to iron out the bugs. Hope this addresses some of your concerns. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (kexec/kdump) 2005-06-23 0:58 ` -mm -> 2.6.13 merge status (kexec/kdump) Vara Prasad @ 2005-06-23 1:08 ` Andrew Morton 0 siblings, 0 replies; 620+ messages in thread From: Andrew Morton @ 2005-06-23 1:08 UTC (permalink / raw) To: Vara Prasad; +Cc: gh, jgarzik, linux-kernel Vara Prasad <prasadav@us.ibm.com> wrote: > > I think all the alternatives out there are less reliable than Kdump > based on the design. Vendors are currently shipping other solutions > since they didn't have any better alternatives until now. The existing > solutions in the two major distro's doesn't work lot of times. I don't > know what percentage of times they work as i only get involved when they > don't work, but i can certainly tell you they don't work many a times. > It is very embarrassing to tell the customer sorry we couldn't get dump > can you try reproducing the problem again. At least two major distros > expressed interest in replacing their current solutions with kdump once > it matures. As you are well aware we are doing testing with as many > configurations as we can to iron out the bugs. Hope this addresses some > of your concerns. Yes, thanks. And the meta-goodness here is that at least we have a *design* which is acceptable from this-is-sane standpoint. So at least everyone will be pulling in the same direction. So as I said, it's a bit of a bet at this point in time, but we've gone as far as we can get with it out-of-tree, so let's merge it and hope that it matures into an acceptably useful dumper. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:22 ` -mm -> 2.6.13 merge status Andrew Morton 2005-06-21 20:56 ` Gerrit Huizenga @ 2005-06-21 21:28 ` Carsten Otte 2005-06-22 23:32 ` Chris Wright 2005-06-22 16:53 ` Eric W. Biederman 2005-06-24 4:06 ` Paul Jackson 3 siblings, 1 reply; 620+ messages in thread From: Carsten Otte @ 2005-06-21 21:28 UTC (permalink / raw) To: Andrew Morton; +Cc: Jeff Garzik, linux-kernel On 6/21/05, Andrew Morton <akpm@osdl.org> wrote: > > and indeed vendors ARE shipping > > other crashdump methods. > > Which ones? For 390, we ship standalone bootable crashdump tools with both sles and rhel. As for kexec, I'd like to see a kexec based 390 bootloader in the future which would be more flexible then our current one. So I'd like to vote for merging kexec/kdump. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 21:28 ` -mm -> 2.6.13 merge status Carsten Otte @ 2005-06-22 23:32 ` Chris Wright 2005-06-23 13:04 ` Carsten Otte 0 siblings, 1 reply; 620+ messages in thread From: Chris Wright @ 2005-06-22 23:32 UTC (permalink / raw) To: Carsten Otte; +Cc: Andrew Morton, Jeff Garzik, linux-kernel * Carsten Otte (cotte.de@gmail.com) wrote: > On 6/21/05, Andrew Morton <akpm@osdl.org> wrote: > > > and indeed vendors ARE shipping > > > other crashdump methods. > > > > Which ones? > For 390, we ship standalone bootable crashdump tools with both sles > and rhel. As for kexec, I'd like to see a kexec based 390 bootloader > in the future which would be more flexible then our current one. So > I'd like to vote for merging kexec/kdump. Xen is making similar noises w.r.t. using kexec for flexible bootloader. thanks, -chris ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-22 23:32 ` Chris Wright @ 2005-06-23 13:04 ` Carsten Otte 0 siblings, 0 replies; 620+ messages in thread From: Carsten Otte @ 2005-06-23 13:04 UTC (permalink / raw) To: Chris Wright; +Cc: Andrew Morton, Jeff Garzik, linux-kernel On 6/23/05, Chris Wright <chrisw@osdl.org> wrote: > * Carsten Otte (cotte.de@gmail.com) wrote: > > For 390, we ship standalone bootable crashdump tools with both sles > > and rhel. As for kexec, I'd like to see a kexec based 390 bootloader > > in the future which would be more flexible then our current one. So > > I'd like to vote for merging kexec/kdump. > > Xen is making similar noises w.r.t. using kexec for flexible bootloader. Oh cool, then we should look at what they're doing instead of reinventing the wheel. Any pointer we can follow, or person we would contact? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:22 ` -mm -> 2.6.13 merge status Andrew Morton 2005-06-21 20:56 ` Gerrit Huizenga 2005-06-21 21:28 ` -mm -> 2.6.13 merge status Carsten Otte @ 2005-06-22 16:53 ` Eric W. Biederman 2005-06-22 20:52 ` Andrew Morton 2005-06-24 4:06 ` Paul Jackson 3 siblings, 1 reply; 620+ messages in thread From: Eric W. Biederman @ 2005-06-22 16:53 UTC (permalink / raw) To: Andrew Morton; +Cc: Jeff Garzik, linux-kernel, fastboot Andrew Morton <akpm@osdl.org> writes: > Jeff Garzik <jgarzik@pobox.com> wrote: > > > > > sparsemem > > > > > > OK by me for a merge. Need to poke arch maintainers first, check that > > > they've looked at it sufficiently closely. > > > > seems sane, though there are some whitespace niggles that should be > > cleaned up > > > > There are? I thought I fixed most of them. > > *general sigh*. I wish people would absorb CodingStyle. It's not hard, > and fixing the style post-facto creates a real mess. I now have a great > string of kexec patches followed by a "kexec-code-cleanup.patch" which > totally buggers up the patch sequencing and really needs to be split into > 18 parts and sprinkled back over the entire series. It looks like I have another hole in my schedule where I can put some work into kexec so I will see what I can do. If you want people to absorb CodingStyle it needs to be more explicit. Of the things that patch fixes you almost have to read between the lines of CodingStyle to see. If there is anything backing it up at all. Until the problems were pointed out to me I simply could not see them, and reading CodingStyle was not enlightening. I point this out not to complain but more so people know which part of the process needs fixing. > > > kexec and kdump > > > > > > I guess we should merge these. > > > > > > I'm still concerned that the various device shutdown problems will > > > mean that the success rate for crashing kernels is not high enough for > > > kdump to be considered a success. In which case in six months time we'll > > > > hear rumours about vendors shipping wholly different crashdump > > > implementations, which would be quite bad. > > > > > > But I think this has gone as far as it can go in -mm, so it's a bit of > > > a punt. > > > > I'm not particularly pleased with these, > > How come? Please tell. With respect to users of crashdumps there are at least two groups converging on kexec based crashdumps. Is there active development on any of the rest of the tools. On to the practical response. The kexec has effectively zero impact on the kernel, except when it is invoked, and the code is small. Kexec is also useful for a lot more than just crash dumps. It happens that crashdumps seem to be the only case where the other alternatives are noticeably less sane. There is also another important piece about kexec based crashdumps that is not usually envisioned. The kexec based solution is much more flexible. For example on a cluster the worst case scenario for a network based crashdump is all 1000+ nodes will output a crashdump simultaneously. Poor crashdump server. Where with the kexec based approach it is possible to have the machines send an SNMP alert and then you can come in and have the machine dump only when you are ready. Or you might even start a gdb-stubs server and interact with a live core dump. :) And all of this happens to fall out naturally from using a kernel after the crash. There are a few associated bug fixes that are problematic but I think they are fixable. For the crashdump case the work really is going into getting hardening linux so it can run on imperfectly behaving hardware. I.e. things that are bugs in any event but that using the kernel to take a crashdump exacerbates. Andrew the good news is unless something comes up I will have time starting about Monday to help with the 2.6.13 merge. It looks like the first thing I should do is split up the indent patch so it pairs well with the rest. And then I have a few pending patches for the user space and I think I can fix a number of the device_shutdown problems, for at least the normal kexec path. Eric ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-22 16:53 ` Eric W. Biederman @ 2005-06-22 20:52 ` Andrew Morton 2005-06-23 2:14 ` Eric W. Biederman 0 siblings, 1 reply; 620+ messages in thread From: Andrew Morton @ 2005-06-22 20:52 UTC (permalink / raw) To: Eric W. Biederman; +Cc: jgarzik, linux-kernel, fastboot ebiederm@xmission.com (Eric W. Biederman) wrote: > > Andrew the good news is unless something comes up I will have time > starting about Monday to help with the 2.6.13 merge. It looks like > the first thing I should do is split up the indent patch so it pairs > well with the rest. And then I have a few pending patches for the user > space and I think I can fix a number of the device_shutdown problems, > for at least the normal kexec path. I'd be inclined to leave it as-is, really. I'm not sure that it's worth the effort+risk of significantly refactoring the patches. I've cut it down from 58 patches to 45 and will take another pass at it in the next day or two, but it's looking like 40-odd patches is the right number. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-22 20:52 ` Andrew Morton @ 2005-06-23 2:14 ` Eric W. Biederman 0 siblings, 0 replies; 620+ messages in thread From: Eric W. Biederman @ 2005-06-23 2:14 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel, fastboot Andrew Morton <akpm@osdl.org> writes: > ebiederm@xmission.com (Eric W. Biederman) wrote: > > > > Andrew the good news is unless something comes up I will have time > > starting about Monday to help with the 2.6.13 merge. It looks like > > the first thing I should do is split up the indent patch so it pairs > > well with the rest. And then I have a few pending patches for the user > > space and I think I can fix a number of the device_shutdown problems, > > for at least the normal kexec path. > > I'd be inclined to leave it as-is, really. I'm not sure that it's worth > the effort+risk of significantly refactoring the patches. > > I've cut it down from 58 patches to 45 and will take another pass at it in > the next day or two, but it's looking like 40-odd patches is the right > number. Ok. Then I won't put a priority on that piece. I will still look at cleaning up the sematics of the reboot path so fewer things break. Eric ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:22 ` -mm -> 2.6.13 merge status Andrew Morton ` (2 preceding siblings ...) 2005-06-22 16:53 ` Eric W. Biederman @ 2005-06-24 4:06 ` Paul Jackson 2005-06-24 4:54 ` randy_dunlap 3 siblings, 1 reply; 620+ messages in thread From: Paul Jackson @ 2005-06-24 4:06 UTC (permalink / raw) To: Andrew Morton; +Cc: jgarzik, linux-kernel > I wish people would absorb CodingStyle. Some people just can't see it, Andrew. Just like some people are tone deaf, some people don't notice minor variations in code spacing and layout, unless pointed out in tedious detail. Not that I disagree with you ... ;). -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <pj@sgi.com> 1.925.600.0401 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-24 4:06 ` Paul Jackson @ 2005-06-24 4:54 ` randy_dunlap 0 siblings, 0 replies; 620+ messages in thread From: randy_dunlap @ 2005-06-24 4:54 UTC (permalink / raw) To: Paul Jackson; +Cc: akpm, jgarzik, linux-kernel On Thu, 23 Jun 2005 21:06:38 -0700 Paul Jackson wrote: | > I wish people would absorb CodingStyle. | | Some people just can't see it, Andrew. Just like some people | are tone deaf, some people don't notice minor variations in | code spacing and layout, unless pointed out in tedious detail. | | Not that I disagree with you ... ;). I also agree. The problem (for me at least) is that bad coding style needs to be fixed before I can do a functional code review, so it slows down the review cycle quite a bit. Further, it's mostly well-known what the requirements are, so there aren't very many good excuses not to follow CodingStyle... --- ~Randy ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (9 preceding siblings ...) 2005-06-21 15:26 ` -mm -> 2.6.13 merge status Jeff Garzik @ 2005-06-21 15:50 ` Lee Revell 2005-06-21 18:56 ` Nish Aravamudan 2005-06-21 20:32 ` Andrew Morton 2005-06-21 16:26 ` -mm -> 2.6.13 merge status (HZ -> 250?) Lee Revell ` (7 subsequent siblings) 18 siblings, 2 replies; 620+ messages in thread From: Lee Revell @ 2005-06-21 15:50 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel On Mon, 2005-06-20 at 23:54 -0700, Andrew Morton wrote: > CONFIG_HZ for x86 and ia64: changes default HZ to 250, make HZ > Kconfigurable. > > Will merge (will switch default to 1000 Hz later if that seems > necessary) Are you serious? You're changing the *default* HZ in a stable kernel series?!? This is a big regression, it degrades the resolution of system calls. Lee ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 15:50 ` Lee Revell @ 2005-06-21 18:56 ` Nish Aravamudan 2005-06-22 18:00 ` Nish Aravamudan 2005-06-21 20:32 ` Andrew Morton 1 sibling, 1 reply; 620+ messages in thread From: Nish Aravamudan @ 2005-06-21 18:56 UTC (permalink / raw) To: Lee Revell; +Cc: Andrew Morton, linux-kernel On 6/21/05, Lee Revell <rlrevell@joe-job.com> wrote: > On Mon, 2005-06-20 at 23:54 -0700, Andrew Morton wrote: > > CONFIG_HZ for x86 and ia64: changes default HZ to 250, make HZ > > Kconfigurable. > > > > Will merge (will switch default to 1000 Hz later if that seems > > necessary) > > Are you serious? You're changing the *default* HZ in a stable kernel > series?!? > > This is a big regression, it degrades the resolution of system calls. Not that my opinion should sway anybody else, but I really would prefer more of the in-kernel sleep callers were converted to use human-time units (and thus were verified to be correct) so that switching HZ will result in the *same* latencies. How much of moving to lower HZ values is due to the fact that everything is request 10ms for 1 jiffy of sleep instead of 1 ms? It's hard to tell if the gain is there or from the lower frequency of interrupts. I've sent out a lot of patches in this direction (using msleep() and msleep_interruptible() and my soft-timer rework on top of John Stultz's timeofday rework converts the entire soft-timer subsystem to use human-time instead of jiffies as it's unit of expiration), but there is still *a lot* of work left to do :( I will keep sending patches, but am being pulled in other directions currently. Just my $.02. Thanks, Nish ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 18:56 ` Nish Aravamudan @ 2005-06-22 18:00 ` Nish Aravamudan 0 siblings, 0 replies; 620+ messages in thread From: Nish Aravamudan @ 2005-06-22 18:00 UTC (permalink / raw) To: Lee Revell; +Cc: Andrew Morton, linux-kernel On 6/21/05, Nish Aravamudan <nish.aravamudan@gmail.com> wrote: > On 6/21/05, Lee Revell <rlrevell@joe-job.com> wrote: > > On Mon, 2005-06-20 at 23:54 -0700, Andrew Morton wrote: > > > CONFIG_HZ for x86 and ia64: changes default HZ to 250, make HZ > > > Kconfigurable. > > > > > > Will merge (will switch default to 1000 Hz later if that seems > > > necessary) > > > > Are you serious? You're changing the *default* HZ in a stable kernel > > series?!? > > > > This is a big regression, it degrades the resolution of system calls. > > Not that my opinion should sway anybody else, but I really would > prefer more of the in-kernel sleep callers were converted to use > human-time units (and thus were verified to be correct) so that > switching HZ will result in the *same* latencies. How much of moving > to lower HZ values is due to the fact that everything is request 10ms > for 1 jiffy of sleep instead of 1 ms? It's hard to tell if the gain is > there or from the lower frequency of interrupts. After some further consideration, I don't think that my patches would be at all changed by adjusting HZ's default value. I just want to make sure maintainers are still responsive to appropriate patches to split time-based delays from tick-based delays. So, CONFIG_HZ is ok by me, but I consider it a band-aid. Thanks, Nish ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 15:50 ` Lee Revell 2005-06-21 18:56 ` Nish Aravamudan @ 2005-06-21 20:32 ` Andrew Morton 2005-06-21 20:37 ` Lee Revell 1 sibling, 1 reply; 620+ messages in thread From: Andrew Morton @ 2005-06-21 20:32 UTC (permalink / raw) To: Lee Revell; +Cc: linux-kernel Lee Revell <rlrevell@joe-job.com> wrote: > > On Mon, 2005-06-20 at 23:54 -0700, Andrew Morton wrote: > > CONFIG_HZ for x86 and ia64: changes default HZ to 250, make HZ > > Kconfigurable. > > > > Will merge (will switch default to 1000 Hz later if that seems > > necessary) > > Are you serious? You're changing the *default* HZ in a stable kernel > series?!? > > This is a big regression, it degrades the resolution of system calls. > Well we'll see what happens. As I said, if it's determined to be a real problem we'll put the default back to 1000Hz prior to 2.6.13 release. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 20:32 ` Andrew Morton @ 2005-06-21 20:37 ` Lee Revell 0 siblings, 0 replies; 620+ messages in thread From: Lee Revell @ 2005-06-21 20:37 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel On Tue, 2005-06-21 at 13:32 -0700, Andrew Morton wrote: > Lee Revell <rlrevell@joe-job.com> wrote: > > > > On Mon, 2005-06-20 at 23:54 -0700, Andrew Morton wrote: > > > CONFIG_HZ for x86 and ia64: changes default HZ to 250, make HZ > > > Kconfigurable. > > > > > > Will merge (will switch default to 1000 Hz later if that seems > > > necessary) > > > > Are you serious? You're changing the *default* HZ in a stable kernel > > series?!? > > > > This is a big regression, it degrades the resolution of system calls. > > > > Well we'll see what happens. As I said, if it's determined to be a real > problem we'll put the default back to 1000Hz prior to 2.6.13 release. > I just think it's silly to merge CONFIG_HZ this late in the game, when dynamic tick and high res timers are right around the corner. Seems like more trouble than it's worth. Lee ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (HZ -> 250?) 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (10 preceding siblings ...) 2005-06-21 15:50 ` Lee Revell @ 2005-06-21 16:26 ` Lee Revell 2005-06-21 18:09 ` Alan Cox 2005-06-21 17:20 ` xtensa architecture (-mm -> 2.6.13 merge status) Chris Zankel ` (6 subsequent siblings) 18 siblings, 1 reply; 620+ messages in thread From: Lee Revell @ 2005-06-21 16:26 UTC (permalink / raw) To: Andrew Morton; +Cc: George Anzinger, linux-kernel On Mon, 2005-06-20 at 23:54 -0700, Andrew Morton wrote: > CONFIG_HZ for x86 and ia64: changes default HZ to 250, make HZ > Kconfigurable. > > Will merge (will switch default to 1000 Hz later if that seems > necessary) How about delaying this until the high res timers patches are ready? That way you can save power and avoid the latency regression, in fact it would be a huge improvement from user POV. Consider a program with a 5ms RT constraint, like a game or mplayer. Currently it uses the RTC on 2.4/HZ=100 systems and usleep() on 2.6/HZ=1000. Allowing HZ to regress to 250 would force us to handle 2.4, 2.6.1 - 2.6.12, and 2.6.13+ separately. It would be a huge mess. Lee ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (HZ -> 250?) 2005-06-21 16:26 ` -mm -> 2.6.13 merge status (HZ -> 250?) Lee Revell @ 2005-06-21 18:09 ` Alan Cox 0 siblings, 0 replies; 620+ messages in thread From: Alan Cox @ 2005-06-21 18:09 UTC (permalink / raw) To: Lee Revell; +Cc: Andrew Morton, George Anzinger, Linux Kernel Mailing List On Maw, 2005-06-21 at 17:26, Lee Revell wrote: > Consider a program with a 5ms RT constraint, like a game or mplayer. > Currently it uses the RTC on 2.4/HZ=100 systems and usleep() on > 2.6/HZ=1000. Allowing HZ to regress to 250 would force us to handle > 2.4, 2.6.1 - 2.6.12, and 2.6.13+ separately. It would be a huge mess. Vendors already ship 100Hz and 1KHz kernels. 2.4 and 2.6 are different already. I can see the argument for not picking another new value though. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: xtensa architecture (-mm -> 2.6.13 merge status) 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (11 preceding siblings ...) 2005-06-21 16:26 ` -mm -> 2.6.13 merge status (HZ -> 250?) Lee Revell @ 2005-06-21 17:20 ` Chris Zankel 2005-06-22 3:35 ` OCFS (was Re: -mm " Rik Van Riel ` (5 subsequent siblings) 18 siblings, 0 replies; 620+ messages in thread From: Chris Zankel @ 2005-06-21 17:20 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel Andrew Morton wrote: > xtensa architecture > > Is xtensa now, or will it be in the future a sufficiently popular > architecture to justify the cost of having this code in the tree? > > Heaven knows. Will merge. Andrew, I understand your concern and am glad that you give Xtensa and the other smaller non-mainstream architectures a chance. The Xtensa architecture is highly configurable and usually buried inside an SOC device. So, if you buy a new printer, digital camera, or cell phone, there is a chance that there is an Xtensa inside even though you don't know it (sometimes as a small audio-engine or as a control CPU). Linux hasn't been adopted widely with Xtensa yet, but with Linux growing in the embedded space, I am sure it will become much more important -- at least this is where I bet my time (and spare time) on. To minimize the impact on other developers, I do understand that changes that affect all architectures will only be applied to the mainstream architectures and that the maintainers of the non-mainstream architectures then have to pick it up. Luckily, the architecture dependent files have their own confined space in the arch and asm directories. In my opinion, as long as an architecture, driver, etc. is maintained and not obviously obsolete, it should be allowed to remain in the kernel. I do have a few small patches in the queue but am struggling with some changes I want to make to the syscalls that might break some older code. Thanks, ~Chris ^ permalink raw reply [flat|nested] 620+ messages in thread
* OCFS (was Re: -mm -> 2.6.13 merge status) 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (12 preceding siblings ...) 2005-06-21 17:20 ` xtensa architecture (-mm -> 2.6.13 merge status) Chris Zankel @ 2005-06-22 3:35 ` Rik Van Riel 2005-06-22 5:23 ` Wim Coekaerts 2005-06-22 4:08 ` -mm -> 2.6.13 merge status (configfs) David Teigland ` (4 subsequent siblings) 18 siblings, 1 reply; 620+ messages in thread From: Rik Van Riel @ 2005-06-22 3:35 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel On Mon, 20 Jun 2005, Andrew Morton wrote: > git-ocfs > > The OCFS2 filesystem. OK by me, although I'm not sure it's had enough > review. The only problem I can see with this is that people will want to use OCFS together with CLVM, and both use a different cluster infrastructure. IMHO it would be good if they both used the same underlying cluster infrastructure... -- The Theory of Escalating Commitment: "The cost of continuing mistakes is borne by others, while the cost of admitting mistakes is borne by yourself." -- Joseph Stiglitz, Nobel Laureate in Economics ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: OCFS (was Re: -mm -> 2.6.13 merge status) 2005-06-22 3:35 ` OCFS (was Re: -mm " Rik Van Riel @ 2005-06-22 5:23 ` Wim Coekaerts 0 siblings, 0 replies; 620+ messages in thread From: Wim Coekaerts @ 2005-06-22 5:23 UTC (permalink / raw) To: Rik Van Riel; +Cc: Andrew Morton, linux-kernel On Tue, Jun 21, 2005 at 11:35:10PM -0400, Rik Van Riel wrote: > On Mon, 20 Jun 2005, Andrew Morton wrote: > > > git-ocfs > The only problem I can see with this is that people will want > to use OCFS together with CLVM, and both use a different cluster > infrastructure. > > IMHO it would be good if they both used the same underlying > cluster infrastructure... as clvm stuff and infra get submitted and come around we will definitely be working with it. but it's a feature more than a requirement. altho we have every intend to make our stuff work with clvm and I think some pieces of ocfs2 are already being used or looked at to be used for this infrastructure so it's looking good. Wim ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (configfs) 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (13 preceding siblings ...) 2005-06-22 3:35 ` OCFS (was Re: -mm " Rik Van Riel @ 2005-06-22 4:08 ` David Teigland 2005-06-22 4:19 ` Andrew Morton 2005-06-22 5:51 ` -mm -> 2.6.13 merge status Christoph Hellwig ` (3 subsequent siblings) 18 siblings, 1 reply; 620+ messages in thread From: David Teigland @ 2005-06-22 4:08 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel On 6/21/05, Andrew Morton <akpm@osdl.org> wrote: > git-ocfs > > The OCFS2 filesystem. OK by me, although I'm not sure it's had enough > review. Does this include configfs? I'd especially like to see that sooner rather than later. Dave ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status (configfs) 2005-06-22 4:08 ` -mm -> 2.6.13 merge status (configfs) David Teigland @ 2005-06-22 4:19 ` Andrew Morton 0 siblings, 0 replies; 620+ messages in thread From: Andrew Morton @ 2005-06-22 4:19 UTC (permalink / raw) To: teigland; +Cc: dteigland, linux-kernel David Teigland <dteigland@gmail.com> wrote: > > On 6/21/05, Andrew Morton <akpm@osdl.org> wrote: > > git-ocfs > > > > The OCFS2 filesystem. OK by me, although I'm not sure it's had enough > > review. > > Does this include configfs? I'd especially like to see that sooner > rather than later. There's not a lot of point in adding a fs which has no in-kernel users. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (14 preceding siblings ...) 2005-06-22 4:08 ` -mm -> 2.6.13 merge status (configfs) David Teigland @ 2005-06-22 5:51 ` Christoph Hellwig 2005-06-27 9:06 ` Christoph Hellwig ` (2 subsequent siblings) 18 siblings, 0 replies; 620+ messages in thread From: Christoph Hellwig @ 2005-06-22 5:51 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel > git-ocfs > > The OCFS2 filesystem. OK by me, although I'm not sure it's had enough > review. I'll try to take a look next week. A major blocker is that it's not endian-clean yet. Even if other review items where perfect that's something preventing it from going to mainline completely. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (15 preceding siblings ...) 2005-06-22 5:51 ` -mm -> 2.6.13 merge status Christoph Hellwig @ 2005-06-27 9:06 ` Christoph Hellwig 2005-06-27 14:25 ` Vladimir Saveliev 2005-06-27 19:30 ` -mm -> 2.6.13 merge status Christoph Hellwig 2005-06-27 20:19 ` Christoph Hellwig 18 siblings, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-06-27 9:06 UTC (permalink / raw) To: Andrew Morton, zam; +Cc: linux-kernel On Mon, Jun 20, 2005 at 11:54:58PM -0700, Andrew Morton wrote: > reiser4 So looking over the code a little for the plugin debate I found that a reason patch introduces a ->put_inode method in reiser4. We plan to kill ->put_inode because it's the wrong abstraction and almost impossible to use non-racy (reiser4 usage is racy). I had a discussion with someone from namesys how to solve this correctly, but I don't remember the details of either the solution or problem anymore. Unless someone else does let's describe the problem again so we can find a proper fix. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-27 9:06 ` Christoph Hellwig @ 2005-06-27 14:25 ` Vladimir Saveliev 2005-06-27 19:26 ` Christoph Hellwig 0 siblings, 1 reply; 620+ messages in thread From: Vladimir Saveliev @ 2005-06-27 14:25 UTC (permalink / raw) To: Christoph Hellwig; +Cc: Andrew Morton, zam, linux-kernel Hello On Mon, 2005-06-27 at 13:06, Christoph Hellwig wrote: > On Mon, Jun 20, 2005 at 11:54:58PM -0700, Andrew Morton wrote: > > reiser4 > > So looking over the code a little for the plugin debate I found that a > reason patch introduces a ->put_inode method in reiser4. We plan to > kill ->put_inode because it's the wrong abstraction and almost impossible > to use non-racy (reiser4 usage is racy). Sorry, would you please explain what is wrong in having the below if (inode->i_nlink != 0 || atomic_read(&inode->i_count) > 1) in reiser4_put_inode. > I had a discussion with someone from namesys how to solve this correctly, We were trying to avoid need to have reiser4_drop_inode because you said drop_inode is a hack for hugetlbfs. The problem is that on file removal reiser4 wants to do truncate_inode_pages in reiser4_delete_inode. We used reiser4_drop_inode for that. As long as drop_inode was about to die, we decided to do file predeletion in reiser4_put_inode when inode is about to get into iput_final with inode->i_nlink == 0. It is a pity that put_inode is also wrong abstraction. > but I don't remember the details of either the solution or problem anymore. You said: -------- So what you want is actually to move the truncate_inode_pages out of generic_delete_inode and into ->delete_inode? Looking at the code another strategt makes more sense: - move the truncate_inode_pages at the beginning of clear_inode. All callers but one already do it just before that call, but the one that doesn't will require a full audit of all ->delete_inode instances - make the first half of clear_inode into a helper (__clear_inode or whatever), and make ->clear_inode responsible for calling it as first thing for a normal fs or call it in clear_inode if ->clear_inode doesn't exist. that way we can also move the invalidate_inode_buffers out there easily later for filesystems that don't use buffer_heads at all. p.s. please try to keep -fsdevel Cc'ed on the mail related to core changes ------- I hoped that we can solve the problem internally in reiser4. If put_inode is about to be removed we will have to do ssomething like that. > Unless someone else does let's describe the problem again so we can find > a proper fix. > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-27 14:25 ` Vladimir Saveliev @ 2005-06-27 19:26 ` Christoph Hellwig 2005-06-27 19:44 ` Joel Becker 0 siblings, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-06-27 19:26 UTC (permalink / raw) To: Vladimir Saveliev; +Cc: Christoph Hellwig, Andrew Morton, zam, linux-kernel On Mon, Jun 27, 2005 at 06:25:43PM +0400, Vladimir Saveliev wrote: > Sorry, would you please explain what is wrong in having the below > > if (inode->i_nlink != 0 || atomic_read(&inode->i_count) > 1) > > in reiser4_put_inode. Between that atomic_read(&inode->i_count) and the atomic_dec_and_lock(&inode->i_count, &inode_lock)) in iput someone could have grabbed a reference. > The problem is that on file removal reiser4 wants to do > truncate_inode_pages in reiser4_delete_inode. We used reiser4_drop_inode > for that. As long as drop_inode was about to die, we decided to do file drop_inode is not going to die, we need it to support filesystems that want to call generic_delete_inode even for a non-null i_nlink. What's hopefully going to die is the last instance of it that isn't either generic_drop_inode or generic_delete_inode. > You said: > -------- > So what you want is actually to move the truncate_inode_pages out of > generic_delete_inode and into ->delete_inode? > > > Looking at the code another strategt makes more sense: > > - move the truncate_inode_pages at the beginning of clear_inode. > All callers but one already do it just before that call, but > the one that doesn't will require a full audit of all ->delete_inode > instances > - make the first half of clear_inode into a helper (__clear_inode or > whatever), and make ->clear_inode responsible for calling it as first > thing for a normal fs or call it in clear_inode if ->clear_inode > doesn't > exist. that way we can also move the invalidate_inode_buffers out > there > easily later for filesystems that don't use buffer_heads at all. > > p.s. please try to keep -fsdevel Cc'ed on the mail related to core > changes > ------- > > I hoped that we can solve the problem internally in reiser4. If > put_inode is about to be removed we will have to do ssomething like > that. In fact I know from some cluster filesystem folks that have a similar problems as yours. So getting the truncate_inode_pages under control of the filesystems sounds like a very good choice. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-27 19:26 ` Christoph Hellwig @ 2005-06-27 19:44 ` Joel Becker 2005-06-27 20:26 ` Christoph Hellwig 0 siblings, 1 reply; 620+ messages in thread From: Joel Becker @ 2005-06-27 19:44 UTC (permalink / raw) To: Christoph Hellwig, Andrew Morton, linux-kernel On Mon, Jun 27, 2005 at 08:26:51PM +0100, Christoph Hellwig wrote: > drop_inode is not going to die, we need it to support filesystems that > want to call generic_delete_inode even for a non-null i_nlink. What's > hopefully going to die is the last instance of it that isn't either > generic_drop_inode or generic_delete_inode. OCFS2 uses drop_inode as well, as it must handle last-close when another node did the unlink. It fixes up i_nlink in that case, then calls generic_drop_inode(). If there's a more elegant solution, we're all ears. Joel -- "When choosing between two evils, I always like to try the one I've never tried before." - Mae West Joel Becker Senior Member of Technical Staff Oracle E-mail: joel.becker@oracle.com Phone: (650) 506-8127 ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-27 19:44 ` Joel Becker @ 2005-06-27 20:26 ` Christoph Hellwig 2005-06-27 22:06 ` generic_drop_inode Mark Fasheh 0 siblings, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-06-27 20:26 UTC (permalink / raw) To: Andrew Morton, linux-kernel On Mon, Jun 27, 2005 at 12:44:02PM -0700, Joel Becker wrote: > On Mon, Jun 27, 2005 at 08:26:51PM +0100, Christoph Hellwig wrote: > > drop_inode is not going to die, we need it to support filesystems that > > want to call generic_delete_inode even for a non-null i_nlink. What's > > hopefully going to die is the last instance of it that isn't either > > generic_drop_inode or generic_delete_inode. > > OCFS2 uses drop_inode as well, as it must handle last-close when > another node did the unlink. It fixes up i_nlink in that case, then > calls generic_drop_inode(). > If there's a more elegant solution, we're all ears. I think this still qualifies as calling generic_delete_inode because it's a trivial wrapper. Manipulating i_nlink seems rather odd to me, I'd say you should rather call into generic_delete_inode directly if OCFS2_INODE_MAYBE_ORPHANED is set (that's what generic_drop_inode will do for i_nlink == 0 anyway). In fact given every cluster and possibly many network filesystems will need this it might make sense to take the OCFS2_INODE_MAYBE_ORPHANED into the VFS, i.e. make it an i_state flag (after fixing can_unuse to not do something totally stupid with i_state) ^ permalink raw reply [flat|nested] 620+ messages in thread
* generic_drop_inode 2005-06-27 20:26 ` Christoph Hellwig @ 2005-06-27 22:06 ` Mark Fasheh 2005-06-28 14:54 ` generic_drop_inode Christoph Hellwig 0 siblings, 1 reply; 620+ messages in thread From: Mark Fasheh @ 2005-06-27 22:06 UTC (permalink / raw) To: Christoph Hellwig, Andrew Morton, linux-kernel, joel.becker On Mon, Jun 27, 2005 at 09:26:48PM +0100, Christoph Hellwig wrote: > I think this still qualifies as calling generic_delete_inode because it's > a trivial wrapper. Manipulating i_nlink seems rather odd to me, I'd > say you should rather call into generic_delete_inode directly if > OCFS2_INODE_MAYBE_ORPHANED is set (that's what generic_drop_inode will > do for i_nlink == 0 anyway). Allowing the file system to make the full decision and call the proper function would be fine, but then we'd need generic_forget_inode exported too :) > In fact given every cluster and possibly many network filesystems will > need this it might make sense to take the OCFS2_INODE_MAYBE_ORPHANED into > the VFS, i.e. make it an i_state flag (after fixing can_unuse to not do > something totally stupid with i_state) Yes, the problem certainly isn't necessarily specific to OCFS2 so I'd be more than happy to see that as a generic VFS feature. As is, the export and callback get us past quite a few problems with tracking orphaned inodes in the cluster. This however, brings me to a related issue for which I'd appreciate some advice. If a node crashes or otherwise fails to complete the unlink(2) then the inode in qusetion will *not* have actually been orphaned. Of course, the local node doesn't know this and eventually gets to ocfs2_delete_inode() (via the MAYBE_ORPHANED flag) which will do the work of determining whether the inode is actually orphaned. The problem is that by the time we've gotten there, generic_delete_inode() has done a truncate_inode_pages() which might throw out data for a file which otherwise wants it on disk. In fact, because OCFS2 supports POSIX style unlink-while-open across the entire cluster, we might get to ocfs2_delete_inode() and during the voting process, discover that the file is still open on another node in which case we don't want to wipe it but would have lost file data due to the truncate_inode_pages() anyway. Essentially, we get to ocfs2_delete_inode() before we have a chance to take the cluster lock and determine the true state of the inode. By then it is too late. We really need to be able to make the determination before calling generic_drop_inode(), but drop_inode() is under the inode lock, and we can't call out to the cluster. The question becomes where and how to do this work? --Mark -- Mark Fasheh Senior Software Developer, Oracle mark.fasheh@oracle.com ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: generic_drop_inode 2005-06-27 22:06 ` generic_drop_inode Mark Fasheh @ 2005-06-28 14:54 ` Christoph Hellwig 0 siblings, 0 replies; 620+ messages in thread From: Christoph Hellwig @ 2005-06-28 14:54 UTC (permalink / raw) To: Mark Fasheh; +Cc: Christoph Hellwig, Andrew Morton, linux-kernel, joel.becker On Mon, Jun 27, 2005 at 03:06:12PM -0700, Mark Fasheh wrote: > Allowing the file system to make the full decision and call the proper > function would be fine, but then we'd need generic_forget_inode exported > too :) shouldn't be much of a problem if it's actually an _GPL export. > If a node crashes or otherwise fails to complete the unlink(2) then the > inode in qusetion will *not* have actually been orphaned. Of course, the > local node doesn't know this and eventually gets to ocfs2_delete_inode() (via > the MAYBE_ORPHANED flag) which will do the work of determining whether the > inode is actually orphaned. The problem is that by the time we've gotten > there, generic_delete_inode() has done a truncate_inode_pages() which might > throw out data for a file which otherwise wants it on disk. As discussed in a different subthread of the 2.6.13 merge plan we'd like to move towards not having truncate_inode_pages in that place, because reiser4, hugetlbfs and some other cluster filesystems don't like it either. But that alone isn't going to help you a lot.. > In fact, because OCFS2 supports POSIX style unlink-while-open across the > entire cluster, we might get to ocfs2_delete_inode() and during the voting > process, discover that the file is still open on another node in which case > we don't want to wipe it but would have lost file data due to the > truncate_inode_pages() anyway. > > Essentially, we get to ocfs2_delete_inode() before we have a chance to > take the cluster lock and determine the true state of the inode. By > then it is too late. We really need to be able to make the > determination before calling generic_drop_inode(), but drop_inode() is > under the inode lock, and we can't call out to the cluster. The > question becomes where and how to do this work? I don't think that's doable without a major rework of that area of code. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (16 preceding siblings ...) 2005-06-27 9:06 ` Christoph Hellwig @ 2005-06-27 19:30 ` Christoph Hellwig 2005-06-27 20:37 ` Hans Reiser 2005-06-27 20:19 ` Christoph Hellwig 18 siblings, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-06-27 19:30 UTC (permalink / raw) To: Andrew Morton, reiser; +Cc: linux-kernel > reiser4 sparse isn't to happy about this: hch@macfly:/work/linux-2.6.12$ make C=1 SUBDIRS=fs/reiser4/ >/dev/null 2>err.log && wc -l err.log 2286 err.log The log is at http://verein.lst.de/~hch/linux-2.6.12-mm2-fs-reiser4-errors.log ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-27 19:30 ` -mm -> 2.6.13 merge status Christoph Hellwig @ 2005-06-27 20:37 ` Hans Reiser 2005-06-30 18:30 ` Vitaly Fertman 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-27 20:37 UTC (permalink / raw) To: Christoph Hellwig Cc: Andrew Morton, linux-kernel, Reiserfs developers mail-list, vitaly Christoph Hellwig wrote: >>reiser4 >> >> > >sparse isn't to happy about this: > >hch@macfly:/work/linux-2.6.12$ make C=1 SUBDIRS=fs/reiser4/ >/dev/null 2>err.log && wc -l err.log >2286 err.log > >The log is at http://verein.lst.de/~hch/linux-2.6.12-mm2-fs-reiser4-errors.log > > > > Thanks for telling us about sparse, we will work on fixing these. Vitaly, can you do this? Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-27 20:37 ` Hans Reiser @ 2005-06-30 18:30 ` Vitaly Fertman 0 siblings, 0 replies; 620+ messages in thread From: Vitaly Fertman @ 2005-06-30 18:30 UTC (permalink / raw) To: Hans Reiser Cc: Christoph Hellwig, Andrew Morton, linux-kernel, Reiserfs developers mail-list On Tuesday 28 June 2005 00:37, Hans Reiser wrote: > Christoph Hellwig wrote: > > >>reiser4 > >> > >> > > > >sparse isn't to happy about this: > > > >hch@macfly:/work/linux-2.6.12$ make C=1 SUBDIRS=fs/reiser4/ >/dev/null 2>err.log && wc -l err.log > >2286 err.log > > > >The log is at http://verein.lst.de/~hch/linux-2.6.12-mm2-fs-reiser4-errors.log > > > > > > > > > Thanks for telling us about sparse, we will work on fixing these. > Vitaly, can you do this? fixed, will be included into the next mm kernel. -- Thanks, Vitaly Fertman ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton ` (17 preceding siblings ...) 2005-06-27 19:30 ` -mm -> 2.6.13 merge status Christoph Hellwig @ 2005-06-27 20:19 ` Christoph Hellwig 2005-07-13 18:23 ` (v9fs) " Eric Van Hensbergen 18 siblings, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-06-27 20:19 UTC (permalink / raw) To: Andrew Morton, ericvh, rminnich; +Cc: linux-kernel > v9fs > > I'm not sure that this has a sufficiently high > usefulness-to-maintenance-cost ratio. Personally I think this is very useful to have. It provides a portable way to have simple userland or remote filesystems, and it's been around for a long time in others OSes. That beeing said there's a few issues with the code still I'd like to see fixed: - there's three sparse warnings still. Two of them are easily fixed by moving externs to headers, one doesn't look fixable until we get a sane in-kernel api for socket operations - some dentry handling looks rather odd. Why are you for example calling d_drop in v9fs_vfs_symlink, v9fs_vfs_mknod and v9fs_vfs_link? Shouldn't all these call d_instantatiate to actually reuse the dentry as in v9fs_vfs_create? Also what's the issue with v9fs_fid_insert? It would seem better and more logical to me to always set d_fsdata in create/mknod/symlink/open before hashing it and then beeing able to rely on it beeing non-NULL. - buf_check_sizep, buf_check_size and buf_check_sizev should be made inlines, and lose the implict return. Please don't hide such things in macros - please avoid using hlist_for_each, usually hlist_for_each_entry is a much better choice - dito for list_for_each_safe vs list_for_each_entry_safe - can you please check whether lib/idr.c fullfills your needs so we can get rid of idpool.c? - v9fs_inode2v9ses has lots of useless checks, inode->i_sb can never be NULL, and inode->i_sb->s_fs_info can't be either once set in fill_inode, which is before the first inode on the filesystem is created. Also the argument is never NULL. Because of that you can also kill all the return value checks in the callers. - do you really need to keep v9fs_dentry_delete just for the dprintk? - no need to check for a NULL file in v9fs_dir_readdir, the VFS gurantees it's not. And if it was you'd better be off panic because something is enormously fscked. - Dito for v9fs_file_open - And the inode in v9fs_file_lock - And dir, file, file->d_inode, sb, v9ses in v9fs_remove. - And dir, sb and v9ses in v9fs_vfs_lookup - And dir, sb and v9ses in v9fs_vfs_symlink - And dir, sb and v9ses in v9fs_vfs_link - And dir, sb and v9ses in v9fs_vfs_mknod - copy_from_user returns the bytes actually copied in the failure case, but you should return -EFAULT instead of that number in v9fs_file_write - No need to implement v9fs_file_mmap, do_mmap_pgoff makes sure to error out if it's not present (and actually returns the correct errno) - I think it's pretty similar for all these checks for fid (=private_data) checks. You always set them in open, so they can't be NULL - kfree can be called with a NULL argument just fine, you can remove lots of ifs for that. You also often set pointers to NULL just before freeing a structure - that's pretty useless as slab debugging will catch bugs with stary references very well, and overwrites these NULLs ASAP. - The call to ->put_inode in the error case of v9fs_get_inode is very wrong. You'd actually panic if you ever hit this as v9fs doesn't implement a ->put_inode :-) - All the ISDIR checks in v9fs_remove can go, VFS makes sure to only call ->remove and ->rmdir on directories, and only the right one for each kind of child. - Please try to use generic_readlink instead of your own v9fs_vfs_readlink, as you're implemting ->follow_link and ->put_link that should just work - the last error case in v9fs_get_sb needs a dput on ->s_root Also did you look into the VFS/NFS lookup intent bits to solve your atomic create and open issue? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: (v9fs) -mm -> 2.6.13 merge status 2005-06-27 20:19 ` Christoph Hellwig @ 2005-07-13 18:23 ` Eric Van Hensbergen 2005-07-14 20:04 ` Christoph Hellwig 0 siblings, 1 reply; 620+ messages in thread From: Eric Van Hensbergen @ 2005-07-13 18:23 UTC (permalink / raw) To: Christoph Hellwig, Andrew Morton, ericvh, rminnich, linux-kernel, v9fs-developer On 6/27/05, Christoph Hellwig <hch@infradead.org> wrote: > > That beeing said there's a few issues with the code still I'd like to > see fixed: > Sorry I didn't get to these quicker - was on vacation and basically off-line for the past week and a half. I've made 90% of the changes suggested and committed them to my git tree, I'll combine the changes into a single patch and then split them by file-group before sending them to akpm to more closely match the existing patches. The 10% I didn't address I'll comment on below, most of them represent harder problems that I'd like to think about a bit more. > > - there's three sparse warnings still. Two of them are easily fixed > by moving externs to headers, one doesn't look fixable until we get > a sane in-kernel api for socket operations done > - some dentry handling looks rather odd. Why are you for example > calling d_drop in v9fs_vfs_symlink, v9fs_vfs_mknod and v9fs_vfs_link? > Shouldn't all these call d_instantatiate to actually reuse the > dentry as in v9fs_vfs_create? Also what's the issue with > v9fs_fid_insert? It would seem better and more logical to me to > always set d_fsdata in create/mknod/symlink/open before hashing it > and then beeing able to rely on it beeing non-NULL. All of this is kind of tricky due to the association of fids with dentry elements and the special way we handle certain features (such as special files and symlinks). The current code aggressively invalidates fids to prevent the dcache from masking operations that may be semantically important to synthetic file systems. If you look in v9fs_create we actually d_drop the dentry for created directories as well. The only reason we don't d_drop normal files is because we are trying to preserve the atomic create/open semantics. I'm not 100% confident this is the right solution, but its the closest I've been able to come so far -- there's actually been a fair amount of discussion on this in the v9fs-developer's list. If you want more details, it's probably worth a separate thread to discuss the reasons behind why we want to aggressively invalidate the dcache and how we have tried to accomplish this -- or we could just catch up at OLS. > - buf_check_sizep, buf_check_size and buf_check_sizev should be made > inlines, and lose the implict return. Please don't hide such > things in macros done > - please avoid using hlist_for_each, usually hlist_for_each_entry is > a much better choice > - dito for list_for_each_safe vs list_for_each_entry_safe done > - can you please check whether lib/idr.c fullfills your needs so we > can get rid of idpool.c? Last time I looked idr didn't do exactly what I wanted, but looking over it again I realize its just doing more than I want -- so I've eliminated idpool.*, but still have wrapper functions to encapsulate locking and retry -- it strikes me that there may be a case for generalizing these wrapper functions and putting them in lib/idr.c, but figured that could wait. > - v9fs_inode2v9ses has lots of useless checks, inode->i_sb can never > be NULL, and inode->i_sb->s_fs_info can't be either once set in > fill_inode, which is before the first inode on the filesystem is > created. Also the argument is never NULL. Because of that you > can also kill all the return value checks in the callers. > - do you really need to keep v9fs_dentry_delete just for the dprintk? > - no need to check for a NULL file in v9fs_dir_readdir, the VFS gurantees > it's not. And if it was you'd better be off panic because something > is enormously fscked. > - Dito for v9fs_file_open > - And the inode in v9fs_file_lock > - And dir, file, file->d_inode, sb, v9ses in v9fs_remove. > - And dir, sb and v9ses in v9fs_vfs_lookup > - And dir, sb and v9ses in v9fs_vfs_symlink > - And dir, sb and v9ses in v9fs_vfs_link > - And dir, sb and v9ses in v9fs_vfs_mknod Yeah, all of these were sanity checks during initial development while I was still understanding the VFS API. I think I got most of them this time. > - copy_from_user returns the bytes actually copied in the failure case, > but you should return -EFAULT instead of that number in v9fs_file_write fixed > - No need to implement v9fs_file_mmap, do_mmap_pgoff makes sure to error > out if it's not present (and actually returns the correct errno) > - I think it's pretty similar for all these checks for fid (=private_data) > checks. You always set them in open, so they can't be NULL > - kfree can be called with a NULL argument just fine, you can remove > lots of ifs for that. You also often set pointers to NULL just before > freeing a structure - that's pretty useless as slab debugging will > catch bugs with stary references very well, and overwrites these NULLs > ASAP. > - The call to ->put_inode in the error case of v9fs_get_inode is very > wrong. You'd actually panic if you ever hit this as v9fs doesn't > implement a ->put_inode :-) > - All the ISDIR checks in v9fs_remove can go, VFS makes sure to only > call ->remove and ->rmdir on directories, and only the right one > for each kind of child. done > - Please try to use generic_readlink instead of your own > v9fs_vfs_readlink, as you're implemting ->follow_link and ->put_link > that should just work I tried this and the kernel crashed - which may mean I've done something wrong in follow_link or put_link. I'll revisit this and try to figure out what happened, but for now I've left my readlink in. > - the last error case in v9fs_get_sb needs a dput on ->s_root done > > Also did you look into the VFS/NFS lookup intent bits to solve your > atomic create and open issue? > I had considered the lookup intent bits to solve several different problems associated with our fid handling - but never really felt confident in my understanding of how the intent stuff was supposed to work. It is certainly worth revisiting now that things have calmed down a bit. -eric ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: (v9fs) -mm -> 2.6.13 merge status 2005-07-13 18:23 ` (v9fs) " Eric Van Hensbergen @ 2005-07-14 20:04 ` Christoph Hellwig 2005-07-14 22:12 ` Alexey Dobriyan 0 siblings, 1 reply; 620+ messages in thread From: Christoph Hellwig @ 2005-07-14 20:04 UTC (permalink / raw) To: Eric Van Hensbergen; +Cc: Andrew Morton, rminnich, linux-kernel, v9fs-developer On Wed, Jul 13, 2005 at 01:23:24PM -0500, Eric Van Hensbergen wrote: > Sorry I didn't get to these quicker - was on vacation and basically > off-line for the past week and a half. I've made 90% of the changes > suggested and committed them to my git tree, I'll combine the changes > into a single patch and then split them by file-group before sending normally we prefer a patch per actual change, not per file so the description fits. Given that all these are pretty trivial fixes one patch would have done it aswell, though. With these changes the code is fine for mainline in my opinion. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: (v9fs) -mm -> 2.6.13 merge status 2005-07-14 20:04 ` Christoph Hellwig @ 2005-07-14 22:12 ` Alexey Dobriyan 2005-07-14 22:15 ` Eric Van Hensbergen 0 siblings, 1 reply; 620+ messages in thread From: Alexey Dobriyan @ 2005-07-14 22:12 UTC (permalink / raw) To: Eric Van Hensbergen Cc: Christoph Hellwig, Andrew Morton, rminnich, linux-kernel, v9fs-developer On Friday 15 July 2005 00:04, Christoph Hellwig wrote: > normally we prefer a patch per actual change, not per file so the > description fits. Given that all these are pretty trivial fixes one > patch would have done it aswell, though. > > With these changes the code is fine for mainline in my opinion. Can I make one more nitpicking comment? All these functions can use cpu_to_le*() and le*_to_cpu(). > --- /dev/null > +++ 25-akpm/fs/9p/conv.c > +static inline void buf_put_int16(struct cbuf *buf, u16 val) > +{ > + buf_check_sizev(buf, 2); > + > + buf->p[0] = val; > + buf->p[1] = val >> 8; > + buf->p += 2; > +} > + > +static inline void buf_put_int32(struct cbuf *buf, u32 val) > +{ > + buf_check_sizev(buf, 4); > + > + buf->p[0] = val; > + buf->p[1] = val >> 8; > + buf->p[2] = val >> 16; > + buf->p[3] = val >> 24; > + buf->p += 4; > +} > + > +static inline void buf_put_int64(struct cbuf *buf, u64 val) > +{ > + buf_check_sizev(buf, 8); > + > + buf->p[0] = val; > + buf->p[1] = val >> 8; > + buf->p[2] = val >> 16; > + buf->p[3] = val >> 24; > + buf->p[4] = val >> 32; > + buf->p[5] = val >> 40; > + buf->p[6] = val >> 48; > + buf->p[7] = val >> 56; > + buf->p += 8; > +} > +static inline u16 buf_get_int16(struct cbuf *buf) > +{ > + u16 ret = 0; > + > + buf_check_size(buf, 2); > + ret = buf->p[0] | (buf->p[1] << 8); > + > + buf->p += 2; > + > + return ret; > +} > + > +static inline u32 buf_get_int32(struct cbuf *buf) > +{ > + u32 ret = 0; > + > + buf_check_size(buf, 4); > + ret = > + buf->p[0] | (buf->p[1] << 8) | (buf->p[2] << 16) | (buf-> > + p[3] << 24); > + > + buf->p += 4; > + > + return ret; > +} > + > +static inline u64 buf_get_int64(struct cbuf *buf) > +{ > + u64 ret = 0; > + > + buf_check_size(buf, 8); > + ret = (u64) buf->p[0] | ((u64) buf->p[1] << 8) | > + ((u64) buf->p[2] << 16) | ((u64) buf->p[3] << 24) | > + ((u64) buf->p[4] << 32) | ((u64) buf->p[5] << 40) | > + ((u64) buf->p[6] << 48) | ((u64) buf->p[7] << 56); > + > + buf->p += 8; > + > + return ret; > +} ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: (v9fs) -mm -> 2.6.13 merge status 2005-07-14 22:12 ` Alexey Dobriyan @ 2005-07-14 22:15 ` Eric Van Hensbergen 0 siblings, 0 replies; 620+ messages in thread From: Eric Van Hensbergen @ 2005-07-14 22:15 UTC (permalink / raw) To: Alexey Dobriyan Cc: Christoph Hellwig, Andrew Morton, rminnich, linux-kernel, v9fs-developer On 7/14/05, Alexey Dobriyan <adobriyan@gmail.com> wrote: > On Friday 15 July 2005 00:04, Christoph Hellwig wrote: > > normally we prefer a patch per actual change, not per file so the > > description fits. Given that all these are pretty trivial fixes one > > patch would have done it aswell, though. > > > > With these changes the code is fine for mainline in my opinion. > > Can I make one more nitpicking comment? > > All these functions can use cpu_to_le*() and le*_to_cpu(). > I need to rethink some parts of conv.c, I'll incorporate your suggestion during the rework. Thanks Alexey. -eric ^ permalink raw reply [flat|nested] 620+ messages in thread
[parent not found: <20050620235458.5b437274.akpm@osdl.org.suse.lists.linux.kernel>]
* Re: -mm -> 2.6.13 merge status [not found] <20050620235458.5b437274.akpm@osdl.org.suse.lists.linux.kernel> @ 2005-06-21 11:14 ` Andi Kleen 2005-06-21 18:44 ` Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: Andi Kleen @ 2005-06-21 11:14 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel Andrew Morton <akpm@osdl.org> writes: > perfctr > > Not yet, but getting closer. The PPC64 guys still need to sort out a > few interface issues with Mikael. We might be able to fit this into > 2.6.13 if people get a move on. So the problems IA64 had with this are resolved now? Unfortunately there is a perfmon for i386/x86-64 implementation floating around now (with some unmergeable stuff but might be fixable) which is kind of competing now :/ > reiser4 > > Merge it, I guess. > > The patches still contain all the reiser4-specific namespace > enhancements, only it is disabled, so it is effectively dead code. Maybe > we should ask that it actually be removed? Has there been actually any serious review on this? Last time I looked there was a lot of very ugly code in there. Also I'm not sure things like comming with an own profiler and spinlock debugger are really acceptable. At least this stuff should be removed too. -Andi ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 11:14 ` Andi Kleen @ 2005-06-21 18:44 ` Hans Reiser 2005-06-21 19:56 ` Andi Kleen 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-21 18:44 UTC (permalink / raw) To: Andi Kleen, Alexander Zarochentcev, vs; +Cc: Andrew Morton, linux-kernel vs and zam, please comment on what we get from our profiler and spinlock debugger that the standard tools don't supply. I am sure you have a reason, but now is the time to articulate it. We would like to keep the disabled code in there until we have a chance to prove (or fail to prove) that cycle detection can be resolved effectively, and then with a solution in hand argue its merits. Hans Andi Kleen wrote: > Also I'm not sure things like comming with an own profiler > >and spinlock debugger are really acceptable. At least this stuff >should be removed too. > >-Andi > >- >To unsubscribe from this list: send the line "unsubscribe linux-kernel" in >the body of a message to majordomo@vger.kernel.org >More majordomo info at http://vger.kernel.org/majordomo-info.html >Please read the FAQ at http://www.tux.org/lkml/ > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 18:44 ` Hans Reiser @ 2005-06-21 19:56 ` Andi Kleen 2005-06-21 20:21 ` Christoph Hellwig 2005-06-22 1:38 ` Hans Reiser 0 siblings, 2 replies; 620+ messages in thread From: Andi Kleen @ 2005-06-21 19:56 UTC (permalink / raw) To: Hans Reiser Cc: Andi Kleen, Alexander Zarochentcev, vs, Andrew Morton, linux-kernel On Tue, Jun 21, 2005 at 11:44:55AM -0700, Hans Reiser wrote: > vs and zam, please comment on what we get from our profiler and spinlock > debugger that the standard tools don't supply. I am sure you have a > reason, but now is the time to articulate it. > > We would like to keep the disabled code in there until we have a chance > to prove (or fail to prove) that cycle detection can be resolved > effectively, and then with a solution in hand argue its merits. How about the review of your code base? Has reiser4 ever been fully reviewed by people outside your group? Normally full review is a requirement for merging. -Andi ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 19:56 ` Andi Kleen @ 2005-06-21 20:21 ` Christoph Hellwig 2005-06-22 1:38 ` Hans Reiser 1 sibling, 0 replies; 620+ messages in thread From: Christoph Hellwig @ 2005-06-21 20:21 UTC (permalink / raw) To: Andi Kleen Cc: Hans Reiser, Alexander Zarochentcev, vs, Andrew Morton, linux-kernel On Tue, Jun 21, 2005 at 09:56:43PM +0200, Andi Kleen wrote: > On Tue, Jun 21, 2005 at 11:44:55AM -0700, Hans Reiser wrote: > > vs and zam, please comment on what we get from our profiler and spinlock > > debugger that the standard tools don't supply. I am sure you have a > > reason, but now is the time to articulate it. > > > > We would like to keep the disabled code in there until we have a chance > > to prove (or fail to prove) that cycle detection can be resolved > > effectively, and then with a solution in hand argue its merits. > > How about the review of your code base? Has reiser4 ever been > fully reviewed by people outside your group? I don't think so. Everyone used the previous criteria of the broken core changes, broken filesystem semantics and it's own useless abtraction layer (*) as an excuse not to look deeply at this huge mess yet. (*) which isn't gone yet. and I need to look again if the core changes are okay yet. And the broken sematics should go completely aswell, if you want to reintroduce them it needs to happen at the VFS level anyway. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-21 19:56 ` Andi Kleen 2005-06-21 20:21 ` Christoph Hellwig @ 2005-06-22 1:38 ` Hans Reiser 2005-06-22 1:57 ` Jeff Garzik ` (2 more replies) 1 sibling, 3 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-22 1:38 UTC (permalink / raw) To: Andi Kleen, Alexander Lyamin aka FLX Cc: Alexander Zarochentcev, vs, Andrew Morton, linux-kernel, ReiserFS List Andi Kleen wrote: >On Tue, Jun 21, 2005 at 11:44:55AM -0700, Hans Reiser wrote: > > >>vs and zam, please comment on what we get from our profiler and spinlock >>debugger that the standard tools don't supply. I am sure you have a >>reason, but now is the time to articulate it. >> >>We would like to keep the disabled code in there until we have a chance >>to prove (or fail to prove) that cycle detection can be resolved >>effectively, and then with a solution in hand argue its merits. >> >> > >How about the review of your code base? Has reiser4 ever been >fully reviewed by people outside your group? > >Normally full review is a requirement for merging. > > V4 has a mailing list, and a large number of testers who read the code and comment on it. V4 has been reviewed and tested much more than V3 was before merging. Given that we sent it in quite some time ago, your suggestion that an additional review by unspecified additional others be a requirement for merging seems untimely. Do you see my point of view on this? I would however enjoy receiving coding suggestions at ANY time. We don't get as much of that as I would like. I would in particular love to have you Andi Kleen do a full review of V4 if you could be that generous with your time, as I liked much of the advice you gave us on V3. Unspecified others doing a review, well, who knows, I will surely take the time to consider what is said by them though..... I would prefer to not get reviews from authors of other filesystems who prefer their own code, skim through our code without taking the time to grok our philosophy and approach in depth, and then complain that our code is different from what they chose to write, and think that our changing to be like them should be mandated. I will not name names here.... Some of the suggestions on our mailing list are great, some reflect a lack of 5 years working with our code, perhaps I should feed our mailing list into the linux kernel mailing list so that people on the kernel mailing list are more aware that we exist and are active? >-Andi > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-22 1:38 ` Hans Reiser @ 2005-06-22 1:57 ` Jeff Garzik 2005-06-22 1:57 ` Andi Kleen 2005-06-23 6:22 ` Pekka Enberg 2 siblings, 0 replies; 620+ messages in thread From: Jeff Garzik @ 2005-06-22 1:57 UTC (permalink / raw) To: Hans Reiser Cc: Andi Kleen, Alexander Lyamin aka FLX, Alexander Zarochentcev, vs, Andrew Morton, linux-kernel, ReiserFS List Hans Reiser wrote: > V4 has a mailing list, and a large number of testers who read the code > and comment on it. V4 has been reviewed and tested much more than V3 > was before merging. Given that we sent it in quite some time ago, your > suggestion that an additional review by unspecified additional others be > a requirement for merging seems untimely. Do you see my point of view > on this? > > I would however enjoy receiving coding suggestions at ANY time. We > don't get as much of that as I would like. I would in particular love > to have you Andi Kleen do a full review of V4 if you could be that > generous with your time, as I liked much of the advice you gave us on V3. > > Unspecified others doing a review, well, who knows, I will surely take > the time to consider what is said by them though..... > > I would prefer to not get reviews from authors of other filesystems who > prefer their own code, skim through our code without taking the time to > grok our philosophy and approach in depth, and then complain that our > code is different from what they chose to write, and think that our > changing to be like them should be mandated. I will not name names here.... The Linux system isn't the greatest in the world, but here's reality: when a merge is imminent, a lot more attention is paid. Andrew regularly uses this knowledge of human psychology to his (and Linux's) benefit :) The MAJOR downside is that merge-stopping issues are sometimes ignored until an upstream merge is imminent. If you want to get your code merged, you gotta work with the system, and LISTEN to the feedback. Jeff, who doesn't have a filesystem axe to grind ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-22 1:38 ` Hans Reiser 2005-06-22 1:57 ` Jeff Garzik @ 2005-06-22 1:57 ` Andi Kleen 2005-06-22 2:55 ` Hans Reiser 2005-06-23 6:22 ` Pekka Enberg 2 siblings, 1 reply; 620+ messages in thread From: Andi Kleen @ 2005-06-22 1:57 UTC (permalink / raw) To: Hans Reiser Cc: Andi Kleen, Alexander Lyamin aka FLX, Alexander Zarochentcev, vs, Andrew Morton, linux-kernel, ReiserFS List On Tue, Jun 21, 2005 at 06:38:07PM -0700, Hans Reiser wrote: > V4 has a mailing list, and a large number of testers who read the code > and comment on it. V4 has been reviewed and tested much more than V3 > was before merging. Given that we sent it in quite some time ago, your > suggestion that an additional review by unspecified additional others be > a requirement for merging seems untimely. Do you see my point of view > on this? The point of the merge review is that people who are familiar with the existing Linux code double check that the way your code interfacts with the rest of the kernel is sane, does not have obvious bugs and follows the existing good practice. Once the code is in mainline it will get maintained and fixed when needed, but to make this possible without undue work on the mainline hackers it is needed to do a full review first. AFAIK reiserfs has not gotten such a full review yet. Also good reviewers are rare so it is not a good idea to be picky here. > Unspecified others doing a review, well, who knows, I will surely take > the time to consider what is said by them though..... > > I would prefer to not get reviews from authors of other filesystems who > prefer their own code, skim through our code without taking the time to > grok our philosophy and approach in depth, and then complain that our > code is different from what they chose to write, and think that our > changing to be like them should be mandated. I will not name names here.... Someone qualified to review a new file system for inclusion will have need necessary some experience in Linux file systems, and that can be hardly gotten without ever having touched one. So you will have to live with other file system authors commenting on your code. The main philosophy that is of concern here is also the philosophy of the core VFS and other kernel interfaces. > Some of the suggestions on our mailing list are great, some reflect a > lack of 5 years working with our code, perhaps I should feed our mailing > list into the linux kernel mailing list so that people on the kernel > mailing list are more aware that we exist and are active? Nobody doubts that you are active. Just there are doubts that your code follows the Linux coding standards enough to not be a undue mainteance burden in mainline. A review with the following changes could probably fix that. -Andi ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-22 1:57 ` Andi Kleen @ 2005-06-22 2:55 ` Hans Reiser 2005-06-22 3:01 ` Jeff Garzik 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-22 2:55 UTC (permalink / raw) To: Andi Kleen Cc: Alexander Lyamin aka FLX, Alexander Zarochentcev, vs, Andrew Morton, linux-kernel, ReiserFS List Andi Kleen wrote: > > Just there are doubts that your >code follows the Linux coding standards enough to not be a undue >mainteance burden in mainline. > We get only a few bugfixes from outsiders, and the rest are done by us. The customers who buy licenses in addition to the GPL from us for hundreds of thousands of dollars tend to make remarks to the effect of "we licensed your code for more money in part because it was way easier to work on than XXX linux filesystem". I like feedback on our code, and I particularly like feedback from a Mr. Andi Kleen, but there is no need to tie it to merging. If, however, it serves as an effective excuse to get some of your time allocated by SuSE management, sure, go for it.;-) Hans ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-22 2:55 ` Hans Reiser @ 2005-06-22 3:01 ` Jeff Garzik 2005-06-22 8:09 ` Hans Reiser 0 siblings, 1 reply; 620+ messages in thread From: Jeff Garzik @ 2005-06-22 3:01 UTC (permalink / raw) To: Hans Reiser Cc: Andi Kleen, Alexander Lyamin aka FLX, Alexander Zarochentcev, vs, Andrew Morton, linux-kernel, ReiserFS List Hans Reiser wrote: > I like feedback on our code, and I particularly like feedback from a Mr. > Andi Kleen, but there is no need to tie it to merging. If, however, it > serves as an effective excuse to get some of your time allocated by SuSE > management, sure, go for it.;-) All merges of new code go like this. You've been around here for a while, this should not be a shock. "Hans' team says its good stuff" is not a criteria for merging. Jeff ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-22 3:01 ` Jeff Garzik @ 2005-06-22 8:09 ` Hans Reiser 2005-06-22 8:24 ` Jeff Garzik 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-22 8:09 UTC (permalink / raw) To: Jeff Garzik Cc: Andi Kleen, Alexander Lyamin aka FLX, Alexander Zarochentcev, vs, Andrew Morton, linux-kernel, ReiserFS List Jeff Garzik wrote: > > > "Hans' team says its good stuff" is not a criteria for merging. > > Try benchmarking it. Maybe benchmarks mean more than our chattering..... at least to the users..... ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-22 8:09 ` Hans Reiser @ 2005-06-22 8:24 ` Jeff Garzik 0 siblings, 0 replies; 620+ messages in thread From: Jeff Garzik @ 2005-06-22 8:24 UTC (permalink / raw) To: Hans Reiser Cc: Andi Kleen, Alexander Lyamin aka FLX, Alexander Zarochentcev, vs, Andrew Morton, linux-kernel, ReiserFS List Hans Reiser wrote: > Jeff Garzik wrote: >>"Hans' team says its good stuff" is not a criteria for merging. > Try benchmarking it. Maybe benchmarks mean more than our > chattering..... at least to the users..... Still not a criteria for merging. We have to care about the code behind the benchmarks. Jeff ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-22 1:38 ` Hans Reiser 2005-06-22 1:57 ` Jeff Garzik 2005-06-22 1:57 ` Andi Kleen @ 2005-06-23 6:22 ` Pekka Enberg 2005-06-23 7:42 ` Hans Reiser 2005-06-23 18:37 ` Jeff Mahoney 2 siblings, 2 replies; 620+ messages in thread From: Pekka Enberg @ 2005-06-23 6:22 UTC (permalink / raw) To: Hans Reiser Cc: Andi Kleen, Alexander Lyamin aka FLX, Alexander Zarochentcev, vs, Andrew Morton, linux-kernel, ReiserFS List, Pekka Enberg Hi Hans, On 6/22/05, Hans Reiser <reiser@namesys.com> wrote: > I would in particular love to have you Andi Kleen do a full review of V4 > if you could be that generous with your time, as I liked much of the > advice you gave us on V3. Well, I am not Andi Kleen and this is not even in the ballpark of a full review but here goes... P.S. Would it be possible to have a version without the plugin stuff submitted (and perhaps file as directory)? It would make much more sense for the rest of us to review reiser4 without the most controversial bits and get the bits that people agree on merged. Pekka > --- /dev/null 2003-09-23 21:59:22.000000000 +0400 > +++ linux-2.6.11-vs/fs/reiser4/pool.c 2005-06-03 17:49:38.668204642 +0400 > +/* initialise new pool */ > +reiser4_internal void > +reiser4_init_pool(reiser4_pool * pool /* pool to initialise */ , > + size_t obj_size /* size of objects in @pool */ , > + int num_of_objs /* number of preallocated objects */ , > + char *data /* area for preallocated objects */ ) > +{ > + reiser4_pool_header *h; > + int i; > + > + assert("nikita-955", pool != NULL); These assertion codes are meaningless to the rest of us so please drop them. > --- /dev/null 2003-09-23 21:59:22.000000000 +0400 > +++ linux-2.6.11-vs/fs/reiser4/type_safe_hash.h 2005-06-03 17:49:38.751209197 +0400 > @@ -0,0 +1,320 @@ > +/* Copyright 2001, 2002, 2003 by Hans Reiser, licensing governed by > + * reiser4/README */ > + > +/* A hash table class that uses hash chains (singly-linked) and is > + parametrized to provide type safety. */ This belongs to include/linux, not reiser4. > --- /dev/null 2003-09-23 21:59:22.000000000 +0400 > +++ linux-2.6.11-vs/fs/reiser4/type_safe_list.h 2005-06-03 17:49:38.755209417 +0400 > @@ -0,0 +1,436 @@ > +/* Copyright 2001, 2002, 2003 by Hans Reiser, licensing governed by reiser4/README */ > + > +#ifndef __REISER4_TYPE_SAFE_LIST_H__ > +#define __REISER4_TYPE_SAFE_LIST_H__ > + > +#include "debug.h" > +/* A circular doubly linked list that differs from the previous > + <linux/list.h> implementation because it is parametrized to provide > + type safety. This data structure is also useful as a queue or stack. This belongs to include/linux, not reiser4. > --- /dev/null 2003-09-23 21:59:22.000000000 +0400 > +++ linux-2.6.11-vs/fs/reiser4/vfs_ops.c 2005-06-03 17:51:28.110210237 +0400 > +/* ->get_sb() method of file_system operations. */ > +static struct super_block * > +reiser4_get_sb(struct file_system_type *fs_type /* file > + * system > + * type */ , > + int flags /* flags */ , > + const char *dev_name /* device name */ , > + void *data /* mount options */ ) Please drop the useless parameter comments. > +/* > + * Initialization stages for reiser4. > + * > + * These enumerate various things that have to be done during reiser4 > + * startup. Initialization code (init_reiser4()) keeps track of what stage was > + * reached, so that proper undo can be done if error occurs during > + * initialization. > + */ > +typedef enum { > + INIT_NONE, /* nothing is initialized yet */ > + INIT_INODECACHE, /* inode cache created */ > + INIT_CONTEXT_MGR, /* list of active contexts created */ > + INIT_ZNODES, /* znode slab created */ > + INIT_PLUGINS, /* plugins initialized */ > + INIT_PLUGIN_SET, /* psets initialized */ > + INIT_TXN, /* transaction manager initialized */ > + INIT_FAKES, /* fake inode initialized */ > + INIT_JNODES, /* jnode slab initialized */ > + INIT_EFLUSH, /* emergency flush initialized */ > + INIT_FQS, /* flush queues initialized */ > + INIT_DENTRY_FSDATA, /* dentry_fsdata slab initialized */ > + INIT_FILE_FSDATA, /* file_fsdata slab initialized */ > + INIT_D_CURSOR, /* d_cursor suport initialized */ > + INIT_FS_REGISTERED, /* reiser4 file system type registered */ > +} reiser4_init_stage; Please use regular gotos instead. This is a silly hack especially since you don't have release function for all of the stages. > +reiser4_internal void reiser4_handle_error(void) > +{ > + struct super_block *sb = reiser4_get_current_sb(); > + > + if ( !sb ) > + return; > + reiser4_status_write(REISER4_STATUS_DAMAGED, 0, "Filesystem error occured"); > + switch ( get_super_private(sb)->onerror ) { > + case 0: > + reiser4_panic("foobar-42", "Filesystem error occured\n"); > + case 1: > + if ( sb->s_flags & MS_RDONLY ) > + return; > + sb->s_flags |= MS_RDONLY; > + break; > + case 2: > + machine_restart(NULL); Probably not something you should do at fs level. > +/* free this znode */ > +reiser4_internal void > +zfree(znode * node /* znode to free */ ) > +{ > + assert("nikita-465", node != NULL); > + assert("nikita-2120", znode_page(node) == NULL); > + assert("nikita-2301", owners_list_empty(&node->lock.owners)); > + assert("nikita-2302", requestors_list_empty(&node->lock.requestors)); > + assert("nikita-2663", capture_list_is_clean(ZJNODE(node)) && NODE_LIST(ZJNODE(node)) == NOT_CAPTURED); > + assert("nikita-2773", !JF_ISSET(ZJNODE(node), JNODE_EFLUSH)); > + assert("nikita-3220", list_empty(&ZJNODE(node)->jnodes)); > + assert("nikita-3293", !znode_is_right_connected(node)); > + assert("nikita-3294", !znode_is_left_connected(node)); > + assert("nikita-3295", node->left == NULL); > + assert("nikita-3296", node->right == NULL); Are all these still required? Seems bit too defensive for the kernel. > + > + > + /* not yet phash_jnode_destroy(ZJNODE(node)); */ > + > + /* poison memory. */ > + ON_DEBUG(memset(node, 0xde, sizeof *node)); Poisoning belongs to slab, not fs. > +/* allocate fresh znode */ > +reiser4_internal znode * > +zalloc(int gfp_flag /* allocation flag */ ) > +{ > + znode *node; > + > + node = kmem_cache_alloc(znode_slab, gfp_flag); > + return node; > +} Please drop this useless wrapper. > +reiser4_internal void > +znode_remove(znode * node /* znode to remove */ , reiser4_tree * tree) > +{ > + assert("nikita-2108", node != NULL); > + assert("nikita-470", node->c_count == 0); > + assert("zam-879", rw_tree_is_write_locked(tree)); > + > + /* remove reference to this znode from cbk cache */ > + cbk_cache_invalidate(node, tree); > + > + /* update c_count of parent */ > + if (znode_parent(node) != NULL) { > + assert("nikita-472", znode_parent(node)->c_count > 0); > + /* father, onto your hands I forward my spirit... */ > + znode_parent(node)->c_count --; > + node->in_parent.node = NULL; > + } else { > + /* orphaned znode?! Root? */ Drop the useless else branch. > --- /dev/null 2003-09-23 21:59:22.000000000 +0400 > +++ linux-2.6.11-vs/fs/reiser4/debug.c 2005-06-03 17:49:38.293184063 +0400 > @@ -0,0 +1,447 @@ > +/* Copyright 2001, 2002, 2003 by Hans Reiser, licensing governed by > + * reiser4/README */ > + > +/* Debugging facilities. */ > + > +/* > + * This file contains generic debugging functions used by reiser4. Roughly > + * following: > + * > + * panicking: reiser4_do_panic(), reiser4_print_prefix(). > + * > + * locking: schedulable(), lock_counters(), print_lock_counters(), > + * no_counters_are_held(), commit_check_locks() > + * > + * {debug,trace,log}_flags: reiser4_are_all_debugged(), > + * reiser4_is_debugged(), get_current_trace_flags(), > + * get_current_log_flags(). > + * > + * kmalloc/kfree leak detection: reiser4_kmalloc(), reiser4_kfree(), > + * reiser4_kfree_in_sb(). Please don't do this! We've had enough trouble trying to get the current subsystem specific allocators out, so do not introduce a new one. If you need memory leak detection, make it generic and submit that. Reiser4, like all other subsystems, should use kmalloc() and friends directly. > --- /dev/null 2003-09-23 21:59:22.000000000 +0400 > +++ linux-2.6.11-vs/fs/reiser4/debug.h 2005-06-03 17:49:38.297184283 +0400 > + > +/* > + * Error code tracing facility. (Idea is borrowed from XFS code.) Could this be turned into a generic facility? > + * > + * Suppose some strange and/or unexpected code is returned from some function > + * (for example, write(2) returns -EEXIST). It is possible to place a > + * breakpoint in the reiser4_write(), but it is too late here. How to find out -- > +#define RETERR(code) \ > +({ \ > + typeof(code) __code; \ > + \ > + __code = (code); \ > + return_err(__code, __FILE__, __LINE__); \ > + __code; \ > +}) > +#define reiser4_internal Please drop the above useless #define. > --- /dev/null 2003-09-23 21:59:22.000000000 +0400 > +++ linux-2.6.11-vs/fs/reiser4/emergency_flush.c 2005-06-03 17:51:59.000905353 +0400 > @@ -0,0 +1,913 @@ > +/* Copyright 2002, 2003 by Hans Reiser, licensing governed by reiser4/README */ > + > +/* This file exists only until VM gets fixed to reserve pages properly, which > + * might or might not be very political. */ Please fix vm instead of working around it in fs. > --- /dev/null 2003-09-23 21:59:22.000000000 +0400 > +++ linux-2.6.11-vs/fs/reiser4/init_super.c 2005-06-03 17:51:27.959201950 +0400 > + > +#define _INIT_PARAM_LIST (struct super_block * s, reiser4_context * ctx, void * data, int silent) > +#define _DONE_PARAM_LIST (struct super_block * s) > + > +#define _INIT_(subsys) static int _init_##subsys _INIT_PARAM_LIST > +#define _DONE_(subsys) static void _done_##subsys _DONE_PARAM_LIST Please remove this macro obfuscation. > + > +_DONE_EMPTY(exit_context) > + > +struct reiser4_subsys { > + int (*init) _INIT_PARAM_LIST; > + void (*done) _DONE_PARAM_LIST; > +}; > + > +#define _SUBSYS(subsys) {.init = &_init_##subsys, .done = &_done_##subsys} > +static struct reiser4_subsys subsys_array[] = { > + _SUBSYS(mount_flags_check), > + _SUBSYS(sinfo), > + _SUBSYS(context), > + _SUBSYS(parse_options), > + _SUBSYS(object_ops), > + _SUBSYS(read_super), > + _SUBSYS(tree0), > + _SUBSYS(txnmgr), > + _SUBSYS(ktxnmgrd_context), > + _SUBSYS(ktxnmgrd), > + _SUBSYS(entd), > + _SUBSYS(formatted_fake), > + _SUBSYS(disk_format), > + _SUBSYS(sb_counters), > + _SUBSYS(d_cursor), > + _SUBSYS(fs_root), > + _SUBSYS(safelink), > + _SUBSYS(exit_context) > +}; The above is overkill and silly. parse_options and read_super, for example, are _not_ a subsystem inits but regular fs ops. Please consider dropping this altogether but at least trim it down to something sane. > --- /dev/null 2003-09-23 21:59:22.000000000 +0400 > +++ linux-2.6.11-vs/fs/reiser4/page_cache.c 2005-06-03 17:51:59.003905518 +0400 > +/* one-time initialization of fake inodes handling functions. */ > +reiser4_internal int > +init_fakes() > +{ > + return 0; > +} Please drop this empty function. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 6:22 ` Pekka Enberg @ 2005-06-23 7:42 ` Hans Reiser 2005-06-23 8:08 ` Pekka J Enberg 2005-06-23 16:15 ` Vladimir Saveliev 2005-06-23 18:37 ` Jeff Mahoney 1 sibling, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-23 7:42 UTC (permalink / raw) To: Pekka Enberg, vs Cc: Andi Kleen, Alexander Lyamin aka FLX, Alexander Zarochentcev, vs, Andrew Morton, linux-kernel, ReiserFS List, Pekka Enberg Pekka Enberg wrote: >Hi Hans, > >On 6/22/05, Hans Reiser <reiser@namesys.com> wrote: > > >>I would in particular love to have you Andi Kleen do a full review of V4 >>if you could be that generous with your time, as I liked much of the >>advice you gave us on V3. >> >> > >Well, I am not Andi Kleen and this is not even in the ballpark of a full >review but here goes... > > thanks kindly for your time, your comments were appreciated >P.S. Would it be possible to have a version without the plugin stuff >submitted (and perhaps file as directory)? > No, I am sorry. It is like asking for ext2 without directories. > It would make much more >sense for the rest of us to review reiser4 without the most controversial >bits and get the bits that people agree on merged. > > Pekka > > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>+++ linux-2.6.11-vs/fs/reiser4/pool.c 2005-06-03 17:49:38.668204642 +0400 >>+/* initialise new pool */ >>+reiser4_internal void >>+reiser4_init_pool(reiser4_pool * pool /* pool to initialise */ , >>+ size_t obj_size /* size of objects in @pool */ , >>+ int num_of_objs /* number of preallocated objects */ , >>+ char *data /* area for preallocated objects */ ) >>+{ >>+ reiser4_pool_header *h; >>+ int i; >>+ >>+ assert("nikita-955", pool != NULL); >> >> > >These assertion codes are meaningless to the rest of us so please drop >them. > > I think you don't appreciate the role of assertions in making code easier to audit and debug. > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>+++ linux-2.6.11-vs/fs/reiser4/type_safe_hash.h 2005-06-03 17:49:38.751209197 +0400 >>@@ -0,0 +1,320 @@ >>+/* Copyright 2001, 2002, 2003 by Hans Reiser, licensing governed by >>+ * reiser4/README */ >>+ >>+/* A hash table class that uses hash chains (singly-linked) and is >>+ parametrized to provide type safety. */ >> >> > >This belongs to include/linux, not reiser4. > > Too much politics are involved in modifying other peoples directories, or I'd agree with you. The first person outside the reiser4 project to use it should move it into a different place. > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>+++ linux-2.6.11-vs/fs/reiser4/type_safe_list.h 2005-06-03 17:49:38.755209417 +0400 >>@@ -0,0 +1,436 @@ >>+/* Copyright 2001, 2002, 2003 by Hans Reiser, licensing governed by reiser4/README */ >>+ >>+#ifndef __REISER4_TYPE_SAFE_LIST_H__ >>+#define __REISER4_TYPE_SAFE_LIST_H__ >>+ >>+#include "debug.h" >>+/* A circular doubly linked list that differs from the previous >>+ <linux/list.h> implementation because it is parametrized to provide >>+ type safety. This data structure is also useful as a queue or stack. >> >> > >This belongs to include/linux, not reiser4. > > Yes, but see above about first person outside reiser4 project should..... > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>+++ linux-2.6.11-vs/fs/reiser4/vfs_ops.c 2005-06-03 17:51:28.110210237 +0400 >>+/* ->get_sb() method of file_system operations. */ >>+static struct super_block * >>+reiser4_get_sb(struct file_system_type *fs_type /* file >>+ * system >>+ * type */ , >>+ int flags /* flags */ , >>+ const char *dev_name /* device name */ , >>+ void *data /* mount options */ ) >> >> > >Please drop the useless parameter comments. > > vs, figure out who added the flags and device name comments and ask them to prepare a patch. If nobody admits to the shameful act,;-) have Edward fix it. > > >>+/* >>+ * Initialization stages for reiser4. >>+ * >>+ * These enumerate various things that have to be done during reiser4 >>+ * startup. Initialization code (init_reiser4()) keeps track of what stage was >>+ * reached, so that proper undo can be done if error occurs during >>+ * initialization. >>+ */ >>+typedef enum { >>+ INIT_NONE, /* nothing is initialized yet */ >>+ INIT_INODECACHE, /* inode cache created */ >>+ INIT_CONTEXT_MGR, /* list of active contexts created */ >>+ INIT_ZNODES, /* znode slab created */ >>+ INIT_PLUGINS, /* plugins initialized */ >>+ INIT_PLUGIN_SET, /* psets initialized */ >>+ INIT_TXN, /* transaction manager initialized */ >>+ INIT_FAKES, /* fake inode initialized */ >>+ INIT_JNODES, /* jnode slab initialized */ >>+ INIT_EFLUSH, /* emergency flush initialized */ >>+ INIT_FQS, /* flush queues initialized */ >>+ INIT_DENTRY_FSDATA, /* dentry_fsdata slab initialized */ >>+ INIT_FILE_FSDATA, /* file_fsdata slab initialized */ >>+ INIT_D_CURSOR, /* d_cursor suport initialized */ >>+ INIT_FS_REGISTERED, /* reiser4 file system type registered */ >>+} reiser4_init_stage; >> >> > >Please use regular gotos instead. This is a silly hack especially since you >don't have release function for all of the stages. > > I'll let vs comment. > > >>+reiser4_internal void reiser4_handle_error(void) >>+{ >>+ struct super_block *sb = reiser4_get_current_sb(); >>+ >>+ if ( !sb ) >>+ return; >>+ reiser4_status_write(REISER4_STATUS_DAMAGED, 0, "Filesystem error occured"); >>+ switch ( get_super_private(sb)->onerror ) { >>+ case 0: >>+ reiser4_panic("foobar-42", "Filesystem error occured\n"); >>+ case 1: >>+ if ( sb->s_flags & MS_RDONLY ) >>+ return; >>+ sb->s_flags |= MS_RDONLY; >>+ break; >>+ case 2: >>+ machine_restart(NULL); >> >> > >Probably not something you should do at fs level. > > Not sure why you say that. vs, given the existence of case 0, why do we need to have case 2? > > >>+/* free this znode */ >>+reiser4_internal void >>+zfree(znode * node /* znode to free */ ) >>+{ >>+ assert("nikita-465", node != NULL); >>+ assert("nikita-2120", znode_page(node) == NULL); >>+ assert("nikita-2301", owners_list_empty(&node->lock.owners)); >>+ assert("nikita-2302", requestors_list_empty(&node->lock.requestors)); >>+ assert("nikita-2663", capture_list_is_clean(ZJNODE(node)) && NODE_LIST(ZJNODE(node)) == NOT_CAPTURED); >>+ assert("nikita-2773", !JF_ISSET(ZJNODE(node), JNODE_EFLUSH)); >>+ assert("nikita-3220", list_empty(&ZJNODE(node)->jnodes)); >>+ assert("nikita-3293", !znode_is_right_connected(node)); >>+ assert("nikita-3294", !znode_is_left_connected(node)); >>+ assert("nikita-3295", node->left == NULL); >>+ assert("nikita-3296", node->right == NULL); >> >> > >Are all these still required? Seems bit too defensive for the kernel. > > Hmm, someday must put you in a room with DARPA guys, and let you get us another round of funding by trying to convince them that our code is too defensive.;-) This is not too defensive. Nikita did well here. The culture of code auditors is very different from most programmers. Like most specialists, they have wisdom to offer those who listen to them. In essence, they teach that every function should have a specification of every possible restriction on allowed input that can be imagined and is correct as a restriction. Code auditors love assertions like this. I look at my understanding of this before DARPA, and I wince. DARPA patiently taught me much in this area as I listened to security talks at our meetings, and I thank them for it. Large scale projects like reiser4 are crippled by debugging costs. Anything that reduces debugging time is worth so much..... > > >>+ >>+ >>+ /* not yet phash_jnode_destroy(ZJNODE(node)); */ >>+ >>+ /* poison memory. */ >>+ ON_DEBUG(memset(node, 0xde, sizeof *node)); >> >> > >Poisoning belongs to slab, not fs. > > vs, please comment. > > >>+/* allocate fresh znode */ >>+reiser4_internal znode * >>+zalloc(int gfp_flag /* allocation flag */ ) >>+{ >>+ znode *node; >>+ >>+ node = kmem_cache_alloc(znode_slab, gfp_flag); >>+ return node; >>+} >> >> > >Please drop this useless wrapper. > > Thanks. vs, I think he is right here.... I see zalloc used in only two places..... Or is the desire to ease future porting work? > > >>+reiser4_internal void >>+znode_remove(znode * node /* znode to remove */ , reiser4_tree * tree) >>+{ >>+ assert("nikita-2108", node != NULL); >>+ assert("nikita-470", node->c_count == 0); >>+ assert("zam-879", rw_tree_is_write_locked(tree)); >>+ >>+ /* remove reference to this znode from cbk cache */ >>+ cbk_cache_invalidate(node, tree); >>+ >>+ /* update c_count of parent */ >>+ if (znode_parent(node) != NULL) { >>+ assert("nikita-472", znode_parent(node)->c_count > 0); >>+ /* father, onto your hands I forward my spirit... */ >>+ znode_parent(node)->c_count --; >>+ node->in_parent.node = NULL; >>+ } else { >>+ /* orphaned znode?! Root? */ >> >> > >Drop the useless else branch. > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>+++ linux-2.6.11-vs/fs/reiser4/debug.c 2005-06-03 17:49:38.293184063 +0400 >>@@ -0,0 +1,447 @@ >>+/* Copyright 2001, 2002, 2003 by Hans Reiser, licensing governed by >>+ * reiser4/README */ >>+ >>+/* Debugging facilities. */ >>+ >>+/* >>+ * This file contains generic debugging functions used by reiser4. Roughly >>+ * following: >>+ * >>+ * panicking: reiser4_do_panic(), reiser4_print_prefix(). >>+ * >>+ * locking: schedulable(), lock_counters(), print_lock_counters(), >>+ * no_counters_are_held(), commit_check_locks() >>+ * >>+ * {debug,trace,log}_flags: reiser4_are_all_debugged(), >>+ * reiser4_is_debugged(), get_current_trace_flags(), >>+ * get_current_log_flags(). >>+ * >>+ * kmalloc/kfree leak detection: reiser4_kmalloc(), reiser4_kfree(), >>+ * reiser4_kfree_in_sb(). >> >> > >Please don't do this! We've had enough trouble trying to get the >current subsystem specific allocators out, so do not introduce a new one. If >you need memory leak detection, make it generic and submit that. Reiser4, like >all other subsystems, should use kmalloc() and friends directly. > > I will let vs comment. > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>+++ linux-2.6.11-vs/fs/reiser4/debug.h 2005-06-03 17:49:38.297184283 +0400 >>+ >>+/* >>+ * Error code tracing facility. (Idea is borrowed from XFS code.) >> >> > >Could this be turned into a generic facility? > > Probably it already is one. Getting it used as such sounds like a lot of political work though. Maybe the first person outside the reiser4 project to want to use it should move the code into a different location. > > >>+ * >>+ * Suppose some strange and/or unexpected code is returned from some function >>+ * (for example, write(2) returns -EEXIST). It is possible to place a >>+ * breakpoint in the reiser4_write(), but it is too late here. How to find out >> >> >-- > > >>+#define RETERR(code) \ >>+({ \ >>+ typeof(code) __code; \ >>+ \ >>+ __code = (code); \ >>+ return_err(__code, __FILE__, __LINE__); \ >>+ __code; \ >>+}) >> >> > > > >>+#define reiser4_internal >> >> > >Please drop the above useless #define. > > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>+++ linux-2.6.11-vs/fs/reiser4/emergency_flush.c 2005-06-03 17:51:59.000905353 +0400 >>@@ -0,0 +1,913 @@ >>+/* Copyright 2002, 2003 by Hans Reiser, licensing governed by reiser4/README */ >>+ >>+/* This file exists only until VM gets fixed to reserve pages properly, which >>+ * might or might not be very political. */ >> >> > >Please fix vm instead of working around it in fs. > > actually I want to see emergency flush die very very much. As for fixing vm, easier said than done, politically. > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>+++ linux-2.6.11-vs/fs/reiser4/init_super.c 2005-06-03 17:51:27.959201950 +0400 >>+ >>+#define _INIT_PARAM_LIST (struct super_block * s, reiser4_context * ctx, void * data, int silent) >>+#define _DONE_PARAM_LIST (struct super_block * s) >>+ >>+#define _INIT_(subsys) static int _init_##subsys _INIT_PARAM_LIST >>+#define _DONE_(subsys) static void _done_##subsys _DONE_PARAM_LIST >> >> > >Please remove this macro obfuscation. > > vs, I think he is right, do you? > > >>+ >>+_DONE_EMPTY(exit_context) >>+ >>+struct reiser4_subsys { >>+ int (*init) _INIT_PARAM_LIST; >>+ void (*done) _DONE_PARAM_LIST; >>+}; >>+ >>+#define _SUBSYS(subsys) {.init = &_init_##subsys, .done = &_done_##subsys} >>+static struct reiser4_subsys subsys_array[] = { >>+ _SUBSYS(mount_flags_check), >>+ _SUBSYS(sinfo), >>+ _SUBSYS(context), >>+ _SUBSYS(parse_options), >>+ _SUBSYS(object_ops), >>+ _SUBSYS(read_super), >>+ _SUBSYS(tree0), >>+ _SUBSYS(txnmgr), >>+ _SUBSYS(ktxnmgrd_context), >>+ _SUBSYS(ktxnmgrd), >>+ _SUBSYS(entd), >>+ _SUBSYS(formatted_fake), >>+ _SUBSYS(disk_format), >>+ _SUBSYS(sb_counters), >>+ _SUBSYS(d_cursor), >>+ _SUBSYS(fs_root), >>+ _SUBSYS(safelink), >>+ _SUBSYS(exit_context) >>+}; >> >> > >The above is overkill and silly. parse_options and read_super, for >example, are _not_ a subsystem inits but regular fs ops. Please consider >dropping this altogether but at least trim it down to something sane. > > vs please comment. > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>+++ linux-2.6.11-vs/fs/reiser4/page_cache.c 2005-06-03 17:51:59.003905518 +0400 >>+/* one-time initialization of fake inodes handling functions. */ >>+reiser4_internal int >>+init_fakes() >>+{ >>+ return 0; >>+} >> >> > >Please drop this empty function. > > > > sure. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 7:42 ` Hans Reiser @ 2005-06-23 8:08 ` Pekka J Enberg 2005-06-23 13:10 ` Hans Reiser 2005-06-23 16:15 ` Vladimir Saveliev 1 sibling, 1 reply; 620+ messages in thread From: Pekka J Enberg @ 2005-06-23 8:08 UTC (permalink / raw) To: Hans Reiser Cc: Pekka Enberg, vs, Andi Kleen, Alexander Lyamin aka FLX, Alexander Zarochentcev, Andrew Morton, linux-kernel, ReiserFS List Hi Hans, On Thu, 2005-06-23 at 00:42 -0700, Hans Reiser wrote: > > These assertion codes are meaningless to the rest of us so please drop > > them. > > I think you don't appreciate the role of assertions in making code > easier to audit and debug. I did not say you should drop the assertions. I referred to the "nikita-955" part which is redundant and pointless. Using __FILE__:__LINE__ (or BUG_ON even) will give you enough information to identify where the error occured. On Thu, 2005-06-23 at 00:42 -0700, Hans Reiser wrote: > > This belongs to include/linux, not reiser4. > > Too much politics are involved in modifying other peoples directories, > or I'd agree with you. The first person outside the reiser4 project to > use it should move it into a different place. What politics? Please do hide generic code in your fs. How should someone outside reiser4 know you have type-safe hash tables and linked lists in there? Why should they care? It is obvious that you did not think <linux/list.h> and <linux/jhash.h> were sufficient so please fix those instead and use them. > >>+reiser4_internal void reiser4_handle_error(void) > >>+{ > >>+ struct super_block *sb = reiser4_get_current_sb(); > >>+ > >>+ if ( !sb ) > >>+ return; > >>+ reiser4_status_write(REISER4_STATUS_DAMAGED, 0, "Filesystem error occured"); > >>+ switch ( get_super_private(sb)->onerror ) { > >>+ case 0: > >>+ reiser4_panic("foobar-42", "Filesystem error occured\n"); > >>+ case 1: > >>+ if ( sb->s_flags & MS_RDONLY ) > >>+ return; > >>+ sb->s_flags |= MS_RDONLY; > >>+ break; > >>+ case 2: > >>+ machine_restart(NULL); > >> > >> > > > > Probably not something you should do at fs level. > > Not sure why you say that. Because Reiser4 hitting an error condition and restarting the machine silently is silly. Just do panic() there. > This is not too defensive. Nikita did well here. The culture of code > auditors is very different from most programmers. Like most > specialists, they have wisdom to offer those who listen to them. In > essence, they teach that every function should have a specification of > every possible restriction on allowed input that can be imagined and is > correct as a restriction. Code auditors love assertions like this. I > look at my understanding of this before DARPA, and I wince. DARPA > patiently taught me much in this area as I listened to security talks at > our meetings, and I thank them for it. Hans, I am aware of techniques such as design-by-contract but it is not the kernel coding style. That's because it makes the code harder to read and refactor. > Large scale projects like reiser4 are crippled by debugging costs. > Anything that reduces debugging time is worth so much..... Then start writing automated unit tests. > >>+/* allocate fresh znode */ > >>+reiser4_internal znode * > >>+zalloc(int gfp_flag /* allocation flag */ ) > >>+{ > >>+ znode *node; > >>+ > >>+ node = kmem_cache_alloc(znode_slab, gfp_flag); > >>+ return node; > >>+} > >> > >> > > > >Please drop this useless wrapper. > > > > > Thanks. vs, I think he is right here.... I see zalloc used in only two > places..... Or is the desire to ease future porting work? Then add it later. It is a useless wrapper now. > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 > >>+++ linux-2.6.11-vs/fs/reiser4/debug.h 2005-06-03 17:49:38.297184283 +0400 > >>+ > >>+/* > >>+ * Error code tracing facility. (Idea is borrowed from XFS code.) > >> > >> > > > >Could this be turned into a generic facility? > > > Probably it already is one. Getting it used as such sounds like a lot > of political work though. Maybe the first person outside the reiser4 > project to want to use it should move the code into a different location. What political work? Just whoop up a patch to introduce it as a generic facility and let others review it. There are plenty of janitors that are willing to convert the existing code. Please note that you're now duplicating code from XFS (even if it is just an idea you borrowed). This stuff is fairly simple: if the technique you're using is good, it should probably be generic; if it isn't, you shouldn't be using it either. Please don't let the pissing contest between you and hch create more work for the rest of us. > actually I want to see emergency flush die very very much. As for > fixing vm, easier said than done, politically. As you might have noticed, I don't care for politics. Just post the patch to fix the vm for review. Pekka ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 8:08 ` Pekka J Enberg @ 2005-06-23 13:10 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-23 13:10 UTC (permalink / raw) To: Pekka J Enberg, vs Cc: Pekka Enberg, Andi Kleen, Alexander Lyamin aka FLX, Alexander Zarochentcev, Andrew Morton, linux-kernel, ReiserFS List Pekka J Enberg wrote: > Hi Hans, > On Thu, 2005-06-23 at 00:42 -0700, Hans Reiser wrote: > >> > These assertion codes are meaningless to the rest of us so please drop >> > them. >> I think you don't appreciate the role of assertions in making code >> easier to audit and debug. > > > I did not say you should drop the assertions. I referred to the > "nikita-955" part which is redundant and pointless. Using > __FILE__:__LINE__ (or BUG_ON even) will give you enough information to > identify where the error occured. but then it does not tell me who I assign the bug to. > > Because Reiser4 hitting an error condition and restarting the machine > silently is silly. Just do panic() there. Well, it seems we agreed and did not realize it. Oh well. The silent restart seems like a silly option to have available. If someone can think of a case where it is useful, let me know, otherwise vs please remove it. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 7:42 ` Hans Reiser 2005-06-23 8:08 ` Pekka J Enberg @ 2005-06-23 16:15 ` Vladimir Saveliev 2005-06-23 16:23 ` Olivier Galibert 2005-06-23 17:17 ` Hans Reiser 1 sibling, 2 replies; 620+ messages in thread From: Vladimir Saveliev @ 2005-06-23 16:15 UTC (permalink / raw) To: Pekka Enberg; +Cc: Alexander Zarochentcev, linux-kernel, ReiserFS List Hello On Thu, 2005-06-23 at 11:42, Hans Reiser wrote: > Pekka Enberg wrote: > > > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 > >>+++ linux-2.6.11-vs/fs/reiser4/pool.c 2005-06-03 17:49:38.668204642 +0400 > >>+/* initialise new pool */ > >>+reiser4_internal void > >>+reiser4_init_pool(reiser4_pool * pool /* pool to initialise */ , > >>+ size_t obj_size /* size of objects in @pool */ , > >>+ int num_of_objs /* number of preallocated objects */ , > >>+ char *data /* area for preallocated objects */ ) > >>+{ > >>+ reiser4_pool_header *h; > >>+ int i; > >>+ > >>+ assert("nikita-955", pool != NULL); > >> > >> > > > >These assertion codes are meaningless to the rest of us so please drop > >them. > > Pekka, am I correct that you object aginst assertion ids like "joe-700"? > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 > >>+++ linux-2.6.11-vs/fs/reiser4/type_safe_hash.h 2005-06-03 17:49:38.751209197 +0400 > >>@@ -0,0 +1,320 @@ > >>+/* Copyright 2001, 2002, 2003 by Hans Reiser, licensing governed by > >>+ * reiser4/README */ > >>+ > >>+/* A hash table class that uses hash chains (singly-linked) and is > >>+ parametrized to provide type safety. */ > > > >This belongs to include/linux, not reiser4. ok. > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 > >>+++ linux-2.6.11-vs/fs/reiser4/type_safe_list.h 2005-06-03 17:49:38.755209417 +0400 > >>@@ -0,0 +1,436 @@ > >>+/* Copyright 2001, 2002, 2003 by Hans Reiser, licensing governed by reiser4/README */ > >>+ > >>+#ifndef __REISER4_TYPE_SAFE_LIST_H__ > >>+#define __REISER4_TYPE_SAFE_LIST_H__ > >>+ > >>+#include "debug.h" > >>+/* A circular doubly linked list that differs from the previous > >>+ <linux/list.h> implementation because it is parametrized to provide > >>+ type safety. This data structure is also useful as a queue or stack. > > > >This belongs to include/linux, not reiser4. > > ok > >>+/* > >>+ * Initialization stages for reiser4. > >>+ * > >>+ * These enumerate various things that have to be done during reiser4 > >>+ * startup. Initialization code (init_reiser4()) keeps track of what stage was > >>+ * reached, so that proper undo can be done if error occurs during > >>+ * initialization. > >>+ */ > >>+typedef enum { > >>+ INIT_NONE, /* nothing is initialized yet */ > >>+ INIT_INODECACHE, /* inode cache created */ > >>+ INIT_CONTEXT_MGR, /* list of active contexts created */ > >>+ INIT_ZNODES, /* znode slab created */ > >>+ INIT_PLUGINS, /* plugins initialized */ > >>+ INIT_PLUGIN_SET, /* psets initialized */ > >>+ INIT_TXN, /* transaction manager initialized */ > >>+ INIT_FAKES, /* fake inode initialized */ > >>+ INIT_JNODES, /* jnode slab initialized */ > >>+ INIT_EFLUSH, /* emergency flush initialized */ > >>+ INIT_FQS, /* flush queues initialized */ > >>+ INIT_DENTRY_FSDATA, /* dentry_fsdata slab initialized */ > >>+ INIT_FILE_FSDATA, /* file_fsdata slab initialized */ > >>+ INIT_D_CURSOR, /* d_cursor suport initialized */ > >>+ INIT_FS_REGISTERED, /* reiser4 file system type registered */ > >>+} reiser4_init_stage; > >> > >> > > > >Please use regular gotos instead. This is a silly hack especially since you > >don't have release function for all of the stages. > > > > > I'll let vs comment. > IMHO, it is convinient. But lets change it as requested. > > > > > >>+reiser4_internal void reiser4_handle_error(void) > >>+{ > >>+ struct super_block *sb = reiser4_get_current_sb(); > >>+ > >>+ if ( !sb ) > >>+ return; > >>+ reiser4_status_write(REISER4_STATUS_DAMAGED, 0, "Filesystem error occured"); > >>+ switch ( get_super_private(sb)->onerror ) { > >>+ case 0: > >>+ reiser4_panic("foobar-42", "Filesystem error occured\n"); > >>+ case 1: > >>+ if ( sb->s_flags & MS_RDONLY ) > >>+ return; > >>+ sb->s_flags |= MS_RDONLY; > >>+ break; > >>+ case 2: > >>+ machine_restart(NULL); > >> > >> > > > >Probably not something you should do at fs level. > > ok > >>+ > >>+ /* not yet phash_jnode_destroy(ZJNODE(node)); */ > >>+ > >>+ /* poison memory. */ > >>+ ON_DEBUG(memset(node, 0xde, sizeof *node)); > >> > >> > > > >Poisoning belongs to slab, not fs. > > ok > > > > > >>+/* allocate fresh znode */ > >>+reiser4_internal znode * > >>+zalloc(int gfp_flag /* allocation flag */ ) > >>+{ > >>+ znode *node; > >>+ > >>+ node = kmem_cache_alloc(znode_slab, gfp_flag); > >>+ return node; > >>+} > >> > >> > > > >Please drop this useless wrapper. > > ok > > > > > >>+reiser4_internal void > >>+znode_remove(znode * node /* znode to remove */ , reiser4_tree * tree) > >>+{ > >>+ assert("nikita-2108", node != NULL); > >>+ assert("nikita-470", node->c_count == 0); > >>+ assert("zam-879", rw_tree_is_write_locked(tree)); > >>+ > >>+ /* remove reference to this znode from cbk cache */ > >>+ cbk_cache_invalidate(node, tree); > >>+ > >>+ /* update c_count of parent */ > >>+ if (znode_parent(node) != NULL) { > >>+ assert("nikita-472", znode_parent(node)->c_count > 0); > >>+ /* father, onto your hands I forward my spirit... */ > >>+ znode_parent(node)->c_count --; > >>+ node->in_parent.node = NULL; > >>+ } else { > >>+ /* orphaned znode?! Root? */ > >> > >> > > > >Drop the useless else branch. > > > > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 > >>+++ linux-2.6.11-vs/fs/reiser4/debug.c 2005-06-03 17:49:38.293184063 +0400 > >>@@ -0,0 +1,447 @@ > >>+/* Copyright 2001, 2002, 2003 by Hans Reiser, licensing governed by > >>+ * reiser4/README */ > >>+ > >>+/* Debugging facilities. */ > >>+ > >>+/* > >>+ * This file contains generic debugging functions used by reiser4. Roughly > >>+ * following: > >>+ * > >>+ * panicking: reiser4_do_panic(), reiser4_print_prefix(). > >>+ * > >>+ * locking: schedulable(), lock_counters(), print_lock_counters(), > >>+ * no_counters_are_held(), commit_check_locks() > >>+ * > >>+ * {debug,trace,log}_flags: reiser4_are_all_debugged(), > >>+ * reiser4_is_debugged(), get_current_trace_flags(), > >>+ * get_current_log_flags(). > >>+ * > >>+ * kmalloc/kfree leak detection: reiser4_kmalloc(), reiser4_kfree(), > >>+ * reiser4_kfree_in_sb(). > >> > >> > > > >Please don't do this! We've had enough trouble trying to get the > >current subsystem specific allocators out, so do not introduce a new one. If > >you need memory leak detection, make it generic and submit that. Reiser4, like > >all other subsystems, should use kmalloc() and friends directly. > > > > > I will let vs comment. > I agree with Pekka. > > > > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 > >>+++ linux-2.6.11-vs/fs/reiser4/debug.h 2005-06-03 17:49:38.297184283 +0400 > >>+ > >>+/* > >>+ * Error code tracing facility. (Idea is borrowed from XFS code.) > >> > >> > > > >Could this be turned into a generic facility? > > I do not think that it will ever become accepted. To get use of that task_t would have to be extended with fields to store error code, file name and line in it, and several return addresses. Moreover lines like return -ENOENT; would have to be replaced with: return RETERR(-ENOENT); This is debugging feature, we should probably move it to our internal debugging patch. > > > > > > > >>+#define reiser4_internal > >> > >> > > > >Please drop the above useless #define. > > ok > > > >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 > >>+++ linux-2.6.11-vs/fs/reiser4/init_super.c 2005-06-03 17:51:27.959201950 +0400 > >>+ > >>+#define _INIT_PARAM_LIST (struct super_block * s, reiser4_context * ctx, void * data, int silent) > >>+#define _DONE_PARAM_LIST (struct super_block * s) > >>+ > >>+#define _INIT_(subsys) static int _init_##subsys _INIT_PARAM_LIST > >>+#define _DONE_(subsys) static void _done_##subsys _DONE_PARAM_LIST > >> > >> > > > >Please remove this macro obfuscation. > > > > > vs, I think he is right, do you? > > > > > > >>+ > >>+_DONE_EMPTY(exit_context) > >>+ > >>+struct reiser4_subsys { > >>+ int (*init) _INIT_PARAM_LIST; > >>+ void (*done) _DONE_PARAM_LIST; > >>+}; > >>+ > >>+#define _SUBSYS(subsys) {.init = &_init_##subsys, .done = &_done_##subsys} > >>+static struct reiser4_subsys subsys_array[] = { > >>+ _SUBSYS(mount_flags_check), > >>+ _SUBSYS(sinfo), > >>+ _SUBSYS(context), > >>+ _SUBSYS(parse_options), > >>+ _SUBSYS(object_ops), > >>+ _SUBSYS(read_super), > >>+ _SUBSYS(tree0), > >>+ _SUBSYS(txnmgr), > >>+ _SUBSYS(ktxnmgrd_context), > >>+ _SUBSYS(ktxnmgrd), > >>+ _SUBSYS(entd), > >>+ _SUBSYS(formatted_fake), > >>+ _SUBSYS(disk_format), > >>+ _SUBSYS(sb_counters), > >>+ _SUBSYS(d_cursor), > >>+ _SUBSYS(fs_root), > >>+ _SUBSYS(safelink), > >>+ _SUBSYS(exit_context) > >>+}; > >> > >> > > > >The above is overkill and silly. parse_options and read_super, for > >example, are _not_ a subsystem inits but regular fs ops. Please consider > >dropping this altogether but at least trim it down to something sane. > > Pekka, would you prefer something like: reiser4_fill_super() { if (init_a() == 0) { if (init_b() == 0) { if (init_c() == 0) { if (init_last() == 0) return 0; else { done_c(); done_b(); done_a(); } } else { done_b(); done_a(); } } else { done_a(); } } } With these macros we have reiser4_fill_super to look like the below, and it remains unchanged when something new is added for initilizing on filesystem mounting. reiser4_fill_super() { for (i = 0; i < REISER4_NR_SUBSYS; i++) { ret = subsys_array[i].init(s, &ctx, data, silent); if (ret) { done_super(s, i - 1); return ret; } } } ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 16:15 ` Vladimir Saveliev @ 2005-06-23 16:23 ` Olivier Galibert 2005-06-23 19:56 ` Ross Biro 2005-06-23 17:17 ` Hans Reiser 1 sibling, 1 reply; 620+ messages in thread From: Olivier Galibert @ 2005-06-23 16:23 UTC (permalink / raw) To: Vladimir Saveliev Cc: Pekka Enberg, Alexander Zarochentcev, linux-kernel, ReiserFS List On Thu, Jun 23, 2005 at 08:15:03PM +0400, Vladimir Saveliev wrote: > Pekka, would you prefer something like: > > reiser4_fill_super() > { > if (init_a() == 0) { > if (init_b() == 0) { > if (init_c() == 0) { > if (init_last() == 0) > return 0; > else { > done_c(); > done_b(); > done_a(); > } > } else { > done_b(); > done_a(); > } > } else { > done_a(); > } > } > } No, I think he means the traditional: reiser4_fill_super() { if (init_a()) goto failed_a; if (init_b()) goto failed_b; if (init_c()) goto failed_c; if (init_last()) goto failed_last; return 0; failed_last: done_c(); failed_c: done_b(); failed_b: done_a(); failed_a: return 1; } ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 16:23 ` Olivier Galibert @ 2005-06-23 19:56 ` Ross Biro 0 siblings, 0 replies; 620+ messages in thread From: Ross Biro @ 2005-06-23 19:56 UTC (permalink / raw) To: Olivier Galibert, Vladimir Saveliev, Pekka Enberg, Alexander Zarochentcev, linux-kernel, ReiserFS List On 6/23/05, Olivier Galibert <galibert@pobox.com> wrote: > No, I think he means the traditional: > > reiser4_fill_super() > { > if (init_a()) > goto failed_a; . . . IMO this works very well when the initialization always completes or fails totally in a single routine. When your init routine can leave something partially inited, then putting all of the cleanup code in a single function and using the enums eliminates duplicate code and makes things easier to read. (it's a state machine like many device drivers and network stacks). Also, perhaps a compromise on the asserts at the beggining of functions. If they are moved into a header file, say resier4_contracts.h and replaced with a single macro, you get most of the benefits and elminate most of the clutter. If properly done, there may even be some advantages gained by auto generating the conttact.h file(s) from some sort of formal spec or design doc. Ross ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 16:15 ` Vladimir Saveliev 2005-06-23 16:23 ` Olivier Galibert @ 2005-06-23 17:17 ` Hans Reiser 2005-06-23 21:18 ` Nikita Danilov 1 sibling, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-23 17:17 UTC (permalink / raw) To: Vladimir Saveliev Cc: Pekka Enberg, Alexander Zarochentcev, linux-kernel, ReiserFS List Vladimir Saveliev wrote: > > >>>>+/* >>>>+ * Initialization stages for reiser4. >>>>+ * >>>>+ * These enumerate various things that have to be done during reiser4 >>>>+ * startup. Initialization code (init_reiser4()) keeps track of what stage was >>>>+ * reached, so that proper undo can be done if error occurs during >>>>+ * initialization. >>>>+ */ >>>>+typedef enum { >>>>+ INIT_NONE, /* nothing is initialized yet */ >>>>+ INIT_INODECACHE, /* inode cache created */ >>>>+ INIT_CONTEXT_MGR, /* list of active contexts created */ >>>>+ INIT_ZNODES, /* znode slab created */ >>>>+ INIT_PLUGINS, /* plugins initialized */ >>>>+ INIT_PLUGIN_SET, /* psets initialized */ >>>>+ INIT_TXN, /* transaction manager initialized */ >>>>+ INIT_FAKES, /* fake inode initialized */ >>>>+ INIT_JNODES, /* jnode slab initialized */ >>>>+ INIT_EFLUSH, /* emergency flush initialized */ >>>>+ INIT_FQS, /* flush queues initialized */ >>>>+ INIT_DENTRY_FSDATA, /* dentry_fsdata slab initialized */ >>>>+ INIT_FILE_FSDATA, /* file_fsdata slab initialized */ >>>>+ INIT_D_CURSOR, /* d_cursor suport initialized */ >>>>+ INIT_FS_REGISTERED, /* reiser4 file system type registered */ >>>>+} reiser4_init_stage; >>>> >>>> >>>> >>>> >>>Please use regular gotos instead. This is a silly hack especially since you >>>don't have release function for all of the stages. >>> >>> >>> >>> >>I'll let vs comment. >> >> >> > >IMHO, it is convinient. But lets change it as requested. > > No, if you like it, say so and it stays. >>>>+ * >>>>+ * kmalloc/kfree leak detection: reiser4_kmalloc(), reiser4_kfree(), >>>>+ * reiser4_kfree_in_sb(). >>>> >>>> >>>> >>>> >>>Please don't do this! We've had enough trouble trying to get the >>>current subsystem specific allocators out, so do not introduce a new one. If >>>you need memory leak detection, make it generic and submit that. Reiser4, like >>>all other subsystems, should use kmalloc() and friends directly. >>> >>> >>> >>> >>I will let vs comment. >> >> >> >I agree with Pekka. > > Ok, can you make it generic easily? > > >>> >>> >>> >>> >>>>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>>>+++ linux-2.6.11-vs/fs/reiser4/debug.h 2005-06-03 17:49:38.297184283 +0400 >>>>+ >>>>+/* >>>>+ * Error code tracing facility. (Idea is borrowed from XFS code.) >>>> >>>> >>>> >>>> >>>Could this be turned into a generic facility? >>> >>> >>> > >I do not think that it will ever become accepted. >To get use of that task_t would have to be extended with fields to store >error code, file name and line in it, and several return addresses. >Moreover lines like >return -ENOENT; >would have to be replaced with: >return RETERR(-ENOENT); > >This is debugging feature, we should probably move it to our internal >debugging patch. > > > >>> >>> >>> >>> >>>>+#define reiser4_internal >>>> >>>> >>>> >>>> >>>Please drop the above useless #define. >>> >>> >>> > >ok > > > >>>>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>>>+++ linux-2.6.11-vs/fs/reiser4/init_super.c 2005-06-03 17:51:27.959201950 +0400 >>>>+ >>>>+#define _INIT_PARAM_LIST (struct super_block * s, reiser4_context * ctx, void * data, int silent) >>>>+#define _DONE_PARAM_LIST (struct super_block * s) >>>>+ >>>>+#define _INIT_(subsys) static int _init_##subsys _INIT_PARAM_LIST >>>>+#define _DONE_(subsys) static void _done_##subsys _DONE_PARAM_LIST >>>> >>>> >>>> >>>> >>>Please remove this macro obfuscation. >>> >>> >>> >>> >>vs, I think he is right, do you? >> >> >> >>> >>> >>> >>> >>>>+ >>>>+_DONE_EMPTY(exit_context) >>>>+ >>>>+struct reiser4_subsys { >>>>+ int (*init) _INIT_PARAM_LIST; >>>>+ void (*done) _DONE_PARAM_LIST; >>>>+}; >>>>+ >>>>+#define _SUBSYS(subsys) {.init = &_init_##subsys, .done = &_done_##subsys} >>>>+static struct reiser4_subsys subsys_array[] = { >>>>+ _SUBSYS(mount_flags_check), >>>>+ _SUBSYS(sinfo), >>>>+ _SUBSYS(context), >>>>+ _SUBSYS(parse_options), >>>>+ _SUBSYS(object_ops), >>>>+ _SUBSYS(read_super), >>>>+ _SUBSYS(tree0), >>>>+ _SUBSYS(txnmgr), >>>>+ _SUBSYS(ktxnmgrd_context), >>>>+ _SUBSYS(ktxnmgrd), >>>>+ _SUBSYS(entd), >>>>+ _SUBSYS(formatted_fake), >>>>+ _SUBSYS(disk_format), >>>>+ _SUBSYS(sb_counters), >>>>+ _SUBSYS(d_cursor), >>>>+ _SUBSYS(fs_root), >>>>+ _SUBSYS(safelink), >>>>+ _SUBSYS(exit_context) >>>>+}; >>>> >>>> >>>> >>>> >>>The above is overkill and silly. parse_options and read_super, for >>>example, are _not_ a subsystem inits but regular fs ops. Please consider >>>dropping this altogether but at least trim it down to something sane. >>> >>> >>> > >Pekka, would you prefer something like: > >reiser4_fill_super() >{ > if (init_a() == 0) { > if (init_b() == 0) { > if (init_c() == 0) { > if (init_last() == 0) > return 0; > else { > done_c(); > done_b(); > done_a(); > } > } else { > done_b(); > done_a(); > } > } else { > done_a(); > } > } >} > > I think the above is easier to read than the below. Macros can obscure sometimes, and one of our weaknesses is a tendency to use macros in such a way that it frustrates meta-. use in emacs. Nikita did however mention that there was something that could understand macros when constructing tags files, but I forgot what that was. >With these macros we have reiser4_fill_super to look like the below, and >it remains unchanged when something new is added for initilizing on >filesystem mounting. > >reiser4_fill_super() >{ > for (i = 0; i < REISER4_NR_SUBSYS; i++) { > ret = subsys_array[i].init(s, &ctx, data, silent); > if (ret) { > done_super(s, i - 1); > return ret; > } > } >} > > > > > > ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 17:17 ` Hans Reiser @ 2005-06-23 21:18 ` Nikita Danilov 0 siblings, 0 replies; 620+ messages in thread From: Nikita Danilov @ 2005-06-23 21:18 UTC (permalink / raw) To: Hans Reiser Cc: Pekka Enberg, Alexander Zarochentcev, linux-kernel, ReiserFS List Hans Reiser writes: [...] > I think the above is easier to read than the below. Macros can obscure > sometimes, and one of our weaknesses is a tendency to use macros in such > a way that it frustrates meta-. use in emacs. Nikita did however > mention that there was something that could understand macros when > constructing tags files, but I forgot what that was. xref.el (http://xref-tech.com/xrefactory/main.html). With it one can type current->[TAB] and it shows fields of struct task_struct in the emacs completion buffer. Nikita. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 6:22 ` Pekka Enberg 2005-06-23 7:42 ` Hans Reiser @ 2005-06-23 18:37 ` Jeff Mahoney 2005-06-23 18:43 ` Andrew Morton ` (2 more replies) 1 sibling, 3 replies; 620+ messages in thread From: Jeff Mahoney @ 2005-06-23 18:37 UTC (permalink / raw) To: Pekka Enberg Cc: Hans Reiser, Andi Kleen, Alexander Lyamin aka FLX, Alexander Zarochentcev, vs, Andrew Morton, linux-kernel, ReiserFS List, Pekka Enberg Pekka Enberg wrote: >>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>+++ linux-2.6.11-vs/fs/reiser4/pool.c 2005-06-03 17:49:38.668204642 +0400 >>+/* initialise new pool */ >>+reiser4_internal void >>+reiser4_init_pool(reiser4_pool * pool /* pool to initialise */ , >>+ size_t obj_size /* size of objects in @pool */ , >>+ int num_of_objs /* number of preallocated objects */ , >>+ char *data /* area for preallocated objects */ ) >>+{ >>+ reiser4_pool_header *h; >>+ int i; >>+ >>+ assert("nikita-955", pool != NULL); > > These assertion codes are meaningless to the rest of us so please drop > them. As someone who spends time debugging reiser3 issues, I've grown accustomed to the named assertions. They make discussing a particular assertion much more natural in conversation than file:line. It also makes difficult to reproduce assertions more trackable over time. The assertion number never changes, but the line number can with even the most trivial of patches. That said, one of my gripes with the named assertions in reiser3 (and reiser4 now) is that the development staff changes over time. There are many named assertions in reiser3 that refer to developers no longer employed by Hans. The quoted one is a perfect example. Hans, would it be acceptable to you to keep only numbered assertions and keep a code responsbility list internal to namesys? It would serve a dual purpose of keeping the idea of named assertions, but also remove a huge number of strings that bloat the implementation. -Jeff -- Jeff Mahoney SuSE Labs ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 18:37 ` Jeff Mahoney @ 2005-06-23 18:43 ` Andrew Morton 2005-06-23 19:29 ` Jeff Mahoney 2005-06-23 19:32 ` Jens Axboe 2005-06-23 19:24 ` Hans Reiser 2005-06-24 1:13 ` Hubert Chan 2 siblings, 2 replies; 620+ messages in thread From: Andrew Morton @ 2005-06-23 18:43 UTC (permalink / raw) To: Jeff Mahoney Cc: penberg, reiser, ak, flx, zam, vs, linux-kernel, reiserfs-list, penberg Jeff Mahoney <jeffm@suse.de> wrote: > > >>+ assert("nikita-955", pool != NULL); > > > > These assertion codes are meaningless to the rest of us so please drop > > them. > > As someone who spends time debugging reiser3 issues, I've grown > accustomed to the named assertions. They make discussing a particular > assertion much more natural in conversation than file:line. __FUNCTION__? ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 18:43 ` Andrew Morton @ 2005-06-23 19:29 ` Jeff Mahoney 2005-06-23 19:45 ` Hans Reiser 2005-06-23 19:32 ` Jens Axboe 1 sibling, 1 reply; 620+ messages in thread From: Jeff Mahoney @ 2005-06-23 19:29 UTC (permalink / raw) To: Andrew Morton Cc: penberg, reiser, ak, flx, zam, vs, linux-kernel, reiserfs-list, penberg Andrew Morton wrote: > Jeff Mahoney <jeffm@suse.de> wrote: >>>>+ assert("nikita-955", pool != NULL); >> > >> > These assertion codes are meaningless to the rest of us so please drop >> > them. >> >> As someone who spends time debugging reiser3 issues, I've grown >> accustomed to the named assertions. They make discussing a particular >> assertion much more natural in conversation than file:line. > > __FUNCTION__? __FUNCTION__ gives additional context, sure, but it doesn't address one of the main advantages of numbered assertions: the ability to track the same assertion across different versions without having to verify that it is indeed the same assertion on every one. The line number can still change very easily, and if there are several assertions in a row, it's not immediately obvious (at a glance) that it is the same assertion. Reiser[34] warnings use the same mechanism. I do agree that some pain comes with it, you can end up with assertion numbers that are all over the place or you start by spreading them out in a manner we all thought we left behind when we moved beyond BASIC. I'm not totally committed to it, I just wanted to address the assumption that numbered assertions were worthless to the rest of the world. However, I do want to take a moment to address what I think is a bigger issue in the code. Perhaps it's not enough to be a barrier to inclusion, but since I'm going through the reiser3 code and addressing these now, I thought I'd mention them. All the assertions (a quick grep through the code shows something like 3800 of them) ultimately result in a reiser4_panic, which BUG()'s. Are *all* of these assertions really worth taking the system down? I think a reiser4_error function that can abort just that filesystem and recover somewhat gracefully would be a bit more in order. -Jeff -- Jeff Mahoney SuSE Labs ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 19:29 ` Jeff Mahoney @ 2005-06-23 19:45 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-23 19:45 UTC (permalink / raw) To: Jeff Mahoney Cc: Andrew Morton, penberg, ak, flx, zam, vs, linux-kernel, reiserfs-list, penberg Jeff Mahoney wrote: >All the assertions (a quick grep through the code shows something like >3800 of them) ultimately result in a reiser4_panic, which BUG()'s. Are >*all* of these assertions really worth taking the system down? I think a >reiser4_error function that can abort just that filesystem and recover >somewhat gracefully would be a bit more in order. > > A good point. Thanks for it. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 18:43 ` Andrew Morton 2005-06-23 19:29 ` Jeff Mahoney @ 2005-06-23 19:32 ` Jens Axboe 2005-06-25 16:46 ` Pekka Enberg 1 sibling, 1 reply; 620+ messages in thread From: Jens Axboe @ 2005-06-23 19:32 UTC (permalink / raw) To: Andrew Morton Cc: Jeff Mahoney, penberg, reiser, ak, flx, zam, vs, linux-kernel, reiserfs-list, penberg On Thu, Jun 23 2005, Andrew Morton wrote: > Jeff Mahoney <jeffm@suse.de> wrote: > > > > >>+ assert("nikita-955", pool != NULL); > > > > > > These assertion codes are meaningless to the rest of us so please drop > > > them. > > > > As someone who spends time debugging reiser3 issues, I've grown > > accustomed to the named assertions. They make discussing a particular > > assertion much more natural in conversation than file:line. > > __FUNCTION__? Doesn't help a lot. I've also been annoyed several times in the past when having to lookup a BUG() for a kernel source I don't exactly have at hand right then and there. If you have constructs ala function() { if (cond_a) BUG(); if (cond_b) BUG(); if (cond_c) BUG(); do_stuff... } then it's impossible to know which one it is without the identical source at hand. That said, I don't like the reiser name-number style. If you must do something like this, mark responsibility by using a named identifier covering the layer in question instead. assert("trace_hash-89", is_hashed(foo) != 0); or whatever. -- Jens Axboe ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 19:32 ` Jens Axboe @ 2005-06-25 16:46 ` Pekka Enberg 2005-06-25 19:23 ` Hans Reiser 2005-06-27 7:24 ` Jens Axboe 0 siblings, 2 replies; 620+ messages in thread From: Pekka Enberg @ 2005-06-25 16:46 UTC (permalink / raw) To: Jens Axboe Cc: Andrew Morton, Jeff Mahoney, penberg, reiser, ak, flx, zam, vs, linux-kernel, reiserfs-list Hi, On Thu, 2005-06-23 at 21:32 +0200, Jens Axboe wrote: > then it's impossible to know which one it is without the identical > source at hand. In which case, debugging is risky IMO (the source code could have changed a lot). On Thu, 2005-06-23 at 21:32 +0200, Jens Axboe wrote: > That said, I don't like the reiser name-number style. If you must do > something like this, mark responsibility by using a named identifier > covering the layer in question instead. > > assert("trace_hash-89", is_hashed(foo) != 0); A human readable message would be nicer. For example, "foo was hashed". Pekka ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-25 16:46 ` Pekka Enberg @ 2005-06-25 19:23 ` Hans Reiser 2005-06-25 21:08 ` Theodore Ts'o 2005-06-25 23:54 ` Hubert Chan 2005-06-27 7:24 ` Jens Axboe 1 sibling, 2 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-25 19:23 UTC (permalink / raw) To: Pekka Enberg Cc: Jens Axboe, Andrew Morton, Jeff Mahoney, penberg, ak, flx, zam, vs, linux-kernel, reiserfs-list Pekka Enberg wrote: > > >On Thu, 2005-06-23 at 21:32 +0200, Jens Axboe wrote: > > >>That said, I don't like the reiser name-number style. If you must do >>something like this, mark responsibility by using a named identifier >>covering the layer in question instead. >> >> assert("trace_hash-89", is_hashed(foo) != 0); >> >> Lots of people like corporate anonymity. Some don't. I don't. I like knowing who wrote what. It helps me know who to pay how much. It helps me know who to forward the bug report to. Losing your anonymity exposes you, mostly for better since more communication is on balance a good thing, but the fear is there for some. I don't think we can agree on this, it is an issue of the soul. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-25 19:23 ` Hans Reiser @ 2005-06-25 21:08 ` Theodore Ts'o 2005-06-26 1:03 ` Hans Reiser 2005-06-25 23:54 ` Hubert Chan 1 sibling, 1 reply; 620+ messages in thread From: Theodore Ts'o @ 2005-06-25 21:08 UTC (permalink / raw) To: Hans Reiser Cc: Pekka Enberg, Jens Axboe, Andrew Morton, Jeff Mahoney, penberg, ak, flx, zam, vs, linux-kernel, reiserfs-list On Sat, Jun 25, 2005 at 12:23:41PM -0700, Hans Reiser wrote: > >> > >> assert("trace_hash-89", is_hashed(foo) != 0); > >> > Lots of people like corporate anonymity. Some don't. I don't. I like > knowing who wrote what. It helps me know who to pay how much. It helps > me know who to forward the bug report to. Losing your anonymity > exposes you, mostly for better since more communication is on balance a > good thing, but the fear is there for some. I don't think we can agree > on this, it is an issue of the soul. Fallacy. The assert doesn't tell you who is at fault; it tells you who placed the assert which triggered; it could have triggered due to bugs caused by anyone, including the propietary binary-only module from Nvidia which the user loaded into his system.... - Ted ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-25 21:08 ` Theodore Ts'o @ 2005-06-26 1:03 ` Hans Reiser 0 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-26 1:03 UTC (permalink / raw) To: Theodore Ts'o Cc: Pekka Enberg, Jens Axboe, Andrew Morton, Jeff Mahoney, penberg, ak, flx, zam, vs, linux-kernel, reiserfs-list Theodore Ts'o wrote: >On Sat, Jun 25, 2005 at 12:23:41PM -0700, Hans Reiser wrote: > > >>>> assert("trace_hash-89", is_hashed(foo) != 0); >>>> >>>> >>>> >>Lots of people like corporate anonymity. Some don't. I don't. I like >>knowing who wrote what. It helps me know who to pay how much. It helps >>me know who to forward the bug report to. Losing your anonymity >>exposes you, mostly for better since more communication is on balance a >>good thing, but the fear is there for some. I don't think we can agree >>on this, it is an issue of the soul. >> >> > >Fallacy. > >The assert doesn't tell you who is at fault; it tells you who placed >the assert which triggered; it could have triggered due to bugs caused >by anyone, including the propietary binary-only module from Nvidia >which the user loaded into his system.... > > - Ted > > > > If you read the thread again carefully, you will see that I already said that it doesn't tell you who is at fault for the bug. Furthermore, I said that the basis of the resistance of some developers to the use of this is that they are not fully convinced that others understand that it identifies only the assertion writer not the bug writer. As the boss of the guys writing these assertions, I see no reason to indulge baseless fears. When guys become experienced members of our team, they lose this fear. Sending the bug report to the assertion writer often works nicely as a first step, in my project, in my experience, in the cases where I don't know anything about the likely implications of the assertion myself. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-25 19:23 ` Hans Reiser 2005-06-25 21:08 ` Theodore Ts'o @ 2005-06-25 23:54 ` Hubert Chan 2005-06-26 10:03 ` Nikita Danilov 1 sibling, 1 reply; 620+ messages in thread From: Hubert Chan @ 2005-06-25 23:54 UTC (permalink / raw) To: Hans Reiser Cc: Jens Axboe, Andrew Morton, Jeff Mahoney, penberg, ak, flx, zam, vs, linux-kernel, reiserfs-list On Sat, 25 Jun 2005 12:23:41 -0700, Hans Reiser <reiser@namesys.com> said: >>> assert("trace_hash-89", is_hashed(foo) != 0); > Lots of people like corporate anonymity. Some don't. I don't. I > like knowing who wrote what. ... For what it's worth (I know: not much), I like the named asserts in Reiser4/Reiserfs. Although I haven't been bitten by any BUGs yet (maybe I'm just lucky), whenever I see these on the mailing list, it gives the impression that the users are interacting more directly with the developers, and it helps us to get to know the developers better. If people really want more standard-looking identifiers, I think Namesys should keep the names and make a hybrid identifier, like "nikita-123(<file>:<line>)" -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-25 23:54 ` Hubert Chan @ 2005-06-26 10:03 ` Nikita Danilov 0 siblings, 0 replies; 620+ messages in thread From: Nikita Danilov @ 2005-06-26 10:03 UTC (permalink / raw) To: Hubert Chan Cc: Jens Axboe, Andrew Morton, Jeff Mahoney, penberg, ak, flx, zam, vs, linux-kernel, reiserfs-list Hubert Chan <hubert@uhoreg.ca> writes: > On Sat, 25 Jun 2005 12:23:41 -0700, Hans Reiser <reiser@namesys.com> said: > >>>> assert("trace_hash-89", is_hashed(foo) != 0); > >> Lots of people like corporate anonymity. Some don't. I don't. I >> like knowing who wrote what. ... > > For what it's worth (I know: not much), I like the named asserts in > Reiser4/Reiserfs. Although I haven't been bitten by any BUGs yet (maybe > I'm just lucky), whenever I see these on the mailing list, it gives the > impression that the users are interacting more directly with the > developers, and it helps us to get to know the developers better. > > If people really want more standard-looking identifiers, I think Namesys > should keep the names and make a hybrid identifier, like > "nikita-123(<file>:<line>)" This already happens: together with <uid>-<serial>, reiser4 outputs __FILE__, __LINE__, __FUNCTION__, and a bunch of other stuff: ---------------------------------------------------------------------- reiser4[xine(11262)]: txn_end (fs/reiser4/txnmgr.c:504)[nominaodiosasunt-2967]: code: -2 at fs/reiser4/search.c:1285 reiser4 panicked cowardly: assertion failed: lock_stack_isclean(get_current_lock_stack()) ---------------------------------------------------------------------- Nikita. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-25 16:46 ` Pekka Enberg 2005-06-25 19:23 ` Hans Reiser @ 2005-06-27 7:24 ` Jens Axboe 2005-06-27 7:28 ` Andi Kleen 1 sibling, 1 reply; 620+ messages in thread From: Jens Axboe @ 2005-06-27 7:24 UTC (permalink / raw) To: Pekka Enberg Cc: Andrew Morton, Jeff Mahoney, penberg, reiser, ak, flx, zam, vs, linux-kernel, reiserfs-list On Sat, Jun 25 2005, Pekka Enberg wrote: > Hi, > > On Thu, 2005-06-23 at 21:32 +0200, Jens Axboe wrote: > > then it's impossible to know which one it is without the identical > > source at hand. > > In which case, debugging is risky IMO (the source code could have > changed a lot). That's not an argument, there are plenty of cases where knowing which BUG() triggered provides ample clue to at least start thinking about possible issues. > On Thu, 2005-06-23 at 21:32 +0200, Jens Axboe wrote: > > That said, I don't like the reiser name-number style. If you must do > > something like this, mark responsibility by using a named identifier > > covering the layer in question instead. > > > > assert("trace_hash-89", is_hashed(foo) != 0); > > A human readable message would be nicer. For example, "foo was hashed". Indeed. -- Jens Axboe ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-27 7:24 ` Jens Axboe @ 2005-06-27 7:28 ` Andi Kleen 2005-06-27 7:49 ` Pekka J Enberg 0 siblings, 1 reply; 620+ messages in thread From: Andi Kleen @ 2005-06-27 7:28 UTC (permalink / raw) To: Jens Axboe Cc: Pekka Enberg, Andrew Morton, Jeff Mahoney, penberg, reiser, ak, flx, zam, vs, linux-kernel, reiserfs-list > > On Thu, 2005-06-23 at 21:32 +0200, Jens Axboe wrote: > > > That said, I don't like the reiser name-number style. If you must do > > > something like this, mark responsibility by using a named identifier > > > covering the layer in question instead. > > > > > > assert("trace_hash-89", is_hashed(foo) != 0); > > > > A human readable message would be nicer. For example, "foo was hashed". > > Indeed. You can just dump the expression (with #argument). That is what traditional userspace assert did forever. It won't help for BUG_ON(a || b || c || d || e) but these are bad style anyways and should be avoided. -Andi ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-27 7:28 ` Andi Kleen @ 2005-06-27 7:49 ` Pekka J Enberg 2005-06-27 8:19 ` Jörn Engel 2005-06-27 8:20 ` Andi Kleen 0 siblings, 2 replies; 620+ messages in thread From: Pekka J Enberg @ 2005-06-27 7:49 UTC (permalink / raw) To: Andi Kleen Cc: Jens Axboe, Andrew Morton, Jeff Mahoney, penberg, reiser, flx, zam, vs, linux-kernel, reiserfs-list On Mon, 2005-06-27 at 09:28 +0200, Andi Kleen wrote: > You can just dump the expression (with #argument). That is what > traditional userspace assert did forever. > > It won't help for BUG_ON(a || b || c || d || e) but these > are bad style anyways and should be avoided. Sounds good to me. Jens, would this work for you? Pekka Show BUG_ON expression when assertion fails. Signed-off-by: Pekka Enberg <penberg@cs.helsinki.fi> --- bug.h | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletion(-) Index: 2.6/include/asm-generic/bug.h =================================================================== --- 2.6.orig/include/asm-generic/bug.h +++ 2.6/include/asm-generic/bug.h @@ -13,7 +13,12 @@ #endif #ifndef HAVE_ARCH_BUG_ON -#define BUG_ON(condition) do { if (unlikely((condition)!=0)) BUG(); } while(0) +#define BUG_ON(condition) do { \ + if (unlikely((condition) != 0)) { \ + printk("kernel BUG '%s' at %s:%d!\n", #condition, __FILE__, __LINE__); \ + panic("BUG!"); \ + } \ +} while(0) #endif #ifndef HAVE_ARCH_WARN_ON ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-27 7:49 ` Pekka J Enberg @ 2005-06-27 8:19 ` Jörn Engel 2005-06-27 8:20 ` Andi Kleen 1 sibling, 0 replies; 620+ messages in thread From: Jörn Engel @ 2005-06-27 8:19 UTC (permalink / raw) To: Pekka J Enberg Cc: Andi Kleen, Jens Axboe, Andrew Morton, Jeff Mahoney, penberg, reiser, flx, zam, vs, linux-kernel, reiserfs-list On Mon, 27 June 2005 10:49:19 +0300, Pekka J Enberg wrote: > > #ifndef HAVE_ARCH_BUG_ON > -#define BUG_ON(condition) do { if (unlikely((condition)!=0)) BUG(); } while(0) > +#define BUG_ON(condition) do { \ > + if (unlikely((condition) != 0)) { \ > + printk("kernel BUG '%s' at %s:%d!\n", #condition, __FILE__, __LINE__); \ > + panic("BUG!"); \ > + } \ > +} while(0) > #endif o How about those architectures, where BUG() and panic() are not the same thing? o Embedded people might prefer not to have the additional string constants in the kernel. Some config option would ease their wrath. And no, not all embedded people #define BUG() to nothing. Jörn -- Happiness isn't having what you want, it's wanting what you have. -- unknown ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-27 7:49 ` Pekka J Enberg 2005-06-27 8:19 ` Jörn Engel @ 2005-06-27 8:20 ` Andi Kleen 1 sibling, 0 replies; 620+ messages in thread From: Andi Kleen @ 2005-06-27 8:20 UTC (permalink / raw) To: Pekka J Enberg Cc: Andi Kleen, Jens Axboe, Andrew Morton, Jeff Mahoney, penberg, reiser, flx, zam, vs, linux-kernel, reiserfs-list On Mon, Jun 27, 2005 at 10:49:19AM +0300, Pekka J Enberg wrote: > On Mon, 2005-06-27 at 09:28 +0200, Andi Kleen wrote: > > You can just dump the expression (with #argument). That is what > > traditional userspace assert did forever. > > > > It won't help for BUG_ON(a || b || c || d || e) but these > > are bad style anyways and should be avoided. > > Sounds good to me. Jens, would this work for you? It won't work for me because it'll bloat the kernel .text considerable. There is a reason why BUG is implemented like it is. Compare it. -Andi ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 18:37 ` Jeff Mahoney 2005-06-23 18:43 ` Andrew Morton @ 2005-06-23 19:24 ` Hans Reiser 2005-06-24 1:13 ` Hubert Chan 2 siblings, 0 replies; 620+ messages in thread From: Hans Reiser @ 2005-06-23 19:24 UTC (permalink / raw) To: Jeff Mahoney Cc: Pekka Enberg, Andi Kleen, Alexander Lyamin aka FLX, Alexander Zarochentcev, vs, Andrew Morton, linux-kernel, ReiserFS List, Pekka Enberg Jeff Mahoney wrote: >Pekka Enberg wrote: > > >>>--- /dev/null 2003-09-23 21:59:22.000000000 +0400 >>>+++ linux-2.6.11-vs/fs/reiser4/pool.c 2005-06-03 17:49:38.668204642 +0400 >>>+/* initialise new pool */ >>>+reiser4_internal void >>>+reiser4_init_pool(reiser4_pool * pool /* pool to initialise */ , >>>+ size_t obj_size /* size of objects in @pool */ , >>>+ int num_of_objs /* number of preallocated objects */ , >>>+ char *data /* area for preallocated objects */ ) >>>+{ >>>+ reiser4_pool_header *h; >>>+ int i; >>>+ >>>+ assert("nikita-955", pool != NULL); >>> >>> >>These assertion codes are meaningless to the rest of us so please drop >>them. >> >> > >As someone who spends time debugging reiser3 issues, I've grown >accustomed to the named assertions. They make discussing a particular >assertion much more natural in conversation than file:line. It also >makes difficult to reproduce assertions more trackable over time. The >assertion number never changes, but the line number can with even the >most trivial of patches. > >That said, one of my gripes with the named assertions in reiser3 (and >reiser4 now) is that the development staff changes over time. There are >many named assertions in reiser3 that refer to developers no longer >employed by Hans. The quoted one is a perfect example. > > Yes, but when I get stuck I still send him an email and he still sends me an answer. He is a nice guy even if he can't stand working for me.... >Hans, would it be acceptable to you to keep only numbered assertions and > keep a code responsbility list internal to namesys? > No effort to document who is the current maintainer of a section of our code (and thus presumed to be able to figure something nonobvious about it out) has ever worked as well as these named assertions. Efforts to put at the top of files who worked on what part of it always get miserably out of date, and people are always too shy to update them for valid social reasons. Internal lists are not the open source way. The reason some developers hate these named assertions is because they are afraid that it makes them famous for their bugs, when actually it does not. Assertions hit are not equal to bugs authored, and users understand that better than those developers think. Writing an assertion means you can understand a bug, not that you created it. The real effect of your name being on many assertions is that people know you contributed a lot. It is not necessary for Namesys to be an opaque corporation with no faces on its surface. My name is on the filesystem, every bit of credit I can give to the others I owe them many times over. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 18:37 ` Jeff Mahoney @ 2005-06-24 1:13 ` Hubert Chan 2005-06-23 19:24 ` Hans Reiser 2005-06-24 1:13 ` Hubert Chan 2 siblings, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-24 1:13 UTC (permalink / raw) To: linux-kernel; +Cc: reiserfs-list On Thu, 23 Jun 2005 14:37:05 -0400, Jeff Mahoney <jeffm@suse.de> said: > As someone who spends time debugging reiser3 issues, I've grown > accustomed to the named assertions. They make discussing a particular > assertion much more natural in conversation than file:line. It also > makes difficult to reproduce assertions more trackable over time. The > assertion number never changes, but the line number can with even the > most trivial of patches. How about something of the form "nikita-955(file:line)"? Or the reverse: "file:line(nikita-955)". Would that keep everyone happy? -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status @ 2005-06-24 1:13 ` Hubert Chan 0 siblings, 0 replies; 620+ messages in thread From: Hubert Chan @ 2005-06-24 1:13 UTC (permalink / raw) To: reiserfs-list; +Cc: linux-kernel On Thu, 23 Jun 2005 14:37:05 -0400, Jeff Mahoney <jeffm@suse.de> said: > As someone who spends time debugging reiser3 issues, I've grown > accustomed to the named assertions. They make discussing a particular > assertion much more natural in conversation than file:line. It also > makes difficult to reproduce assertions more trackable over time. The > assertion number never changes, but the line number can with even the > most trivial of patches. How about something of the form "nikita-955(file:line)"? Or the reverse: "file:line(nikita-955)". Would that keep everyone happy? -- Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred. ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-24 1:13 ` Hubert Chan (?) @ 2005-06-26 0:45 ` Christian Trefzer 2005-06-26 1:13 ` Hans Reiser -1 siblings, 1 reply; 620+ messages in thread From: Christian Trefzer @ 2005-06-26 0:45 UTC (permalink / raw) To: Hubert Chan; +Cc: linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 1160 bytes --] Hubert Chan schrieb: > How about something of the form "nikita-955(file:line)"? Or the > reverse: "file:line(nikita-955)". Would that keep everyone happy? > Damn, I was wondering how long it would take until someone would come up with a compromise solution ; ) Compromises everywhere will lead to nowhere, I've learned that the hard way. But this is really not a major issue, so let's not make a showstopper out of this one and the likes. For what I know about the whole inclusion discussion until now, there's been a whole lot of flamewar chickenshit so far. Considering that I have no FS developing abilities whatsoever, I'm pretty pissed at people who do know better in their field and should know better than waste their time on discussions other than constructive ones. Get the personal bullshit out of the way, everyone, please! Get in touch and work out your differences in a productive manner. If every interesting yet at first intrusive extension to the kernel causes as much kindergarten as this one, where will we end up? Stagnation sucks, yet good progress is sometimes slow-paced... Peace, everyone! Chris (hardcore, not hippie) [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 894 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-26 0:45 ` Christian Trefzer @ 2005-06-26 1:13 ` Hans Reiser 2005-06-26 2:23 ` Christian Trefzer 0 siblings, 1 reply; 620+ messages in thread From: Hans Reiser @ 2005-06-26 1:13 UTC (permalink / raw) To: Christian Trefzer; +Cc: Hubert Chan, linux-kernel, reiserfs-list Christian Trefzer wrote: > Hubert Chan schrieb: > >> How about something of the form "nikita-955(file:line)"? Or the >> reverse: "file:line(nikita-955)". Would that keep everyone happy? >> Makes me happy..... ^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-26 1:13 ` Hans Reiser @ 2005-06-26 2:23 ` Christian Trefzer 0 siblings, 0 replies; 620+ messages in thread From: Christian Trefzer @ 2005-06-26 2:23 UTC (permalink / raw) To: Hans Reiser; +Cc: linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 400 bytes --] Hans Reiser schrieb: > Christian Trefzer wrote: > > >>Hubert Chan schrieb: >> >> >>>How about something of the form "nikita-955(file:line)"? Or the >>>reverse: "file:line(nikita-955)". Would that keep everyone happy? >>> > > Makes me happy..... > Nice, how about the others? Hey, if we need some objective and neutral mediators on lkml, I'd be glad to give my reading frenzies a meaning : ) [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 894 bytes --] ^ permalink raw reply [flat|nested] 620+ messages in thread
[parent not found: <4hNoW-1yo-37@gated-at.bofh.it>]
[parent not found: <4hT1h-5V0-41@gated-at.bofh.it>]
[parent not found: <4hXHv-1br-41@gated-at.bofh.it>]
* Re: -mm -> 2.6.13 merge status [not found] ` <4hXHv-1br-41@gated-at.bofh.it> @ 2005-06-22 14:40 ` Bodo Eggert 0 siblings, 0 replies; 620+ messages in thread From: Bodo Eggert @ 2005-06-22 14:40 UTC (permalink / raw) To: Andrew Morton, Jesper Juhl, Linus Torvalds, linux-kernel CC Linus, Jesper Juhl (who's currently doing some cleanups) Andrew Morton <akpm@osdl.org> wrote: > *general sigh*. I wish people would absorb CodingStyle. It's not hard, > and fixing the style post-facto creates a real mess. I now have a great > string of kexec patches followed by a "kexec-code-cleanup.patch" which > totally buggers up the patch sequencing and really needs to be split into > 18 parts and sprinkled back over the entire series. I scripted an automatic whitespace cleanup, which resuled in a fat patchbomb (about 18 MB, split into > 3600 files (because that way, some patches are going to apply cleanly)). Obviously applying that would increase the patch size for the next version by 100%, so that won't be the way to go. (If you still want to look, see http://7eggert.dyndns.org/l/patches/trailing-ws/) Therefore I suggest that I will - make a script that will take a patch, apply it and cleanup the patched files as far as a simple script can do the job, so each patched file will be ws-clean and the amount of patches will still stay low. - a second script that will do some cleanup while the resulting patch is less than an annoying amount of KB, so you can cleanup some files that wouldn't get patched otherwise. -- Ich danke GMX dafür, die Verwendung meiner Adressen mittels per SPF verbreiteten Lügen zu sabotieren. ^ permalink raw reply [flat|nested] 620+ messages in thread
* RE: -mm -> 2.6.13 merge status
@ 2005-06-23 13:20 Ian Pratt
2005-06-23 13:37 ` Mark Williamson
0 siblings, 1 reply; 620+ messages in thread
From: Ian Pratt @ 2005-06-23 13:20 UTC (permalink / raw)
To: Carsten Otte, Chris Wright
Cc: Andrew Morton, Jeff Garzik, linux-kernel, Mark Williamson, ian.pratt
> > Xen is making similar noises w.r.t. using kexec for
> flexible bootloader.
>
> Oh cool, then we should look at what they're doing instead of
> reinventing the wheel. Any pointer we can follow, or person
> we would contact?
Mark.Williamson@cl.cam.ac.uk
Best,
Ian
^ permalink raw reply [flat|nested] 620+ messages in thread
* Re: -mm -> 2.6.13 merge status 2005-06-23 13:20 Ian Pratt @ 2005-06-23 13:37 ` Mark Williamson 0 siblings, 0 replies; 620+ messages in thread From: Mark Williamson @ 2005-06-23 13:37 UTC (permalink / raw) To: Ian Pratt Cc: Carsten Otte, Chris Wright, Andrew Morton, Jeff Garzik, linux-kernel, ian.pratt > > > Xen is making similar noises w.r.t. using kexec for > > > > flexible bootloader. > > > > Oh cool, then we should look at what they're doing instead of > > reinventing the wheel. Any pointer we can follow, or person > > we would contact? Right now, that would be me (hello!). All going well - I should have Xen guests kexec-ing properly in the next day or so. All the machinery is in place, so it's just a question of doing some plumbing. nb. this is a separate issue to kexecing the whole host, which I'll probably look at later. Carsten: can you tell me what you were planning for your bootloader? Also, if you could point me at any docs regarding your current / proposed boot sequence that'd be really interesting. Regarding our kexec-based bootloader: * We call running virtual machines "domains". * Under Xen, guests get built using a kernel specified in domain 0's (the "host") filesystem. That implies that the user of the guest domain can't choose their kernel. * To fix this we now have a bootloader than runs in domain 0, which can poke around in the guest's filesystem, find a menu.lst / grub.conf, then load the appropriate kernel. * This provides the functionality the user wants but implies that programs in dom0 have to access the guest filesystem, which could be malicious. We'd rather not have programs in dom0 trusting guests. * The proposed solution is to initially run a "bootloader kernel", with a bootloader app on an initrd. This will run in the guest itself, so all the filesystem accesses occur in an unprivileged virtual machine. * The bootloader will cause a kexec to the new kernel. * This fixes the isolation problem and immediately allows us to support all the random filesystems Linux supports ;-) My current plan is that the bootloader app will act much like Grub and use Grub's config files. It'll also be possible to use something more heavyweight, such as XenoBoot (http://www.cl.cam.ac.uk/Research/SRG/netos/xeno/xenoboot/). It's possible that XenoBoot has some code we could use in the bootloader - I haven't looked. Feel free to mail me offline. If our goals are compatible, it'd be good to work together on this. Cheers, Mark ^ permalink raw reply [flat|nested] 620+ messages in thread
end of thread, other threads:[~2005-07-14 22:16 UTC | newest] Thread overview: 620+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2005-06-21 6:54 -mm -> 2.6.13 merge status Andrew Morton 2005-06-21 10:22 ` -mm -> 2.6.13 merge status (fuse) Miklos Szeredi 2005-06-21 12:39 ` Alan Cox 2005-06-21 13:13 ` Miklos Szeredi 2005-06-21 14:28 ` Pavel Machek 2005-06-21 14:51 ` John Stoffel 2005-06-21 15:17 ` Miklos Szeredi 2005-06-21 15:13 ` Miklos Szeredi 2005-06-21 22:06 ` Pavel Machek 2005-06-22 6:20 ` Miklos Szeredi 2005-06-22 6:39 ` Andrew Morton 2005-06-22 7:16 ` Miklos Szeredi 2005-06-22 7:49 ` Andrew Morton 2005-06-22 9:07 ` Miklos Szeredi 2005-06-22 9:12 ` Andrew Morton 2005-06-22 9:20 ` Miklos Szeredi 2005-06-22 9:44 ` Andrew Morton 2005-06-22 9:58 ` Miklos Szeredi 2005-06-22 10:19 ` Andrew Morton 2005-06-22 10:31 ` Miklos Szeredi 2005-06-22 12:23 ` Eric Van Hensbergen 2005-06-22 16:28 ` Miklos Szeredi 2005-06-22 16:48 ` Eric Van Hensbergen 2005-06-22 17:19 ` Miklos Szeredi 2005-06-22 9:17 ` Pavel Machek 2005-06-22 17:01 ` Theodore Ts'o 2005-06-22 17:48 ` Miklos Szeredi 2005-06-23 23:34 ` Theodore Ts'o 2005-06-24 5:49 ` Miklos Szeredi 2005-06-26 3:04 ` Theodore Ts'o 2005-06-24 16:00 ` Gábor Lénárt 2005-06-22 8:53 ` Pavel Machek 2005-06-22 12:57 ` Matthias Urlichs 2005-06-22 14:43 ` Eric Van Hensbergen 2005-06-22 15:08 ` Pavel Machek 2005-06-22 15:50 ` Eric Van Hensbergen 2005-06-22 16:34 ` Eric Van Hensbergen 2005-06-22 16:43 ` Alan Cox 2005-06-22 16:56 ` Eric Van Hensbergen 2005-06-21 15:26 ` Avuton Olrich 2005-06-21 15:36 ` Miklos Szeredi 2005-06-21 12:01 ` -mm -> 2.6.13 merge status Andrey Panin 2005-06-21 12:35 ` Alan Cox 2005-06-21 13:07 ` Arjan van de Ven 2005-06-22 10:50 ` Erik Slagter 2005-06-21 12:43 ` Carsten Otte 2005-06-21 12:58 ` Jörn Engel 2005-06-21 13:08 ` -mm -> 2.6.13 merge status (Suspend-to-disk) Nigel Cunningham 2005-06-21 9:44 ` Marcelo Tosatti 2005-06-21 14:16 ` Pavel Machek 2005-06-21 13:51 ` v9fs (-mm -> 2.6.13 merge status) Eric Van Hensbergen 2005-06-21 15:35 ` Uriel 2005-06-22 5:13 ` Martin Atkins [not found] ` <a4e6962a05062112206e823c0a@mail.gmail.com> 2005-06-21 20:52 ` Fwd: " Ronald G. Minnich 2005-06-21 14:08 ` -mm -> 2.6.13 merge status Martin Hicks 2005-06-21 19:54 ` Andrew Morton 2005-06-21 20:00 ` Martin Hicks 2005-06-21 14:19 ` -mm -> 2.6.13 merge status (wireless) Pavel Machek 2005-06-21 20:02 ` Andrew Morton 2005-06-22 3:26 ` Jeff Garzik 2005-06-21 15:26 ` -mm -> 2.6.13 merge status Jeff Garzik 2005-06-21 15:39 ` Robert Love 2005-06-21 19:22 ` Christoph Lameter 2005-06-21 19:38 ` Robert Love 2005-06-21 19:44 ` Christoph Lameter 2005-06-21 20:02 ` Zan Lynx 2005-06-21 20:06 ` Christoph Lameter 2005-06-21 20:07 ` Robert Love 2005-06-21 20:10 ` Christoph Lameter 2005-06-21 20:15 ` Zan Lynx 2005-06-22 5:53 ` Matthias Urlichs 2005-06-21 22:54 ` Alan Cox 2005-06-21 20:52 ` Stephen Hemminger 2005-06-21 22:45 ` Jeff Garzik 2005-06-21 23:30 ` Christoph Lameter 2005-06-21 23:39 ` Jeff Garzik 2005-06-22 6:19 ` Christoph Lameter 2005-06-21 15:43 ` Matt Porter 2005-06-21 19:41 ` randy_dunlap 2005-06-21 20:05 ` Hans Reiser 2005-06-21 20:24 ` Christoph Hellwig 2005-06-22 1:07 ` reiser4 plugins Hans Reiser 2005-06-22 1:14 ` Jeff Garzik 2005-06-22 4:25 ` David Masover 2005-06-22 5:11 ` Bedros Hanounik 2005-06-22 5:18 ` Jeff Garzik 2005-06-22 8:27 ` Hans Reiser 2005-06-22 9:08 ` David Masover 2005-06-22 14:28 ` Nikita Danilov 2005-06-22 16:13 ` Vladimir Saveliev 2005-06-22 16:59 ` Nikita Danilov 2005-06-26 17:02 ` Christoph Hellwig 2005-06-27 9:30 ` Alexander Zarochentsev 2005-06-27 9:42 ` Christoph Hellwig 2005-06-27 11:28 ` Alexander Zarochentsev 2005-06-27 19:22 ` Christoph Hellwig 2005-06-28 14:01 ` Horst von Brand 2005-06-28 18:52 ` Sean 2005-06-28 19:28 ` David Masover 2005-06-23 5:58 ` Hans Reiser 2005-06-23 6:26 ` Pekka Enberg 2005-06-23 14:11 ` David Masover 2005-06-23 19:24 ` Horst von Brand 2005-06-23 20:12 ` Adrian Ulrich 2005-06-23 20:12 ` Adrian Ulrich 2005-06-23 20:49 ` Michael Dreher 2005-06-23 21:54 ` M. 2005-06-23 23:37 ` Alan Cox 2005-06-23 23:57 ` Hans Reiser 2005-06-24 0:16 ` Bernd Eckenfels 2005-06-24 8:52 ` zhilla 2005-06-28 11:14 ` Vitaly Fertman 2005-06-23 21:29 ` Avuton Olrich 2005-06-23 21:36 ` Dan Oglesby 2005-06-24 1:19 ` Jim Crilly 2005-06-23 22:04 ` David Masover 2005-06-23 23:43 ` Alan Cox 2005-06-24 1:12 ` Hans Reiser 2005-06-24 1:45 ` Jeff Garzik 2005-06-24 2:31 ` Lincoln Dale 2005-06-24 3:21 ` Jeff Garzik 2005-06-24 6:49 ` Hans Reiser 2005-06-24 7:10 ` Lincoln Dale 2005-06-24 8:23 ` Hans Reiser 2005-06-24 8:40 ` Lincoln Dale 2005-06-24 15:32 ` Horst von Brand 2005-06-27 19:39 ` Tyson Whitehead 2005-06-24 9:35 ` Timothy Webster 2005-06-24 15:45 ` Horst von Brand 2005-06-24 17:13 ` Perry Kundert 2005-06-24 20:01 ` Fwd: " Perry Kundert 2005-06-24 21:38 ` Valdis.Kletnieks 2005-06-24 22:20 ` Perry Kundert 2005-06-25 0:37 ` Valdis.Kletnieks 2005-06-25 0:53 ` Hans Reiser 2005-06-25 2:20 ` Valdis.Kletnieks 2005-06-25 2:40 ` David Masover 2005-06-25 2:49 ` David Masover 2005-06-24 7:11 ` Al Viro 2005-06-24 9:03 ` Hans Reiser 2005-06-24 14:45 ` Al Viro 2005-06-24 14:53 ` Al Viro 2005-06-24 18:16 ` Hans Reiser 2005-06-24 10:41 ` Alan Cox 2005-06-27 9:18 ` Markus Törnqvist 2005-06-27 9:18 ` mjt 2005-06-27 9:46 ` Nick Piggin 2005-06-27 12:55 ` Markus Törnqvist 2005-06-27 12:55 ` mjt 2005-06-28 0:23 ` Nick Piggin 2005-06-28 20:39 ` Markus Törnqvist 2005-06-28 20:39 ` mjt 2005-06-30 23:20 ` Jesper Juhl 2005-06-27 14:01 ` Alan Cox 2005-06-27 14:01 ` Alan Cox 2005-06-24 3:17 ` David Masover 2005-06-24 3:34 ` Horst von Brand 2005-06-25 3:38 ` David Masover 2005-06-27 9:21 ` Markus Törnqvist 2005-06-27 9:21 ` mjt 2005-06-27 12:42 ` Theodore Ts'o 2005-06-27 15:19 ` David Masover 2005-06-27 16:28 ` Theodore Ts'o 2005-06-27 21:12 ` David Masover 2005-06-30 21:49 ` Theodore Ts'o 2005-06-30 22:34 ` Dmitry Torokhov 2005-07-01 7:03 ` backup (was Re: reiser4 plugins) David Masover 2005-07-01 14:19 ` Theodore Ts'o 2005-07-01 19:49 ` David Masover 2005-07-01 8:08 ` reiser4 plugins Hans Reiser 2005-07-01 8:06 ` Hans Reiser 2005-06-27 19:46 ` Hans Reiser 2005-06-27 20:18 ` Steve Lord 2005-06-27 20:28 ` Theodore Ts'o 2005-06-27 20:47 ` Hans Reiser 2005-06-27 20:58 ` Steve Lord 2005-06-27 23:06 ` Prakash Punnoor 2005-06-28 0:40 ` Hans Reiser 2005-06-28 1:00 ` Zan Lynx 2005-06-28 1:10 ` Gregory Maxwell 2005-06-28 20:15 ` David Masover 2005-06-28 1:07 ` Jim Crilly 2005-06-28 4:03 ` Prakash Punnoor 2005-06-28 4:19 ` Jim Crilly 2005-06-28 6:37 ` Al Boldi 2005-06-27 21:26 ` Theodore Ts'o 2005-06-27 23:00 ` reiser4 merging action list Hans Reiser 2005-06-27 23:23 ` Andrew Morton 2005-06-29 5:41 ` Hans Reiser 2005-06-29 6:18 ` Pekka Enberg 2005-06-29 22:59 ` Hans Reiser 2005-06-28 9:41 ` Christoph Hellwig 2005-06-28 9:48 ` Adrian Bunk 2005-06-28 0:06 ` reiser4 plugins Hans Reiser 2005-06-28 13:44 ` Horst von Brand 2005-06-28 13:44 ` Horst von Brand 2005-06-28 20:47 ` Markus Törnqvist 2005-06-28 20:47 ` mjt 2005-06-28 21:48 ` Jake Maciejewski 2005-06-24 11:34 ` Alan Cox 2005-06-24 19:21 ` Hans Reiser 2005-06-24 22:04 ` Olivier Galibert 2005-06-24 23:06 ` Theodore Ts'o 2005-06-25 0:35 ` Hans Reiser 2005-06-26 17:20 ` Alan Cox 2005-06-26 17:38 ` Grzegorz Kulewski 2005-06-29 16:44 ` torturing filesystems [was Re: reiser4 plugins] Pavel Machek 2005-06-25 3:14 ` reiser4 plugins David Masover 2005-06-25 3:18 ` Jeff Garzik 2005-06-25 4:31 ` Hans Reiser 2005-06-25 4:43 ` Jeff Garzik 2005-06-25 6:01 ` Hans Reiser 2005-06-25 4:49 ` David Masover 2005-06-25 6:15 ` Hans Reiser 2005-06-25 6:38 ` Gregory Maxwell 2005-06-25 6:47 ` David Masover 2005-06-25 7:33 ` Bob R. Taylor 2005-06-25 14:45 ` Diego Calleja 2005-06-29 2:07 ` Matthew Frost 2005-06-25 5:26 ` Jesper Krogh 2005-06-25 5:26 ` Jesper Krogh 2005-06-25 7:46 ` David Masover 2005-06-26 17:23 ` Alan Cox 2005-06-24 12:49 ` Olivier Galibert 2005-06-25 2:52 ` David Masover 2005-06-29 16:36 ` Pavel Machek 2005-06-24 19:46 ` Hans Reiser 2005-06-27 20:52 ` Vitaly Fertman 2005-06-27 21:07 ` David Masover 2005-06-28 8:32 ` Vitaly Fertman 2005-06-28 19:14 ` David Masover 2005-06-24 1:02 ` Hans Reiser 2005-06-24 2:41 ` Horst von Brand 2005-06-24 18:42 ` Hubert Chan 2005-06-24 18:42 ` Hubert Chan 2005-06-25 4:10 ` David Masover 2005-06-25 14:20 ` Valdis.Kletnieks 2005-06-25 18:33 ` David Masover 2005-06-25 20:31 ` Valdis.Kletnieks 2005-06-25 20:52 ` Hans Reiser 2005-06-26 4:59 ` Lincoln Dale 2005-06-26 5:07 ` Gregory Maxwell 2005-06-26 7:16 ` Lincoln Dale 2005-06-26 7:48 ` David Masover 2005-06-26 8:26 ` Lincoln Dale 2005-06-26 9:35 ` David Masover 2005-06-26 18:16 ` Valdis.Kletnieks 2005-06-26 19:58 ` David Masover 2005-06-26 21:05 ` Valdis.Kletnieks 2005-06-26 22:35 ` David Masover 2005-06-26 23:43 ` Hans Reiser 2005-06-27 0:16 ` David Masover 2005-06-27 0:36 ` Valdis.Kletnieks 2005-06-27 3:48 ` Hans Reiser 2005-06-27 5:05 ` Horst von Brand 2005-06-27 5:52 ` Gregory Maxwell 2005-06-27 6:22 ` David Masover 2005-06-27 0:40 ` Valdis.Kletnieks 2005-06-27 2:37 ` David Masover 2005-06-27 3:10 ` Kyle Moffett 2005-06-27 3:24 ` David Masover 2005-06-27 3:40 ` Kyle Moffett 2005-06-27 21:19 ` David Masover 2005-06-27 23:03 ` Kyle Moffett 2005-06-27 23:27 ` David Masover 2005-06-28 2:21 ` Hubert Chan 2005-06-28 2:59 ` Kyle Moffett 2005-06-28 3:45 ` Hubert Chan 2005-06-28 4:38 ` Kyle Moffett 2005-06-28 5:30 ` Hubert Chan 2005-06-28 6:01 ` Kyle Moffett 2005-06-28 17:51 ` Hubert Chan 2005-06-28 19:32 ` David Masover 2005-06-28 19:57 ` Hubert Chan 2005-06-28 19:57 ` Hubert Chan 2005-06-28 20:03 ` Kyle Moffett 2005-06-28 19:59 ` Kyle Moffett 2005-06-29 1:40 ` Hubert Chan 2005-06-29 5:09 ` Horst von Brand 2005-06-29 13:50 ` Douglas McNaught 2005-06-29 13:58 ` Markus Törnqvist 2005-06-29 13:58 ` mjt 2005-06-29 17:19 ` Horst von Brand 2005-06-29 17:19 ` Horst von Brand 2005-06-29 20:57 ` Hubert Chan 2005-06-29 20:57 ` Hubert Chan 2005-06-30 9:59 ` Markus Törnqvist 2005-06-30 9:59 ` mjt [not found] ` <17091.60930.633968.822210@gargle.gargle.HOWL> 2005-06-30 14:21 ` Markus Törnqvist 2005-06-30 14:21 ` mjt [not found] ` <17092.3415.28856.827179@gargle.gargle.HOWL> 2005-06-30 15:37 ` Markus Törnqvist 2005-06-30 15:37 ` mjt 2005-06-30 19:45 ` Diego Calleja 2005-06-30 19:45 ` Diego Calleja 2005-06-30 19:52 ` Horst von Brand 2005-06-30 19:52 ` Horst von Brand 2005-07-05 20:46 ` Hubert Chan 2005-07-10 0:00 ` Stefan Smietanowski 2005-07-11 12:28 ` Jaroslav Soltys 2005-07-11 23:17 ` Hubert Chan 2005-07-01 8:16 ` David Masover 2005-06-29 19:05 ` Valdis.Kletnieks 2005-06-29 19:05 ` Valdis.Kletnieks 2005-06-29 20:56 ` David Weinehall 2005-06-29 23:05 ` Chet Hosey 2005-06-30 10:01 ` Markus Törnqvist 2005-06-30 10:01 ` mjt 2005-06-30 12:45 ` Linux and Plan-9ness Al Boldi 2005-06-30 12:45 ` Al Boldi 2005-06-30 14:08 ` Markus Törnqvist 2005-06-30 14:08 ` mjt 2005-07-04 17:18 ` studdugie 2005-06-30 19:57 ` reiser4 plugins Eric Van Hensbergen 2005-07-01 8:08 ` David Masover 2005-07-01 15:54 ` David Weinehall 2005-07-01 19:55 ` David Masover 2005-07-02 1:48 ` Horst von Brand 2005-07-04 3:42 ` Hans Reiser 2005-07-04 7:17 ` malcolm 2005-07-04 7:22 ` Hans Reiser 2005-07-04 9:16 ` Christoph Hellwig 2005-07-04 18:47 ` David Masover 2005-07-04 20:32 ` Horst von Brand 2005-07-05 22:31 ` David Masover 2005-07-05 22:43 ` Jeremy Maitin-Shepard 2005-07-05 22:52 ` David Masover 2005-07-05 23:12 ` Jeremy Maitin-Shepard 2005-07-06 0:31 ` Hans Reiser 2005-07-06 8:56 ` Christoph Hellwig 2005-07-07 8:27 ` Markus Törnqvist 2005-07-07 8:27 ` mjt 2005-07-07 14:00 ` David Masover 2005-07-07 17:47 ` Miquel van Smoorenburg 2005-06-29 20:40 ` Hubert Chan 2005-06-29 21:34 ` Ross Biro 2005-06-29 23:29 ` Hubert Chan 2005-07-01 8:06 ` David Masover 2005-07-01 8:17 ` Hans Reiser 2005-07-01 8:27 ` David Masover 2005-07-01 8:44 ` Hans Reiser 2005-07-05 21:01 ` Hubert Chan 2005-07-05 22:01 ` Hans Reiser 2005-07-05 22:21 ` David Masover 2005-07-05 22:51 ` Hans Reiser 2005-07-05 23:00 ` David Masover 2005-07-05 23:47 ` Hans Reiser 2005-07-06 0:15 ` Hans Reiser 2005-07-06 14:01 ` David Masover 2005-07-11 4:07 ` Stefan Smietanowski 2005-07-11 20:50 ` David Masover 2005-07-11 20:54 ` Stefan Smietanowski 2005-07-11 22:58 ` Hans Reiser 2005-07-12 2:33 ` Horst von Brand 2005-07-12 5:10 ` Stefan Traby 2005-07-12 5:53 ` Hans Reiser 2005-07-12 23:22 ` David Masover 2005-07-12 23:38 ` Hans Reiser 2005-07-13 3:43 ` David Masover 2005-07-13 2:09 ` Horst von Brand 2005-07-12 1:53 ` Horst von Brand 2005-07-12 7:19 ` Neil Brown 2005-07-12 7:45 ` Hans Reiser 2005-07-13 0:05 ` Neil Brown 2005-07-13 1:20 ` Hans Reiser 2005-07-12 23:13 ` David Masover 2005-07-12 23:40 ` Neil Brown 2005-07-05 23:56 ` Hans Reiser 2005-07-06 0:50 ` Alexander G. M. Smith 2005-07-06 1:16 ` Hubert Chan 2005-07-06 6:44 ` Hans Reiser 2005-07-06 13:09 ` David Masover 2005-07-06 13:43 ` Adrian Ulrich 2005-07-06 14:11 ` David Masover 2005-07-06 14:55 ` Adrian Ulrich 2005-07-06 18:07 ` Hans Reiser 2005-07-06 20:47 ` David Masover 2005-07-06 22:49 ` Hans Reiser 2005-07-06 18:52 ` Jonathan Briggs 2005-07-06 19:51 ` Hubert Chan 2005-07-06 20:33 ` Jonathan Briggs 2005-07-06 20:53 ` David Masover 2005-07-06 21:28 ` Jonathan Briggs 2005-07-06 20:57 ` Hubert Chan 2005-07-06 20:33 ` Horst von Brand 2005-07-06 21:31 ` Hubert Chan 2005-07-07 2:33 ` David Masover 2005-07-07 3:13 ` Jan Harkes 2005-07-07 6:42 ` Hans Reiser 2005-07-08 16:39 ` Hubert Chan 2005-07-08 16:45 ` David Masover 2005-07-06 1:59 ` Neil Brown 2005-07-06 16:06 ` Nate Diller 2005-07-06 18:13 ` Hans Reiser 2005-07-06 2:55 ` Horst von Brand 2005-07-06 13:18 ` David Masover 2005-06-30 3:04 ` Hans Reiser 2005-06-30 4:33 ` Hubert Chan 2005-06-30 4:48 ` Hans Reiser 2005-06-30 6:29 ` David Weinehall 2005-06-30 15:57 ` Hubert Chan 2005-06-30 15:57 ` Hubert Chan 2005-06-30 17:10 ` David Weinehall 2005-06-30 18:47 ` Horst von Brand 2005-06-30 20:08 ` Kevin Bowen 2005-07-01 3:28 ` Horst von Brand 2005-07-01 6:56 ` David Masover 2005-07-01 20:26 ` Kevin Bowen 2005-07-02 2:12 ` Horst von Brand 2005-07-04 19:05 ` David Masover 2005-07-01 7:41 ` Chet Hosey 2005-07-05 20:55 ` Hubert Chan 2005-07-06 2:51 ` Horst von Brand 2005-07-06 13:26 ` David Masover 2005-07-06 20:10 ` Hubert Chan 2005-06-30 20:37 ` Zan Lynx 2005-07-01 7:16 ` David Masover 2005-06-30 19:00 ` Hubert Chan 2005-07-01 7:47 ` David Masover 2005-07-05 15:46 ` Martin Waitz 2005-07-05 22:32 ` Jonathan Briggs 2005-07-06 7:20 ` Martin Waitz 2005-07-06 9:02 ` Hans Reiser 2005-07-06 17:30 ` Horst von Brand 2005-07-07 6:41 ` Hans Reiser 2005-07-01 9:08 ` David Masover 2005-07-01 18:55 ` Jeremy Maitin-Shepard 2005-07-02 12:28 ` Pierre Etchemaïté 2005-07-02 23:09 ` David Masover 2005-06-28 20:07 ` David Masover 2005-06-28 20:22 ` David Masover 2005-06-29 1:51 ` Hubert Chan 2005-07-01 9:00 ` David Masover 2005-06-28 2:34 ` Lee Revell 2005-06-27 5:38 ` Horst von Brand 2005-06-27 6:18 ` David Masover 2005-06-27 4:23 ` Valdis.Kletnieks 2005-06-27 5:31 ` David Masover 2005-06-27 5:41 ` Valdis.Kletnieks 2005-06-27 5:57 ` David Masover 2005-06-27 6:12 ` Valdis.Kletnieks 2005-06-27 6:27 ` David Masover 2005-06-27 6:43 ` Valdis.Kletnieks 2005-06-27 7:00 ` David Masover 2005-06-27 7:00 ` David Masover 2005-06-27 13:47 ` David Weinehall 2005-06-27 15:08 ` David Masover 2005-06-27 16:40 ` Valdis.Kletnieks 2005-06-27 16:40 ` Valdis.Kletnieks 2005-06-27 18:25 ` David Masover 2005-06-27 18:25 ` David Masover 2005-06-28 6:47 ` Valdis.Kletnieks 2005-06-28 6:47 ` Valdis.Kletnieks 2005-06-27 4:27 ` Valdis.Kletnieks 2005-06-27 4:51 ` David Masover 2005-06-27 5:09 ` Hans Reiser 2005-06-27 6:02 ` David Masover 2005-06-27 2:49 ` David Masover 2005-06-27 3:10 ` Hubert Chan 2005-06-27 4:59 ` Valdis.Kletnieks 2005-06-27 5:54 ` David Masover 2005-06-27 6:24 ` Valdis.Kletnieks 2005-06-27 7:07 ` David Masover 2005-06-27 16:53 ` Valdis.Kletnieks 2005-06-27 7:24 ` Artem B. Bityuckiy 2005-06-27 14:58 ` Hubert Chan 2005-06-26 22:54 ` Hans Reiser 2005-06-27 0:59 ` Valdis.Kletnieks 2005-06-29 19:32 ` Thomas Rösner 2005-06-26 5:18 ` David Masover 2005-06-26 7:09 ` Lincoln Dale 2005-06-26 8:00 ` David Masover 2005-06-26 0:24 ` Hubert Chan 2005-06-26 2:46 ` David Masover 2005-06-26 3:14 ` David Masover 2005-06-26 4:32 ` Hans Reiser 2005-06-26 4:01 ` Horst von Brand 2005-06-26 4:53 ` David Masover 2005-06-26 5:45 ` Hubert Chan 2005-06-28 13:54 ` cryptocompress [was Re: reiser4 plugins] Pavel Machek 2005-06-23 21:41 ` reiser4 plugins Alan Cox 2005-06-24 1:23 ` reiser4 plugins (back to flames, oh well) Hans Reiser 2005-06-24 3:26 ` reiser4/VFS plugins David Masover 2005-06-22 5:36 ` reiser4 plugins Christoph Hellwig 2005-06-22 7:46 ` David Masover 2005-06-26 16:52 ` Christoph Hellwig 2005-06-26 17:21 ` David Masover 2005-06-26 17:28 ` Christoph Hellwig 2005-06-26 17:51 ` David Masover 2005-06-22 9:56 ` Stefan Smietanowski 2005-06-22 14:00 ` Rik Van Riel 2005-06-22 14:24 ` Alexander Zarochentsev 2005-06-22 14:29 ` Christoph Hellwig 2005-06-23 3:39 ` Hans Reiser 2005-06-26 16:46 ` Christoph Hellwig 2005-06-26 17:07 ` Artem B. Bityuckiy 2005-06-26 17:25 ` Christoph Hellwig 2005-06-26 18:14 ` randy_dunlap 2005-06-26 23:42 ` Hans Reiser 2005-06-27 8:57 ` Christoph Hellwig 2005-06-23 13:17 ` reiser4 plugins (the technical email in this flame fest) Hans Reiser 2005-06-23 17:31 ` reiser4 plugins Alexander Zarochentsev 2005-06-22 1:18 ` Andrew Morton 2005-06-22 14:56 ` Christophe Saout 2005-06-22 15:10 ` Artem B. Bityuckiy 2005-06-22 15:55 ` Markus Törnqvist 2005-06-22 15:55 ` mjt 2005-06-22 16:20 ` Artem B. Bityuckiy 2005-06-22 16:20 ` Artem B. Bityuckiy 2005-06-22 16:46 ` M. 2005-06-22 16:54 ` Markus Törnqvist 2005-06-22 16:54 ` mjt 2005-06-22 17:33 ` Horst von Brand 2005-06-22 17:33 ` Horst von Brand 2005-06-22 21:51 ` David Masover 2005-06-22 22:23 ` Nikita Danilov 2005-06-23 14:25 ` David Masover 2005-06-23 15:12 ` Hans Reiser 2005-06-23 22:31 ` Nikita Danilov 2005-06-24 6:37 ` Hans Reiser 2005-06-24 7:21 ` David Masover 2005-06-24 11:13 ` Bernd Eckenfels 2005-06-24 10:31 ` Nikita Danilov 2005-06-24 10:31 ` Nikita Danilov 2005-06-22 22:27 ` Roland Dreier 2005-06-23 7:44 ` Artem B. Bityuckiy 2005-06-23 8:08 ` Markus Törnqvist 2005-06-23 8:08 ` mjt 2005-06-23 19:00 ` Hans Reiser 2005-06-25 18:55 ` Alexander Zarochentsev 2005-06-22 5:34 ` Christoph Hellwig 2005-06-23 5:14 ` Hans Reiser 2005-06-21 20:22 ` -mm -> 2.6.13 merge status Andrew Morton 2005-06-21 20:56 ` Gerrit Huizenga 2005-06-21 21:04 ` Andrew Morton 2005-06-21 21:23 ` Gerrit Huizenga 2005-06-21 21:38 ` Arjan van de Ven 2005-06-22 6:33 ` Martin J. Bligh 2005-06-23 0:58 ` -mm -> 2.6.13 merge status (kexec/kdump) Vara Prasad 2005-06-23 1:08 ` Andrew Morton 2005-06-21 21:28 ` -mm -> 2.6.13 merge status Carsten Otte 2005-06-22 23:32 ` Chris Wright 2005-06-23 13:04 ` Carsten Otte 2005-06-22 16:53 ` Eric W. Biederman 2005-06-22 20:52 ` Andrew Morton 2005-06-23 2:14 ` Eric W. Biederman 2005-06-24 4:06 ` Paul Jackson 2005-06-24 4:54 ` randy_dunlap 2005-06-21 15:50 ` Lee Revell 2005-06-21 18:56 ` Nish Aravamudan 2005-06-22 18:00 ` Nish Aravamudan 2005-06-21 20:32 ` Andrew Morton 2005-06-21 20:37 ` Lee Revell 2005-06-21 16:26 ` -mm -> 2.6.13 merge status (HZ -> 250?) Lee Revell 2005-06-21 18:09 ` Alan Cox 2005-06-21 17:20 ` xtensa architecture (-mm -> 2.6.13 merge status) Chris Zankel 2005-06-22 3:35 ` OCFS (was Re: -mm " Rik Van Riel 2005-06-22 5:23 ` Wim Coekaerts 2005-06-22 4:08 ` -mm -> 2.6.13 merge status (configfs) David Teigland 2005-06-22 4:19 ` Andrew Morton 2005-06-22 5:51 ` -mm -> 2.6.13 merge status Christoph Hellwig 2005-06-27 9:06 ` Christoph Hellwig 2005-06-27 14:25 ` Vladimir Saveliev 2005-06-27 19:26 ` Christoph Hellwig 2005-06-27 19:44 ` Joel Becker 2005-06-27 20:26 ` Christoph Hellwig 2005-06-27 22:06 ` generic_drop_inode Mark Fasheh 2005-06-28 14:54 ` generic_drop_inode Christoph Hellwig 2005-06-27 19:30 ` -mm -> 2.6.13 merge status Christoph Hellwig 2005-06-27 20:37 ` Hans Reiser 2005-06-30 18:30 ` Vitaly Fertman 2005-06-27 20:19 ` Christoph Hellwig 2005-07-13 18:23 ` (v9fs) " Eric Van Hensbergen 2005-07-14 20:04 ` Christoph Hellwig 2005-07-14 22:12 ` Alexey Dobriyan 2005-07-14 22:15 ` Eric Van Hensbergen [not found] <20050620235458.5b437274.akpm@osdl.org.suse.lists.linux.kernel> 2005-06-21 11:14 ` Andi Kleen 2005-06-21 18:44 ` Hans Reiser 2005-06-21 19:56 ` Andi Kleen 2005-06-21 20:21 ` Christoph Hellwig 2005-06-22 1:38 ` Hans Reiser 2005-06-22 1:57 ` Jeff Garzik 2005-06-22 1:57 ` Andi Kleen 2005-06-22 2:55 ` Hans Reiser 2005-06-22 3:01 ` Jeff Garzik 2005-06-22 8:09 ` Hans Reiser 2005-06-22 8:24 ` Jeff Garzik 2005-06-23 6:22 ` Pekka Enberg 2005-06-23 7:42 ` Hans Reiser 2005-06-23 8:08 ` Pekka J Enberg 2005-06-23 13:10 ` Hans Reiser 2005-06-23 16:15 ` Vladimir Saveliev 2005-06-23 16:23 ` Olivier Galibert 2005-06-23 19:56 ` Ross Biro 2005-06-23 17:17 ` Hans Reiser 2005-06-23 21:18 ` Nikita Danilov 2005-06-23 18:37 ` Jeff Mahoney 2005-06-23 18:43 ` Andrew Morton 2005-06-23 19:29 ` Jeff Mahoney 2005-06-23 19:45 ` Hans Reiser 2005-06-23 19:32 ` Jens Axboe 2005-06-25 16:46 ` Pekka Enberg 2005-06-25 19:23 ` Hans Reiser 2005-06-25 21:08 ` Theodore Ts'o 2005-06-26 1:03 ` Hans Reiser 2005-06-25 23:54 ` Hubert Chan 2005-06-26 10:03 ` Nikita Danilov 2005-06-27 7:24 ` Jens Axboe 2005-06-27 7:28 ` Andi Kleen 2005-06-27 7:49 ` Pekka J Enberg 2005-06-27 8:19 ` Jörn Engel 2005-06-27 8:20 ` Andi Kleen 2005-06-23 19:24 ` Hans Reiser 2005-06-24 1:13 ` Hubert Chan 2005-06-24 1:13 ` Hubert Chan 2005-06-26 0:45 ` Christian Trefzer 2005-06-26 1:13 ` Hans Reiser 2005-06-26 2:23 ` Christian Trefzer [not found] <4hNoW-1yo-37@gated-at.bofh.it> [not found] ` <4hT1h-5V0-41@gated-at.bofh.it> [not found] ` <4hXHv-1br-41@gated-at.bofh.it> 2005-06-22 14:40 ` Bodo Eggert 2005-06-23 13:20 Ian Pratt 2005-06-23 13:37 ` Mark Williamson
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.