* RFC: Starting a stable kernel series off the 2.6 kernel @ 2005-12-03 13:56 Adrian Bunk 2005-12-03 14:29 ` Jesper Juhl ` (7 more replies) 0 siblings, 8 replies; 639+ messages in thread From: Adrian Bunk @ 2005-12-03 13:56 UTC (permalink / raw) To: linux-kernel The current kernel development model is pretty good for people who always want to use or offer their costumers the maximum amount of the latest bugs^Wfeatures without having to resort on additional patches for them. Problems of the current development model from a user's point of view are: - many regressions in every new release - kernel updates often require updates for the kernel-related userspace (e.g. for udev or the pcmcia tools switch) One problem following from this is that people continue to use older kernels with known security holes because the amount of work for kernel upgrades is too high. These problems follow from the development model. The latest stable kernel series without these problems is 2.4, but 2.4 is becoming more and more obsolete and might e.g. lack driver support for some recent hardware you want to use. Since Andrew and Linus do AFAIK not plan to change the development model, what about the following for getting a stable kernel series without leaving the current development model: Kernel 2.6.16 will be the base for a stable series. After 2.6.16, there will be a 2.6.16.y series with the usual stable rules. After the release of 2.6.17, this 2.6.16.y series will be continued with more relaxed rules similar to the rules in kernel 2.4 since the release of kernel 2.6.0 (e.g. driver updates will be allowed). Q: What is the target audience for this 2.6.16 series? A: The target audience are users still using 2.4 (or who'd still use kernel 2.4 if they weren't forced to upgrade to 2.6 for some reason) who want a stable kernel series including security fixes but excluding many regressions. It might also be interesting for distributions that prefer stability over always using the latest stuff. Q: Does this proposal imply anything for the development between 2.6.15 and 2.6.16? A: In theory not. In practice, it would be a big advantage if some of the bigger changes that might go into 2.6.16 would be postponed to 2.6.17. Q: Why not start with the more relaxed rules before the release of 2.6.17? A: After 2.6.16.y following the usual stable rules, the kernel should be relatively stable and well-tested giving the best possible basis for a long-living series. Q: How long should this 2.6.16 series be maintained? A: Time will tell, but if people use it I'd expect 2 or 3 years. Q: Stable API/ABI for external modules? A: No. Q: Who will maintain this branch? A: I could do it, but if someone more experienced wants to do it that would be even better. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 13:56 RFC: Starting a stable kernel series off the 2.6 kernel Adrian Bunk @ 2005-12-03 14:29 ` Jesper Juhl 2005-12-03 20:19 ` Greg KH 2005-12-03 14:31 ` Ben Collins ` (6 subsequent siblings) 7 siblings, 1 reply; 639+ messages in thread From: Jesper Juhl @ 2005-12-03 14:29 UTC (permalink / raw) To: Adrian Bunk; +Cc: linux-kernel On 12/3/05, Adrian Bunk <bunk@stusta.de> wrote: > The current kernel development model is pretty good for people who > always want to use or offer their costumers the maximum amount of the > latest bugs^Wfeatures without having to resort on additional patches for > them. > > Problems of the current development model from a user's point of view > are: > - many regressions in every new release > - kernel updates often require updates for the kernel-related userspace > (e.g. for udev or the pcmcia tools switch) > > One problem following from this is that people continue to use older > kernels with known security holes because the amount of work for kernel > upgrades is too high. > > These problems follow from the development model. > > The latest stable kernel series without these problems is 2.4, but 2.4 > is becoming more and more obsolete and might e.g. lack driver support > for some recent hardware you want to use. > > Since Andrew and Linus do AFAIK not plan to change the development > model, what about the following for getting a stable kernel series > without leaving the current development model: > > > Kernel 2.6.16 will be the base for a stable series. > [snip] Why can't this be done by distributors/vendors? Any vendor is free to branch off at 2.6.<whatever> and then maintain that indefinately. Why create yet-another-stable-branch? In effect you'd be making 2.6.17+ into a 2.7.x tree and 2.6.16.y would become a 2.6 tree in 2.4.x style, with all the backporting problems and vendor skew that 2.4.x suffered from. Personally I don't like this proposal. In my oppinion 2.6 + the -stable branch as we have it now works well and people who want userspace & kernel in sync are perfectly free to use vendor kernels (which is also the recommended thing to do for most users as far as I know). Just my 0.02euro. -- Jesper Juhl <jesper.juhl@gmail.com> Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html Plain text mails only, please http://www.expita.com/nomime.html ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 14:29 ` Jesper Juhl @ 2005-12-03 20:19 ` Greg KH 2005-12-03 21:04 ` M. ` (5 more replies) 0 siblings, 6 replies; 639+ messages in thread From: Greg KH @ 2005-12-03 20:19 UTC (permalink / raw) To: Jesper Juhl; +Cc: Adrian Bunk, linux-kernel On Sat, Dec 03, 2005 at 03:29:54PM +0100, Jesper Juhl wrote: > > Why can't this be done by distributors/vendors? It already is done by these people, look at the "enterprise" Linux distributions and their 5 years of maintance (or whatever the number is.) If people/customers want stability, they already have this option. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 20:19 ` Greg KH @ 2005-12-03 21:04 ` M. 2005-12-03 21:37 ` James Courtier-Dutton [not found] ` <f0cc38560512031254j3b28d579s539be721c247c10a@mail.gmail.com> ` (4 subsequent siblings) 5 siblings, 1 reply; 639+ messages in thread From: M. @ 2005-12-03 21:04 UTC (permalink / raw) To: linux-kernel; +Cc: Jesper Juhl, Adrian Bunk On 12/3/05, Greg KH <greg@kroah.com> wrote: > On Sat, Dec 03, 2005 at 03:29:54PM +0100, Jesper Juhl wrote: > > > > Why can't this be done by distributors/vendors? > > It already is done by these people, look at the "enterprise" Linux > distributions and their 5 years of maintance (or whatever the number > is.) > > If people/customers want stability, they already have this option. > > thanks, > > greg k-h > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > Yes but not home users with relatively new/bleeding edge hardware or small projects writing for example a wifi driver or a security patch or whatever without full time commitment to tracking kernel changes. Enterprise products are suited for production servers, school/government/companies desktops and not for "enthusiasts" or for small kernel projects (they obviously cant write drivers or patches for custom distro kernels). Those enthusiasts have to get mad with performance regressions, new incompatibilities, new crashes etc. My suggestion was to release a 2.6.X kernel every 6months reducing kernel development fragmentation between different distros and giving away stabler stuff. Michele ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 21:04 ` M. @ 2005-12-03 21:37 ` James Courtier-Dutton 0 siblings, 0 replies; 639+ messages in thread From: James Courtier-Dutton @ 2005-12-03 21:37 UTC (permalink / raw) To: M.; +Cc: linux-kernel, Jesper Juhl, Adrian Bunk M. wrote: > > Yes but not home users with relatively new/bleeding edge hardware or > small projects writing for example a wifi driver or a security patch > or whatever without full time commitment to tracking kernel changes. > If there are "small projects writing" their own wifi driver, they should try to get it included in the kernel ASAP. Then they won't have to track the changes, as the person making the changes will automatically change their little driver to keep it working after the changes. Drivers very rarely impact the stability of the rest of the kernel. It initially gets added as "EXPERIMENTAL" so the user can choose whether to even use it or not. All it takes is for the "small project" to build their own git tree, and then ask the Linus or Andrew to pull it. It should get added pretty easily, so long as the code looks pretty. :-) It is really that simple. There is no logical reason for any external driver not to be added into the main kernel, so why do people not want to add them? James ^ permalink raw reply [flat|nested] 639+ messages in thread
[parent not found: <f0cc38560512031254j3b28d579s539be721c247c10a@mail.gmail.com>]
* Re: RFC: Starting a stable kernel series off the 2.6 kernel [not found] ` <f0cc38560512031254j3b28d579s539be721c247c10a@mail.gmail.com> @ 2005-12-03 21:12 ` Greg KH 2005-12-03 21:31 ` M. 2005-12-06 1:19 ` Florian Weimer 0 siblings, 2 replies; 639+ messages in thread From: Greg KH @ 2005-12-03 21:12 UTC (permalink / raw) To: M., linux-kernel <dragging the converstation back to lkml, where it belongs...> On Sat, Dec 03, 2005 at 09:54:35PM +0100, M. wrote: > On 12/3/05, Greg KH <greg@kroah.com> wrote: > > On Sat, Dec 03, 2005 at 03:29:54PM +0100, Jesper Juhl wrote: > > > > > > Why can't this be done by distributors/vendors? > > > > It already is done by these people, look at the "enterprise" Linux > > distributions and their 5 years of maintance (or whatever the number > > is.) > > > > If people/customers want stability, they already have this option. > > > > thanks, > > > > greg k-h > > Yes but not home users with relatively new/bleeding edge hardware or > small projects writing for example a wifi driver or a security patch > or whatever without full time commitment to tracking kernel changes. If you are a user that wants this kind of support, then use a distro that can handle this. Obvious examples that come to mind are both Debian and Gentoo and Fedora and OpenSuSE, and I'm sure there are others. But if you are a developer, you can usually stay up to date by tracking the main releases, which should be about once a month. If you have problems porting your stuff to the latest kernel when you need to submit it for inclusion, there are lots of people to help point you in the proper direction for what is needed to be done. > Enterprise products are suited for production servers, > school/government/companies desktops and not for "enthusiasts" or for > small kernel projects (they obviously cant write drivers or patches > for custom distro kernels). Those enthusiasts have to get mad with > performance regressions, new incompatibilities, new crashes etc. Sure, then use a different distro for them. That's why Linux has so many different ones, they all are targeted at different users. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 21:12 ` Greg KH @ 2005-12-03 21:31 ` M. 2005-12-03 21:38 ` Arjan van de Ven 2005-12-03 21:54 ` Greg KH 2005-12-06 1:19 ` Florian Weimer 1 sibling, 2 replies; 639+ messages in thread From: M. @ 2005-12-03 21:31 UTC (permalink / raw) To: Greg KH; +Cc: linux-kernel On 12/3/05, Greg KH <greg@kroah.com> wrote: > <dragging the converstation back to lkml, where it belongs...> > > On Sat, Dec 03, 2005 at 09:54:35PM +0100, M. wrote: > > On 12/3/05, Greg KH <greg@kroah.com> wrote: > > > On Sat, Dec 03, 2005 at 03:29:54PM +0100, Jesper Juhl wrote: > > > > > > > > Why can't this be done by distributors/vendors? > > > > > > It already is done by these people, look at the "enterprise" Linux > > > distributions and their 5 years of maintance (or whatever the number > > > is.) > > > > > > If people/customers want stability, they already have this option. > > > > > > thanks, > > > > > > greg k-h > > > > Yes but not home users with relatively new/bleeding edge hardware or > > small projects writing for example a wifi driver or a security patch > > or whatever without full time commitment to tracking kernel changes. > > If you are a user that wants this kind of support, then use a distro > that can handle this. Obvious examples that come to mind are both > Debian and Gentoo and Fedora and OpenSuSE, and I'm sure there are > others. > > But if you are a developer, you can usually stay up to date by tracking > the main releases, which should be about once a month. If you have > problems porting your stuff to the latest kernel when you need to submit > it for inclusion, there are lots of people to help point you in the > proper direction for what is needed to be done. > > > Enterprise products are suited for production servers, > > school/government/companies desktops and not for "enthusiasts" or for > > small kernel projects (they obviously cant write drivers or patches > > for custom distro kernels). Those enthusiasts have to get mad with > > performance regressions, new incompatibilities, new crashes etc. > > Sure, then use a different distro for them. That's why Linux has so > many different ones, they all are targeted at different users. > > thanks, > > greg k-h > <sorry for the direct reply> makes sense, but are you sure having distros like Debian, enterprise products from redhat etc using the same 6months release for their stable versions does not translate in minor fragmentation on kernel development and in benefits for every user? Under this light i think a 6months cycle starts to mean something when stable distros gets older and older (debian and redhat enterprise are released every 18/24 months) and developers and who wants bleeding edge features can always use Fedora, openSUSE, Gentoo etc. Another advantage would be to benefit external projects and hardware producers writing open drivers, enlowering the effort in writing and mantaining a driver. Michele ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 21:31 ` M. @ 2005-12-03 21:38 ` Arjan van de Ven 2005-12-03 21:53 ` M. 2005-12-03 21:54 ` Greg KH 1 sibling, 1 reply; 639+ messages in thread From: Arjan van de Ven @ 2005-12-03 21:38 UTC (permalink / raw) To: M.; +Cc: Greg KH, linux-kernel > <sorry for the direct reply> > > makes sense, but are you sure having distros like Debian, enterprise > products from redhat etc using the same 6months release for their > stable versions does not translate in minor fragmentation on kernel > development I'm quite sure that there isn't significant fragmentation; all those distros in their maintenance generally only take patches that are already upstream (or they send them upstream during the maintenance) just to make sure that their long term costs don't go insane (eg for the $nextversion, the distros can just start clean because they know all bugfixes from maintenance versions are already in the new kernel.org kernel they get; not doing that is REALLY expensive so distros like to avoid that) > and in benefits for every user? you can't have it both ways; you can't be "new" and "old stable" at the same time. > . Another > advantage would be to benefit external projects and hardware producers > writing open drivers, enlowering the effort in writing and mantaining > a driver. there is an even better model for those: Get it merged into kernel.org! There is an even bigger deal here: even if you're not ready to get merged yet, staying on the same old version for 6 months is NOT going to help you. In fact it's worse: it is 10x easier to deal with 6 small steps of 1 month than to deal with 1 big step of 6 months. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 21:38 ` Arjan van de Ven @ 2005-12-03 21:53 ` M. 2005-12-03 22:26 ` Greg KH 2005-12-04 7:56 ` Arjan van de Ven 0 siblings, 2 replies; 639+ messages in thread From: M. @ 2005-12-03 21:53 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Greg KH, linux-kernel On 12/3/05, Arjan van de Ven <arjan@infradead.org> wrote: > > > <sorry for the direct reply> > > > > makes sense, but are you sure having distros like Debian, enterprise > > products from redhat etc using the same 6months release for their > > stable versions does not translate in minor fragmentation on kernel > > development > > I'm quite sure that there isn't significant fragmentation; all those > distros in their maintenance generally only take patches that are > already upstream (or they send them upstream during the maintenance) > just to make sure that their long term costs don't go insane > (eg for the $nextversion, the distros can just start clean because they > know all bugfixes from maintenance versions are already in the new > kernel.org kernel they get; not doing that is REALLY expensive so > distros like to avoid that) > > > > and in benefits for every user? > > you can't have it both ways; you can't be "new" and "old stable" at the > same time. > > > . Another > > advantage would be to benefit external projects and hardware producers > > writing open drivers, enlowering the effort in writing and mantaining > > a driver. > > there is an even better model for those: Get it merged into kernel.org! > > > There is an even bigger deal here: even if you're not ready to get > merged yet, staying on the same old version for 6 months is NOT going to > help you. In fact it's worse: it is 10x easier to deal with 6 small > steps of 1 month than to deal with 1 big step of 6 months. > > from the kernel.org point of view it does make sense but from users pov i think no. Users stuck with old drivers not actively mantained would benefit from this. There are some open drivers wrote by hardware mantainers which will never get into kernel.org cause of code not following kernel style guides and so on. Yeah, you should not buy poorly supported hardware and use bad drivers but a lot of new users have poorly supported hardware and a "more stable than usual and at fixed dates" release could enlower the skills barrier in approaching linux. Michele ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 21:53 ` M. @ 2005-12-03 22:26 ` Greg KH 2005-12-04 7:56 ` Arjan van de Ven 1 sibling, 0 replies; 639+ messages in thread From: Greg KH @ 2005-12-03 22:26 UTC (permalink / raw) To: M.; +Cc: Arjan van de Ven, linux-kernel On Sat, Dec 03, 2005 at 10:53:18PM +0100, M. wrote: > from the kernel.org point of view it does make sense but from users > pov i think no. Users stuck with old drivers not actively mantained > would benefit from this. Any specific examples of this? > There are some open drivers wrote by hardware mantainers which will > never get into kernel.org cause of code not following kernel style > guides and so on. Yeah, you should not buy poorly supported hardware > and use bad drivers but a lot of new users have poorly supported > hardware and a "more stable than usual and at fixed dates" release > could enlower the skills barrier in approaching linux. The skills barrier has _nothing_ to do with the release cycles. And if you want to get a driver into the main tree, that isn't being maintained, just get a piece of the hardware to a kernel developer, along with the driver and it will be maintained. I know I've made that offer many times in the past, and so has others. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 21:53 ` M. 2005-12-03 22:26 ` Greg KH @ 2005-12-04 7:56 ` Arjan van de Ven [not found] ` <f0cc38560512040657i58cc08efqa8596c357fcea82e@mail.gmail.com> 1 sibling, 1 reply; 639+ messages in thread From: Arjan van de Ven @ 2005-12-04 7:56 UTC (permalink / raw) To: M.; +Cc: Greg KH, linux-kernel > > > > > . Another > > > advantage would be to benefit external projects and hardware producers > > > writing open drivers, enlowering the effort in writing and mantaining > > > a driver. > > > > there is an even better model for those: Get it merged into kernel.org! > > > > > > There is an even bigger deal here: even if you're not ready to get > > merged yet, staying on the same old version for 6 months is NOT going to > > help you. In fact it's worse: it is 10x easier to deal with 6 small > > steps of 1 month than to deal with 1 big step of 6 months. > > > > > from the kernel.org point of view it does make sense but from users > pov i think no. Users stuck with old drivers not actively mantained > would benefit from this. actually no. since that only buys them a few months at most, and then you're entirely stuck again. And patching it up after 6 months is a lot harder than doing smaller steps of 1 month, especially for less experienced people. > There are some open drivers wrote by hardware mantainers which will > never get into kernel.org cause of code not following kernel style > guides and so on. which ones did you have in mind? ^ permalink raw reply [flat|nested] 639+ messages in thread
[parent not found: <f0cc38560512040657i58cc08efqa8596c357fcea82e@mail.gmail.com>]
* Re: RFC: Starting a stable kernel series off the 2.6 kernel [not found] ` <f0cc38560512040657i58cc08efqa8596c357fcea82e@mail.gmail.com> @ 2005-12-04 15:10 ` Arjan van de Ven 2005-12-04 16:11 ` Matthias Andree ` (2 more replies) 0 siblings, 3 replies; 639+ messages in thread From: Arjan van de Ven @ 2005-12-04 15:10 UTC (permalink / raw) To: M.; +Cc: Greg KH, linux-kernel On Sun, 2005-12-04 at 15:57 +0100, M. wrote: > > if distros would align on those 6months versions those less > experienced users would get 5 years support on those kernels. no distro gives 5 years of support for a kernel done every 6 months; they start such projects more like every 18 to 24 months (SuSE used to do it a bit more frequently but it seems they also slowed this down). > example: redhat, suse and mandriva are releasing their new product > using the latest 6months (or whatever) kernel; they are not going to > patch it except for new filesystems or bugfixes because of the new dev "except for" is a slipperly slope. And "except for bugfixes" would be wrong... those would be the ones that need to be in the kernel.org kernel. As well as new hardware support. At which point.. what is the difference? Where do 'features' stop and where do 'only needed bugfixes' begin? > model granting them all the needed new features; then, they start to > mantain this kernel for their customers (and they could do it in a > collaborative way, thus mantaining the kernel.org kernel plus their > separate patches) and every user of redhat, suse, mandriva and the > kernel.org 6months kernel they are using would benefit from this and > would get 5 years support on this kernel. that's not practical though. And it's still no better from the regression point of view; those enterprise kernels undergo quite a bit of churn as well, but just very directed churn to the point that I doubt it would satisfy your target audience.... > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 15:10 ` Arjan van de Ven @ 2005-12-04 16:11 ` Matthias Andree 2005-12-04 16:41 ` Arjan van de Ven [not found] ` <f0cc38560512040724re5114c2y76bb34d63c9c5ae0@mail.gmail.com> 2005-12-05 19:35 ` Bill Davidsen 2 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2005-12-04 16:11 UTC (permalink / raw) To: linux-kernel On Sun, 04 Dec 2005, Arjan van de Ven wrote: > On Sun, 2005-12-04 at 15:57 +0100, M. wrote: > > > > > if distros would align on those 6months versions those less > > experienced users would get 5 years support on those kernels. > > no distro gives 5 years of support for a kernel done every 6 months; > they start such projects more like every 18 to 24 months (SuSE used to > do it a bit more frequently but it seems they also slowed this down). SUSE end-user distros (SUSE LINUX <version>) are released every 6 months or so, and are supported for 24 months. Their "enterprise server" is supported for 60 months though, SLES 9 forked off 9.1. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 16:11 ` Matthias Andree @ 2005-12-04 16:41 ` Arjan van de Ven 2005-12-04 20:08 ` Paul Jackson 0 siblings, 1 reply; 639+ messages in thread From: Arjan van de Ven @ 2005-12-04 16:41 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel On Sun, 2005-12-04 at 17:11 +0100, Matthias Andree wrote: > On Sun, 04 Dec 2005, Arjan van de Ven wrote: > > > On Sun, 2005-12-04 at 15:57 +0100, M. wrote: > > > > > > > > if distros would align on those 6months versions those less > > > experienced users would get 5 years support on those kernels. > > > > no distro gives 5 years of support for a kernel done every 6 months; > > they start such projects more like every 18 to 24 months (SuSE used to > > do it a bit more frequently but it seems they also slowed this down). > > SUSE end-user distros (SUSE LINUX <version>) are released every 6 months > or so, and are supported for 24 months. Their "enterprise server" is > supported for 60 months though, SLES 9 forked off 9.1. sure.. but they don't add new hw support really, and I'd not be surprised if they rebase to a newer upstream kernel after a while. I know we did that for RHL, eg RHL 7.(Y-1) got the kernel of RHL7.Y after RHL7.Y was released, not only to keep the maintenance down, but more so to get all the bugfixes and hardware support out to customers. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 16:41 ` Arjan van de Ven @ 2005-12-04 20:08 ` Paul Jackson 0 siblings, 0 replies; 639+ messages in thread From: Paul Jackson @ 2005-12-04 20:08 UTC (permalink / raw) To: Arjan van de Ven; +Cc: matthias.andree, linux-kernel Arjan, responding to Matthias > > SUSE end-user distros (SUSE LINUX <version>) are released every 6 months > > or so, and are supported for 24 months. Their "enterprise server" is > > supported for 60 months though, SLES 9 forked off 9.1. > > sure.. but they don't add new hw support really, and I'd not be > surprised if they rebase to a newer upstream kernel after a while. They will start a new enterprise release, off a new base, every so often, and call the next one SLES 10. But SLES 9 will continue to be supported, off its current kernel.org base, for an extended period of time. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <pj@sgi.com> 1.925.600.0401 ^ permalink raw reply [flat|nested] 639+ messages in thread
[parent not found: <f0cc38560512040724re5114c2y76bb34d63c9c5ae0@mail.gmail.com>]
* Re: RFC: Starting a stable kernel series off the 2.6 kernel [not found] ` <f0cc38560512040724re5114c2y76bb34d63c9c5ae0@mail.gmail.com> @ 2005-12-04 22:47 ` Greg KH [not found] ` <f0cc38560512041503y7abd1f12rbce8bdac0ebdf30d@mail.gmail.com> 2005-12-05 1:03 ` Horst von Brand 0 siblings, 2 replies; 639+ messages in thread From: Greg KH @ 2005-12-04 22:47 UTC (permalink / raw) To: M.; +Cc: Arjan van de Ven, linux-kernel On Sun, Dec 04, 2005 at 04:24:36PM +0100, M. wrote: > On 12/4/05, Arjan van de Ven <arjan@infradead.org> wrote: > > On Sun, 2005-12-04 at 15:57 +0100, M. wrote: > > > if distros would align on those 6months versions those less > > > experienced users would get 5 years support on those kernels. > > > > no distro gives 5 years of support for a kernel done every 6 months; > > they start such projects more like every 18 to 24 months (SuSE used to > > do it a bit more frequently but it seems they also slowed this down). > > > yeah but I would mean if there's a 6months release cycle like GNOME & co. > there would be more opportunities in different distros using the same kernel > like those distros do with GNOME & co. If they use the same 'current' > 6months kernel available in the 18/24 time window this will lead to unified > base kernel for every distro and those distro could mantain it for years The kernel is unlike GNOME in so many different ways, there's just no way to compare their development cycles. People remember, the kernel evolves organically. We don't know what's going to be in the next 2 kernel releases just because we don't know what's going to show up, and what hardware is going to be released, and what kind of problems people are going to have, and what kind of proposed patches are going to work out. The way the kernel is developed is _so_ different from any traditional software development process is defined. So for people to try to put traditional requirements on the kernel (6 month cycles, etc.) is just not realistic. And please for everyone wanting to go with a stable series like is being proposed, go read the thread a while ago on this list that caused the creation of the -stable tree. In it lots of people who know what they are talking about discuss the difficulties of doing a "bug fix only" tree, and other such things. Out of that discussion came the very restrictive guidelines that are described in Documentation/stable_kernel_rules.txt. To try to do more than what is defined there, without lots of money and man-power behind you, is a quick trip to madness... Best of luck to you all, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
[parent not found: <f0cc38560512041503y7abd1f12rbce8bdac0ebdf30d@mail.gmail.com>]
* Re: RFC: Starting a stable kernel series off the 2.6 kernel [not found] ` <f0cc38560512041503y7abd1f12rbce8bdac0ebdf30d@mail.gmail.com> @ 2005-12-04 23:12 ` Greg KH 0 siblings, 0 replies; 639+ messages in thread From: Greg KH @ 2005-12-04 23:12 UTC (permalink / raw) To: M.; +Cc: Arjan van de Ven, linux-kernel On Mon, Dec 05, 2005 at 12:03:20AM +0100, M. wrote: > > The way the kernel is developed is _so_ different from any traditional > > software development process is defined. So for people to try to put > > traditional requirements on the kernel (6 month cycles, etc.) is just > > not realistic. > > Mhhh BSDs and MacOSX kernel are developed without taking "the unknown" into > account: planned releases and bla blah and they dont miss features. Yeah > linux is faster, but there could be a middle point between strict release > cycles and "ok let's put it in cause it's going to make something run > faster" MacOSX is developed this way? I think you will find a lot of Apple engineers disagree with you... And BSD is also quite different than Linux in many different ways, the development community being one of these differences. And that one difference makes a lot of difference. > > And please for everyone wanting to go with a stable series like is being > > proposed, go read the thread a while ago on this list that caused the > > creation of the -stable tree. In it lots of people who know what they > > are talking about discuss the difficulties of doing a "bug fix only" > > tree, and other such things. Out of that discussion came the very > > restrictive guidelines that are described in > > Documentation/stable_kernel_rules.txt. To try to do more than what is > > defined there, without lots of money and man-power behind you, is a > > quick trip to madness... > > > The proposal was in fact to come out with a 2.6.X release every 6 months > trying to align every distro on it and to "get" their man-power and money as > a side effect. Maybe i'm not sufficiently good with english to let you all > understand clearly but the proposal was to do 2/3/4 releases the > normal/current style, even adding new and previously unknown > features/patches/whatever and to do the last release before the next > 2.6.Xwith only bugfix and stabilization in mind; If you think over it > it's the > same approach of every release: > > - patches get in until -rc; > - 2 weeks bugfix only; > - release; > > but applied to a 6months timeline. That will not work. Again, go back and read that thread. We seeing this shorter development cycle get backed up even when it streaches to 2 months. For it to go to 6 months would be madness. If you don't understand why I say this, please go look at the different developer trees that start accumilating patches during this "bugfix only" timeframe. > It's not a -stable/strict/specifics-based tree but a way to align > everyone to the same kernel version and to get stabilization and > maintenance as a side effect. And you believe that the enterprise distros will some how flock to this kernel release instead? Why would they? What would it gain them? thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 22:47 ` Greg KH [not found] ` <f0cc38560512041503y7abd1f12rbce8bdac0ebdf30d@mail.gmail.com> @ 2005-12-05 1:03 ` Horst von Brand 1 sibling, 0 replies; 639+ messages in thread From: Horst von Brand @ 2005-12-05 1:03 UTC (permalink / raw) To: Greg KH; +Cc: M., Arjan van de Ven, linux-kernel Greg KH <greg@kroah.com> wrote: > On Sun, Dec 04, 2005 at 04:24:36PM +0100, M. wrote: [...] > > yeah but I would mean if there's a 6months release cycle like GNOME & co. > > there would be more opportunities in different distros using the same > > kernel like those distros do with GNOME & co. If they use the same > > 'current' 6months kernel available in the 18/24 time window this will > > lead to unified base kernel for every distro and those distro could > > mantain it for years > The kernel is unlike GNOME in so many different ways, there's just no > way to compare their development cycles. Gnome is a /collection/ of (mostly independent) programs, after a while what program+version survives a stress test is decreed to be part of version N + 1 to be released at the 6-month point; lather, rinse, repeat. In that sense it is much more like a distribution (which also have similar release cycles). The kernel is /one/ program, and large and complex to boot. > People remember, the kernel evolves organically. We don't know what's > going to be in the next 2 kernel releases just because we don't know > what's going to show up, and what hardware is going to be released, and > what kind of problems people are going to have, and what kind of > proposed patches are going to work out. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 15:10 ` Arjan van de Ven 2005-12-04 16:11 ` Matthias Andree [not found] ` <f0cc38560512040724re5114c2y76bb34d63c9c5ae0@mail.gmail.com> @ 2005-12-05 19:35 ` Bill Davidsen 2 siblings, 0 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-05 19:35 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Greg KH, linux-kernel Arjan van de Ven wrote: > On Sun, 2005-12-04 at 15:57 +0100, M. wrote: > > >>if distros would align on those 6months versions those less >>experienced users would get 5 years support on those kernels. > > > no distro gives 5 years of support for a kernel done every 6 months; > they start such projects more like every 18 to 24 months (SuSE used to > do it a bit more frequently but it seems they also slowed this down). > > >>example: redhat, suse and mandriva are releasing their new product >>using the latest 6months (or whatever) kernel; they are not going to >>patch it except for new filesystems or bugfixes because of the new dev > > > "except for" is a slipperly slope. And "except for bugfixes" would be > wrong... those would be the ones that need to be in the kernel.org > kernel. As well as new hardware support. At which point.. what is the > difference? Where do 'features' stop and where do 'only needed bugfixes' > begin? Given the examples of 2.2 and 2.4 ongoing low level maintenence, I think that's a poor objection, a stable series (in the old sense) needs one maintainer to make the decisions on what goings in, and typically people will do the actualy work cooperating with the primary maintainer. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 21:31 ` M. 2005-12-03 21:38 ` Arjan van de Ven @ 2005-12-03 21:54 ` Greg KH 1 sibling, 0 replies; 639+ messages in thread From: Greg KH @ 2005-12-03 21:54 UTC (permalink / raw) To: M.; +Cc: linux-kernel On Sat, Dec 03, 2005 at 10:31:03PM +0100, M. wrote: > makes sense, but are you sure having distros like Debian, enterprise > products from redhat etc using the same 6months release for their > stable versions does not translate in minor fragmentation on kernel > development and in benefits for every user? It hasn't so far from my viewpoint. Do you think it has? > Under this light i think a 6months cycle starts to mean something when > stable distros gets older and older (debian and redhat enterprise are > released every 18/24 months) and developers and who wants bleeding > edge features can always use Fedora, openSUSE, Gentoo etc. Another > advantage would be to benefit external projects and hardware producers > writing open drivers, enlowering the effort in writing and mantaining > a driver. Drivers belong in the main kernel.org kernel tree. See Documentation/stable-api-nonsense.txt for why this is so, and Documentation/HOWTO for how to get this accomplished. It is explained in great detail there. If you have further questions about this, please let us know. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 21:12 ` Greg KH 2005-12-03 21:31 ` M. @ 2005-12-06 1:19 ` Florian Weimer 2005-12-06 17:55 ` Greg KH 1 sibling, 1 reply; 639+ messages in thread From: Florian Weimer @ 2005-12-06 1:19 UTC (permalink / raw) To: Greg KH; +Cc: M., linux-kernel * Greg KH: >> Yes but not home users with relatively new/bleeding edge hardware or >> small projects writing for example a wifi driver or a security patch >> or whatever without full time commitment to tracking kernel changes. > > If you are a user that wants this kind of support, then use a distro > that can handle this. Obvious examples that come to mind are both > Debian and Gentoo and Fedora and OpenSuSE, and I'm sure there are > others. IIRC, Gentoo ignores some kinds of security bugs so that the task remains manageable. Debian, in contrast, hasn't released a kernel update for its stable distribution since June (but unstable and even testing is in surprisingly good shape). Maybe the real vendor kernels are better. Without CVE-based bug tracking on their part, it is hard to tell, though. 8-) ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 1:19 ` Florian Weimer @ 2005-12-06 17:55 ` Greg KH 0 siblings, 0 replies; 639+ messages in thread From: Greg KH @ 2005-12-06 17:55 UTC (permalink / raw) To: Florian Weimer; +Cc: M., linux-kernel On Tue, Dec 06, 2005 at 02:19:29AM +0100, Florian Weimer wrote: > * Greg KH: > > >> Yes but not home users with relatively new/bleeding edge hardware or > >> small projects writing for example a wifi driver or a security patch > >> or whatever without full time commitment to tracking kernel changes. > > > > If you are a user that wants this kind of support, then use a distro > > that can handle this. Obvious examples that come to mind are both > > Debian and Gentoo and Fedora and OpenSuSE, and I'm sure there are > > others. > > IIRC, Gentoo ignores some kinds of security bugs so that the task > remains manageable. Do you have specific details about this? I know the Gentoo kernel team is currently revamping the way they handle their security updates, as it is known to need some work. I'm sure they would be glad for input into this process. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 20:19 ` Greg KH 2005-12-03 21:04 ` M. [not found] ` <f0cc38560512031254j3b28d579s539be721c247c10a@mail.gmail.com> @ 2005-12-03 22:51 ` Adrian Bunk 2005-12-03 23:28 ` Greg KH ` (3 more replies) 2005-12-04 3:48 ` Jesper Juhl ` (2 subsequent siblings) 5 siblings, 4 replies; 639+ messages in thread From: Adrian Bunk @ 2005-12-03 22:51 UTC (permalink / raw) To: Greg KH; +Cc: Jesper Juhl, linux-kernel On Sat, Dec 03, 2005 at 12:19:45PM -0800, Greg KH wrote: > On Sat, Dec 03, 2005 at 03:29:54PM +0100, Jesper Juhl wrote: > > > > Why can't this be done by distributors/vendors? > > It already is done by these people, look at the "enterprise" Linux > distributions and their 5 years of maintance (or whatever the number > is.) > > If people/customers want stability, they already have this option. I don't get the point where the advantage is when every distribution creates it's own stable branches. AFAIR one of the reasons for the current 2.6 development model was to reduce the amount of feature patches distributions ship by offering an ftp.kernel.org kernel that gets new features early. What's wrong with offering an unified branch with few regressions for both users and distributions? It's not that every distribution will use it, but as soon as one or two distributions are using it the amount of extra work for maintaining the branch should become pretty low. > thanks, > > greg k-h cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:51 ` Adrian Bunk @ 2005-12-03 23:28 ` Greg KH 2005-12-03 23:35 ` Chris Wright ` (2 subsequent siblings) 3 siblings, 0 replies; 639+ messages in thread From: Greg KH @ 2005-12-03 23:28 UTC (permalink / raw) To: Adrian Bunk; +Cc: Jesper Juhl, linux-kernel On Sat, Dec 03, 2005 at 11:51:05PM +0100, Adrian Bunk wrote: > On Sat, Dec 03, 2005 at 12:19:45PM -0800, Greg KH wrote: > > On Sat, Dec 03, 2005 at 03:29:54PM +0100, Jesper Juhl wrote: > > > > > > Why can't this be done by distributors/vendors? > > > > It already is done by these people, look at the "enterprise" Linux > > distributions and their 5 years of maintance (or whatever the number > > is.) > > > > If people/customers want stability, they already have this option. > > I don't get the point where the advantage is when every distribution > creates it's own stable branches. They only do so because they are on different release cycles. > AFAIR one of the reasons for the current 2.6 development model was to > reduce the amount of feature patches distributions ship by offering an > ftp.kernel.org kernel that gets new features early. Sure, that's one of the reasons. But not the only one by far (I have a whole list somewhere, I'm sure it's in the archives...) > What's wrong with offering an unified branch with few regressions for > both users and distributions? It's not that every distribution will use > it, but as soon as one or two distributions are using it the amount of > extra work for maintaining the branch should become pretty low. "pretty low"? hahahahaha. As Arjan has said, the time and energy to do this for a long period of time is huge. If I were you, I would listen to the people who have and do maintain these kinds of kernels, it's not a simple job by any means. But by all means, if you want to try to do this, please do. I wish you good luck. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:51 ` Adrian Bunk 2005-12-03 23:28 ` Greg KH @ 2005-12-03 23:35 ` Chris Wright 2005-12-06 0:37 ` Rob Landley 2005-12-04 8:07 ` Arjan van de Ven 2005-12-05 20:33 ` Florian Weimer 3 siblings, 1 reply; 639+ messages in thread From: Chris Wright @ 2005-12-03 23:35 UTC (permalink / raw) To: Adrian Bunk; +Cc: Greg KH, Jesper Juhl, linux-kernel * Adrian Bunk (bunk@stusta.de) wrote: > I don't get the point where the advantage is when every distribution > creates it's own stable branches. They often start from a different base. > AFAIR one of the reasons for the current 2.6 development model was to > reduce the amount of feature patches distributions ship by offering an > ftp.kernel.org kernel that gets new features early. > > What's wrong with offering an unified branch with few regressions for > both users and distributions? It's not that every distribution will use > it, but as soon as one or two distributions are using it the amount of > extra work for maintaining the branch should become pretty low. Backporting is real work, and can introduce instabilities of its own. If the goal is to stop breaking userspace ABI, there's no guarantee this will be done via backporting w/out careful inspection, esp. for sysfs and proc entries. More to the point, breaking userspace (I'm not talking about deprecated features) should stop upstream, not only in some frozen branch. If the goal is to make sure old kernels have security fixes, which old kernel branch do you follow, the numbers will only grow. Distros are in a position to look at current -stable updates and see if security fixes are relevant. About the only thing I think is helpful in this case is perhaps one extra -stable cycle on the last branch when newest branch is released (basically flush the queue). That much I'm willing to do in -stable. thanks, -chris ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 23:35 ` Chris Wright @ 2005-12-06 0:37 ` Rob Landley 2005-12-07 21:38 ` Nix 0 siblings, 1 reply; 639+ messages in thread From: Rob Landley @ 2005-12-06 0:37 UTC (permalink / raw) To: Chris Wright; +Cc: Adrian Bunk, Greg KH, Jesper Juhl, linux-kernel On Saturday 03 December 2005 17:35, Chris Wright wrote: > relevant. About the only thing I think is helpful in this case is perhaps > one extra -stable cycle on the last branch when newest branch is released > (basically flush the queue). That much I'm willing to do in -stable. Yay rah cool! Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 0:37 ` Rob Landley @ 2005-12-07 21:38 ` Nix 0 siblings, 0 replies; 639+ messages in thread From: Nix @ 2005-12-07 21:38 UTC (permalink / raw) To: Rob Landley; +Cc: Chris Wright, Adrian Bunk, Greg KH, Jesper Juhl, linux-kernel On 6 Dec 2005, Rob Landley moaned: > On Saturday 03 December 2005 17:35, Chris Wright wrote: >> relevant. About the only thing I think is helpful in this case is perhaps >> one extra -stable cycle on the last branch when newest branch is released >> (basically flush the queue). That much I'm willing to do in -stable. > > Yay rah cool! Seconded (thirded?), this is a very good idea (and as it's just a queue flush is probably quite easy to do). That way those of us who are paranoid can upgrade our experimental boxes immediately, apply the latest -stable to the non-experimental boxes, and then cautiously upgrade those boxes when the experimental ones seem to be working OK. Currently whenever there's a non-stable kernel rev I'm filled with trepidation: do I upgrade the stable boxes and risk instability, or leave them as they are and risk insecurity? -- `Y'know, London's nice at this time of year. If you like your cities freezing cold and full of surly gits.' --- David Damerell ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:51 ` Adrian Bunk 2005-12-03 23:28 ` Greg KH 2005-12-03 23:35 ` Chris Wright @ 2005-12-04 8:07 ` Arjan van de Ven 2005-12-05 20:33 ` Florian Weimer 3 siblings, 0 replies; 639+ messages in thread From: Arjan van de Ven @ 2005-12-04 8:07 UTC (permalink / raw) To: Adrian Bunk; +Cc: Greg KH, Jesper Juhl, linux-kernel > What's wrong with offering an unified branch with few regressions for > both users and distributions? It's not that every distribution will use > it, but as soon as one or two distributions are using it the amount of > extra work for maintaining the branch should become pretty low. the problem is simple: distributions will NOT use it. They can't, they need the newer HW support and the new features. The only difference is that you basically added just another distro branch. If you knew how many people it takes a distro to run such an old tree you'd be scared. (you need to include the QA people as well in this) And distros hardly add hw support (the only hw support the enterprise distros add is contained to like 5 or 10 drivers of "enterprise" stuff, well testable and often validated by the vendor of the hw; and even then there are regressions regularly)... for your branch you target a different audience that does want a lot broader new HW support. And then there's the bugfixes.. those tend to have regressions too, especially if you move them to a different context than they originally came from. I'm not saying that you couldn't or shouldn't do this, I'm just saying that I think it'll be a LOT more work than you think, and that the gain is relatively minor. Especially since the main branch needs to sort the compatibility item ANYWAY. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:51 ` Adrian Bunk ` (2 preceding siblings ...) 2005-12-04 8:07 ` Arjan van de Ven @ 2005-12-05 20:33 ` Florian Weimer 2005-12-06 1:10 ` Horst von Brand 3 siblings, 1 reply; 639+ messages in thread From: Florian Weimer @ 2005-12-05 20:33 UTC (permalink / raw) To: Adrian Bunk; +Cc: Greg KH, Jesper Juhl, linux-kernel * Adrian Bunk: > I don't get the point where the advantage is when every distribution > creates it's own stable branches. Different vendors have different needs WRT proprietary drivers, experimental file systems, network stack tweaks. Their release cycles aren't synchronized, so it's not clear at which point you can make sorely needed architectural changes in a stable kernel series (for example, to fix some egregious performance issues, or complicated security issues). > What's wrong with offering an unified branch with few regressions for > both users and distributions? One user's regression is another's bug fix. And where do those regressions come from if you don't make any changes in functionality? 8-) > It's not that every distribution will use > it, but as soon as one or two distributions are using it the amount of > extra work for maintaining the branch should become pretty low. Maybe, but I don't see why a vendor should give up its kernel branding. You mentioned security issues in your initial post. I think it would help immensely if security bugs would be documented properly (affected versions, configuration requirements, attack range, loss type etc.) when the bug is fixed, by someone who is familiar with the code. (Currently, this information is scraped together mostly by security folks, sometimes after considerable time has passed.) Having a central repository with this kind of information would enable vendors and not-quite-vendors (people who have their own set of kernels for their machines) to address more vulnerabilties promptly, including less critical ones. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 20:33 ` Florian Weimer @ 2005-12-06 1:10 ` Horst von Brand 2005-12-06 10:46 ` Matthias Andree 2005-12-06 14:01 ` Florian Weimer 0 siblings, 2 replies; 639+ messages in thread From: Horst von Brand @ 2005-12-06 1:10 UTC (permalink / raw) To: Florian Weimer; +Cc: Adrian Bunk, Greg KH, Jesper Juhl, linux-kernel Florian Weimer <fw@deneb.enyo.de> wrote: [...] > You mentioned security issues in your initial post. I think it would > help immensely if security bugs would be documented properly (affected > versions, configuration requirements, attack range, loss type etc.) > when the bug is fixed, by someone who is familiar with the code. > (Currently, this information is scraped together mostly by security > folks, sometimes after considerable time has passed.) Having a > central repository with this kind of information would enable vendors > and not-quite-vendors (people who have their own set of kernels for > their machines) to address more vulnerabilties promptly, including > less critical ones. I've fixed bugs which turned out to be security vulnerabilities. And I didn't know (or even care much) at the time. Finding out if some random bug has security implications, and exactly which ones/how much of a risk they pose is normally /much/ harder than to fix the bugs. And rather pointless, after the fix is in. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 1:10 ` Horst von Brand @ 2005-12-06 10:46 ` Matthias Andree 2005-12-06 14:01 ` Florian Weimer 1 sibling, 0 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-06 10:46 UTC (permalink / raw) To: linux-kernel On Mon, 05 Dec 2005, Horst von Brand wrote: > > You mentioned security issues in your initial post. I think it would > > help immensely if security bugs would be documented properly (affected > > versions, configuration requirements, attack range, loss type etc.) > > when the bug is fixed, by someone who is familiar with the code. > > (Currently, this information is scraped together mostly by security > > folks, sometimes after considerable time has passed.) Having a > > central repository with this kind of information would enable vendors > > and not-quite-vendors (people who have their own set of kernels for > > their machines) to address more vulnerabilties promptly, including > > less critical ones. > > I've fixed bugs which turned out to be security vulnerabilities. And I > didn't know (or even care much) at the time. Finding out if some random bug > has security implications, and exactly which ones/how much of a risk they > pose is normally /much/ harder than to fix the bugs. And rather pointless, > after the fix is in. I believe everyone who maintains a nontrivial piece of software has experienced a situation where a bug fix addressed a bug that could actually be exploited and that wasn't clear at the time. Calling this "pointless" after the fix is in leaves people in danger unaware, unless it happens on a branch where every user can be expected to update because only tested fixes are merged. As this isn't the case for the kernel, but everyone moves on at will, doesn't care if a previous bug fix is exploitable and whatnot, the Linux kernel's security is essentially nonexistent, and expecting downstream QA teams to handle this is just ridiculous for many reasons already mentioned. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 1:10 ` Horst von Brand 2005-12-06 10:46 ` Matthias Andree @ 2005-12-06 14:01 ` Florian Weimer 2005-12-06 16:52 ` Gene Heskett 1 sibling, 1 reply; 639+ messages in thread From: Florian Weimer @ 2005-12-06 14:01 UTC (permalink / raw) To: Horst von Brand; +Cc: Adrian Bunk, Greg KH, Jesper Juhl, linux-kernel * Horst von Brand: >> You mentioned security issues in your initial post. I think it would >> help immensely if security bugs would be documented properly (affected >> versions, configuration requirements, attack range, loss type etc.) >> when the bug is fixed, by someone who is familiar with the code. >> (Currently, this information is scraped together mostly by security >> folks, sometimes after considerable time has passed.) Having a >> central repository with this kind of information would enable vendors >> and not-quite-vendors (people who have their own set of kernels for >> their machines) to address more vulnerabilties promptly, including >> less critical ones. > > I've fixed bugs which turned out to be security vulnerabilities. And I > didn't know (or even care much) at the time. Finding out if some random bug > has security implications, and exactly which ones/how much of a risk they > pose is normally /much/ harder than to fix the bugs. I know, it happens all the time: vulnerabilities are fixed because they are bugs, and not because they are vulnerabilities. It's unfortunate if people are unnecessarily exposed to the vulnerability (because they don't know about it and don't apply the fix as a result), but it's better than carrying around the bug indefinitely. But if there's considerable evidence that you might have fixed a security bug, preserving this information (and other bits that are immediately obvious to you as a developer, but not necessarily who reviews the issue) seems worthwhile. Maybe you don't want to put it into the public commit message, but forwarding what you have to some trusted group of volunteers would make sense. The volunteers would distill the information, add more data and assign a CVE if necessary, and declassify the information as soon as the public is ready (in the form of a short security advisory, like the ones you see for most applications). Does this sound too far-fetched? Why don't you think this would be a valuable service to all users, vanilla kernels or not? > And rather pointless, after the fix is in. It doesn't matter much if the fix is in the kernel.org tree, when I'm supposed to use vendor kernels. 8-) ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 14:01 ` Florian Weimer @ 2005-12-06 16:52 ` Gene Heskett 0 siblings, 0 replies; 639+ messages in thread From: Gene Heskett @ 2005-12-06 16:52 UTC (permalink / raw) To: linux-kernel On Tuesday 06 December 2005 09:01, Florian Weimer wrote: >* Horst von Brand: >>> You mentioned security issues in your initial post. I think it >>> would help immensely if security bugs would be documented properly >>> (affected versions, configuration requirements, attack range, loss >>> type etc.) when the bug is fixed, by someone who is familiar with >>> the code. (Currently, this information is scraped together mostly by >>> security folks, sometimes after considerable time has passed.) >>> Having a central repository with this kind of information would >>> enable vendors and not-quite-vendors (people who have their own set >>> of kernels for their machines) to address more vulnerabilties >>> promptly, including less critical ones. >> >> I've fixed bugs which turned out to be security vulnerabilities. And >> I didn't know (or even care much) at the time. Finding out if some >> random bug has security implications, and exactly which ones/how much >> of a risk they pose is normally /much/ harder than to fix the bugs. > >I know, it happens all the time: vulnerabilities are fixed because >they are bugs, and not because they are vulnerabilities. It's >unfortunate if people are unnecessarily exposed to the vulnerability >(because they don't know about it and don't apply the fix as a result), >but it's better than carrying around the bug indefinitely. > >But if there's considerable evidence that you might have fixed a >security bug, preserving this information (and other bits that are >immediately obvious to you as a developer, but not necessarily who >reviews the issue) seems worthwhile. Maybe you don't want to put it >into the public commit message, but forwarding what you have to some >trusted group of volunteers would make sense. The volunteers would >distill the information, add more data and assign a CVE if necessary, >and declassify the information as soon as the public is ready (in the >form of a short security advisory, like the ones you see for most >applications). > >Does this sound too far-fetched? Why don't you think this would be a >valuable service to all users, vanilla kernels or not? But, as you'll recall, ALan Cox fixed a couple of major security things about 2 years ago, and because of the DMCA, those patches weren't commented at all. Apparently the fix could only be determined to be correct by violating the DMCA, and he was very pointed in his comments re the DMCA at the time. So keeping good records might/could be a double edged sword. >> And rather pointless, after the fix is in. > >It doesn't matter much if the fix is in the kernel.org tree, when I'm >supposed to use vendor kernels. 8-) >- >To unsubscribe from this list: send the line "unsubscribe linux-kernel" > in the body of a message to majordomo@vger.kernel.org >More majordomo info at http://vger.kernel.org/majordomo-info.html >Please read the FAQ at http://www.tux.org/lkml/ -- Cheers, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) 99.36% setiathome rank, not too shabby for a WV hillbilly Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2005 by Maurice Eugene Heskett, all rights reserved. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 20:19 ` Greg KH ` (2 preceding siblings ...) 2005-12-03 22:51 ` Adrian Bunk @ 2005-12-04 3:48 ` Jesper Juhl 2005-12-04 11:56 ` Matthias Andree 2005-12-04 17:00 ` Jakob Oestergaard 2005-12-05 14:48 ` Florian Weimer 5 siblings, 1 reply; 639+ messages in thread From: Jesper Juhl @ 2005-12-04 3:48 UTC (permalink / raw) To: Greg KH; +Cc: Adrian Bunk, linux-kernel On 12/3/05, Greg KH <greg@kroah.com> wrote: > On Sat, Dec 03, 2005 at 03:29:54PM +0100, Jesper Juhl wrote: > > > > Why can't this be done by distributors/vendors? > > It already is done by these people, look at the "enterprise" Linux > distributions and their 5 years of maintance (or whatever the number > is.) > > If people/customers want stability, they already have this option. > Yes, I know this is what's done with the "enterprise" distro kernels. Perhaps I should have phrased it "Why can't this job just stay with vendors". -- Jesper Juhl <jesper.juhl@gmail.com> Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html Plain text mails only, please http://www.expita.com/nomime.html ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 3:48 ` Jesper Juhl @ 2005-12-04 11:56 ` Matthias Andree 2005-12-04 23:24 ` Greg KH 0 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2005-12-04 11:56 UTC (permalink / raw) To: linux-kernel On Sun, 04 Dec 2005, Jesper Juhl wrote: > On 12/3/05, Greg KH <greg@kroah.com> wrote: > > On Sat, Dec 03, 2005 at 03:29:54PM +0100, Jesper Juhl wrote: > > > > > > Why can't this be done by distributors/vendors? > > > > It already is done by these people, look at the "enterprise" Linux > > distributions and their 5 years of maintance (or whatever the number > > is.) > > > > If people/customers want stability, they already have this option. > > > > Yes, I know this is what's done with the "enterprise" distro kernels. > Perhaps I should have phrased it "Why can't this job just stay with > vendors". Because this is just shifting the blame for and work to make up for the upstream not providing a stable tree on somebody else and prescinds from the fact that many people are apparently unhappy with 2.6.X policies. I cannot see a project issuing "stable releases" if every other developer bleats "let the distro snapshot and backport fixes on their own". This is exactly the point that turns away half of those who hadn't been scared away by the "Linux has no uniform userland" problem yet. 2.6.0 is now nearly two years old, perhaps the current discussions mean that 2.7/2.8 are long overdue - some people feel the need for more radical code changes, which are 2.7 stuff. The problem is the upstream breaking backwards compatibility for no good reason. This can sometimes be a genuine unintended regression (aka. bug), but quite often this is deliberate breakage because someone wants to get rid of cruft. While the motivation is sound, breaking between 2.6.N and 2.6.M must stop. One of the ideas of the new development style and versioning scheme was to have 2.6 progress faster than 2.3 or 2.5, and to have shorter release cycles. It was found to introduce way too much breakage. Linus's relatively new policy "merge new stuff only during the fortnight after release, then fix up" is a concession to these observations that too many things break if there is a constant influx of feature changes. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 11:56 ` Matthias Andree @ 2005-12-04 23:24 ` Greg KH 2005-12-05 6:26 ` Willy Tarreau ` (2 more replies) 0 siblings, 3 replies; 639+ messages in thread From: Greg KH @ 2005-12-04 23:24 UTC (permalink / raw) To: linux-kernel On Sun, Dec 04, 2005 at 12:56:50PM +0100, Matthias Andree wrote: > The problem is the upstream breaking backwards compatibility for no good > reason. This can sometimes be a genuine unintended regression (aka. > bug), but quite often this is deliberate breakage because someone wants > to get rid of cruft. While the motivation is sound, breaking between > 2.6.N and 2.6.M must stop. What are we breaking that people are complaining so much about? Specifics please. And if you bring up udev, please see my previous comments in this thread about that issue. It isn't userspace stuff that is breaking, as applications built on 2.2 still work just fine here on 2.6 for me. Yes we break in-kernel apis, all the time, that's fine. See Documentation/stable-api-nonsense.txt for details about why we do that. So again, specifics please? thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 23:24 ` Greg KH @ 2005-12-05 6:26 ` Willy Tarreau 2005-12-05 10:00 ` Matthias Andree ` (2 more replies) 2005-12-05 18:51 ` Adrian Bunk 2005-12-06 14:32 ` Florian Weimer 2 siblings, 3 replies; 639+ messages in thread From: Willy Tarreau @ 2005-12-05 6:26 UTC (permalink / raw) To: Greg KH; +Cc: linux-kernel, Adrian Bunk, Matthias Andree Hi Greg, I've been following most of this thread but did hot take the time to jump in. On Sun, Dec 04, 2005 at 03:24:54PM -0800, Greg KH wrote: > On Sun, Dec 04, 2005 at 12:56:50PM +0100, Matthias Andree wrote: > > The problem is the upstream breaking backwards compatibility for no good > > reason. This can sometimes be a genuine unintended regression (aka. > > bug), but quite often this is deliberate breakage because someone wants > > to get rid of cruft. While the motivation is sound, breaking between > > 2.6.N and 2.6.M must stop. > > What are we breaking that people are complaining so much about? > Specifics please. Speaking for myself, I will not be complaining about specific things which break, but I'll explain my point of view on the real problem. The kernel has entered a permanent development phase, with some versions more stable/usable than others. You and Chris do a wonderful job at producing stable versions. I know how difficult it is, for doing the same in 2.4 (and 2.4 moves slower than 2.6). However, the problem is that you stop maintaining old versions and quickly switch to new ones when a new kernel comes up. I know it's not easy, and may be terribly more difficult to maintain several versions in a development kernel than in a stable kernel. But imagine the users' position : they run 2.6.14.3 which finally fixes all their problems. Then they get a new problem, but 2.6.15 comes out. There will not be any other 2.6.14, so they have the choice of staying to 2.6.14.3 or jumping to fresh new and barely tested 2.6.15. People have been surprized that I still maintain several old versions of 2.4 at once, but I've received lots of "thank you" emails from people who still relied on them for a particular tree which does not evolve as fast as mainline. And 2.4 is easier to follow than 2.6. What I think should be done is to still maintain older 2.6 (eg: 2, 3 or 4 previous releases) so that people will have the time to switch to a new one. And I think that what Adrian wants to do would be useful *only* if he proceeds that way. Maybe you should just join forces, eg Chris and you to catch new patches, and Adrian to merge them to older kernels ? Every software maker always supports a few older releases for the people who need to stay on something stable, and it is clearly what is missing now in 2.6. Also, I think differently from Adrian. He wants to backport all new drivers and new features, while I think that they are the most sensible parts and the one which bring the more changes to the kernel. In fact, we should *only* maintain security and critical fixes on older releases. People in the need of a new driver must upgrade for this. This is the difference between staying on an old thing because you don't need to upgrade and switching to a new one because you need this new shiny feature. It follows the "if it ain't broke, don't fix it" rule. Users will never excuse you for breaking their working kernels by adding something they don't use. I would have liked to help in this area (I even discussed about maintaining a 2.6-stable a long time ago) but I don't have enough time for this. 2.4 is already time-consuming. Regards, Willy ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 6:26 ` Willy Tarreau @ 2005-12-05 10:00 ` Matthias Andree 2005-12-05 10:55 ` Lars Marowsky-Bree 2005-12-06 17:54 ` Greg KH 2 siblings, 0 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-05 10:00 UTC (permalink / raw) To: Willy Tarreau; +Cc: Greg KH, linux-kernel, Adrian Bunk, Matthias Andree On Mon, 05 Dec 2005, Willy Tarreau wrote: > However, the problem is that you stop maintaining old versions > and quickly switch to new ones when a new kernel comes up. I know > it's not easy, and may be terribly more difficult to maintain > several versions in a development kernel than in a stable kernel. > But imagine the users' position : they run 2.6.14.3 which finally > fixes all their problems. Then they get a new problem, but 2.6.15 > comes out. There will not be any other 2.6.14, so they have the > choice of staying to 2.6.14.3 or jumping to fresh new and barely > tested 2.6.15. "Regression" as the threat of updating. That was the starting point :) I believe the reason to abandon the previous "stable" branchlet was the assumption that the new kernel had all fixes from the previous "stable" merged, i. e. every patch between 2.6.14 and 2.6.14.3 would become part of 2.6.15 (or the underlying problem addressed by a patch were resolved some other way). > What I think should be done is to still maintain older 2.6 > (eg: 2, 3 or 4 previous releases) so that people will have > the time to switch to a new one. And I think that what Adrian > wants to do would be useful *only* if he proceeds that way. Perhaps a fixed number of releases doesn't cut it. Perhaps a fixed time neither. If the changes betweeen two subsequent releases is low, one or more extra versions come for free, but if lots of changes go in all over the map, it's going to be a royal pain. It ultimately boils down to the question: how far^Wfast the upstream wants to run away^W^Wprogress from its previous release. > Also, I think differently from Adrian. He wants to backport > all new drivers and new features, while I think that they are > the most sensible parts and the one which bring the more > changes to the kernel. In fact, we should *only* maintain > security and critical fixes on older releases. People in the > need of a new driver must upgrade for this. This is the I think there is no such thing as The Single One New Driver[tm]. Some are quite intrusive, some aren't. Sometimes the new driver works with older kernels (if the driver is self-contained, for instance just a dozen new lines of code in an existing driver), sometimes not (because midlayer or core changes are required to support the new driver). -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 6:26 ` Willy Tarreau 2005-12-05 10:00 ` Matthias Andree @ 2005-12-05 10:55 ` Lars Marowsky-Bree 2005-12-05 11:34 ` Willy Tarreau 2005-12-06 17:54 ` Greg KH 2 siblings, 1 reply; 639+ messages in thread From: Lars Marowsky-Bree @ 2005-12-05 10:55 UTC (permalink / raw) To: Willy Tarreau, Greg KH; +Cc: linux-kernel, Adrian Bunk, Matthias Andree On 2005-12-05T07:26:09, Willy Tarreau <willy@w.ods.org> wrote: > What I think should be done is to still maintain older 2.6 > (eg: 2, 3 or 4 previous releases) so that people will have > the time to switch to a new one. And I think that what Adrian > wants to do would be useful *only* if he proceeds that way. > > Maybe you should just join forces, eg Chris and you to catch > new patches, and Adrian to merge them to older kernels ? Every > software maker always supports a few older releases for the > people who need to stay on something stable, and it is clearly > what is missing now in 2.6. Well, this is probably the most useful suggestion so far. The kernel is free land; if you or someone else wants to maintain the upcoming 2.6.16 "forever", and backport fixes or selected features, by all means, do it. Define your policy, set up a tree, and off you go. If Adrian will maintain it, it'll for sure be the most static kernel ever. This won't impact the Linux kernel, which will just continue to run its course. The kernel process as a whole doesn't need to change; just someone needs to do the grunt work. If your kernel is wildly successful and adopted by users as well as distributions, you'll be very happy and tell us 'told ya so!'. If not, no harm will be done either, and you'll have the kernel you want for your own purposes. Be aware however that this is a very painful job. Trust me, I've been involved with the receiving end of maintaining such a kernel for SLES for a couple of releases. ;-) Which is exactly the point: it's so painful that for this, people want to be paid, and don't like doing it in their spare time. You may maintain it for 6 months, sure, which will be less painful than maintaining it for 5, 7 years, but when you rebase, you'll still put your users into the dependency hell, and they won't have tested the intermediate releases... Ouch. Not to mention that not every backported fix is trivial to do. Anyway, good luck to you. The current 2.6.x.y-stable series is quite sane, because they are essentially just fixing very critical bugs in very recent kernels, with little back porting effort. Sincerely, Lars Marowsky-Brée <lmb@suse.de> -- High Availability & Clustering SUSE Labs, Research and Development SUSE LINUX Products GmbH - A Novell Business -- Charles Darwin "Ignorance more frequently begets confidence than does knowledge" ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 10:55 ` Lars Marowsky-Bree @ 2005-12-05 11:34 ` Willy Tarreau 2005-12-05 11:40 ` Lars Marowsky-Bree 0 siblings, 1 reply; 639+ messages in thread From: Willy Tarreau @ 2005-12-05 11:34 UTC (permalink / raw) To: Lars Marowsky-Bree; +Cc: Greg KH, linux-kernel, Adrian Bunk, Matthias Andree On Mon, Dec 05, 2005 at 11:55:36AM +0100, Lars Marowsky-Bree wrote: > On 2005-12-05T07:26:09, Willy Tarreau <willy@w.ods.org> wrote: > > > What I think should be done is to still maintain older 2.6 > > (eg: 2, 3 or 4 previous releases) so that people will have > > the time to switch to a new one. And I think that what Adrian > > wants to do would be useful *only* if he proceeds that way. > > > > Maybe you should just join forces, eg Chris and you to catch > > new patches, and Adrian to merge them to older kernels ? Every > > software maker always supports a few older releases for the > > people who need to stay on something stable, and it is clearly > > what is missing now in 2.6. > > Well, this is probably the most useful suggestion so far. The kernel is > free land; if you or someone else wants to maintain the upcoming 2.6.16 > "forever", and backport fixes or selected features, by all means, do it. > Define your policy, set up a tree, and off you go. > > If Adrian will maintain it, it'll for sure be the most static kernel > ever. > > This won't impact the Linux kernel, which will just continue to run its > course. The kernel process as a whole doesn't need to change; just > someone needs to do the grunt work. > > If your kernel is wildly successful and adopted by users as well as > distributions, you'll be very happy and tell us 'told ya so!'. If not, > no harm will be done either, and you'll have the kernel you want for > your own purposes. > > Be aware however that this is a very painful job. Trust me, I've been > involved with the receiving end of maintaining such a kernel for SLES > for a couple of releases. ;-) > > Which is exactly the point: it's so painful that for this, people want > to be paid, and don't like doing it in their spare time. You may > maintain it for 6 months, sure, which will be less painful than > maintaining it for 5, 7 years, but when you rebase, you'll still put > your users into the dependency hell, and they won't have tested the > intermediate releases... Ouch. Not to mention that not every backported > fix is trivial to do. > > Anyway, good luck to you. > > The current 2.6.x.y-stable series is quite sane, because they are > essentially just fixing very critical bugs in very recent kernels, with > little back porting effort. I agree it is sane. The problem is that it does not exist for long enough. When you have 2.6.14.X working perfectly and you need a fix for a newly discovered security fix which only exists in 2.6.15.Y, then you have to leave 2.6.14 and enter 2.6.15. That is the problem, because for just a fix, you change megabytes of source code which will bring their equivalent in bugs. Regards, willy ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 11:34 ` Willy Tarreau @ 2005-12-05 11:40 ` Lars Marowsky-Bree 2005-12-05 12:01 ` Willy Tarreau 2005-12-05 12:24 ` Bernd Eckenfels 0 siblings, 2 replies; 639+ messages in thread From: Lars Marowsky-Bree @ 2005-12-05 11:40 UTC (permalink / raw) To: Willy Tarreau; +Cc: Greg KH, linux-kernel, Adrian Bunk, Matthias Andree On 2005-12-05T12:34:20, Willy Tarreau <willy@w.ods.org> wrote: > > Anyway, good luck to you. > > > > The current 2.6.x.y-stable series is quite sane, because they are > > essentially just fixing very critical bugs in very recent kernels, with > > little back porting effort. > > I agree it is sane. The problem is that it does not exist for long enough. > When you have 2.6.14.X working perfectly and you need a fix for a newly > discovered security fix which only exists in 2.6.15.Y, then you have to > leave 2.6.14 and enter 2.6.15. That is the problem, because for just a > fix, you change megabytes of source code which will bring their equivalent > in bugs. As I said, please, go on maintaining a release for a longer period of time. Sincerely, Lars Marowsky-Brée <lmb@suse.de> -- High Availability & Clustering SUSE Labs, Research and Development SUSE LINUX Products GmbH - A Novell Business -- Charles Darwin "Ignorance more frequently begets confidence than does knowledge" ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 11:40 ` Lars Marowsky-Bree @ 2005-12-05 12:01 ` Willy Tarreau 2005-12-05 12:24 ` Bernd Eckenfels 1 sibling, 0 replies; 639+ messages in thread From: Willy Tarreau @ 2005-12-05 12:01 UTC (permalink / raw) To: Lars Marowsky-Bree; +Cc: Greg KH, linux-kernel, Adrian Bunk, Matthias Andree On Mon, Dec 05, 2005 at 12:40:28PM +0100, Lars Marowsky-Bree wrote: > On 2005-12-05T12:34:20, Willy Tarreau <willy@w.ods.org> wrote: > > > > Anyway, good luck to you. > > > > > > The current 2.6.x.y-stable series is quite sane, because they are > > > essentially just fixing very critical bugs in very recent kernels, with > > > little back porting effort. > > > > I agree it is sane. The problem is that it does not exist for long enough. > > When you have 2.6.14.X working perfectly and you need a fix for a newly > > discovered security fix which only exists in 2.6.15.Y, then you have to > > leave 2.6.14 and enter 2.6.15. That is the problem, because for just a > > fix, you change megabytes of source code which will bring their equivalent > > in bugs. > > As I said, please, go on maintaining a release for a longer period of > time. As I said, I know this is difficult, I already do this for 2.4 and 2.4 is not moving fast. But what Adrian wants to do might be far more difficult. That's why I suggest him to do "only" this, he will have less work and get a lot of happy users. Regards, Willy ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 11:40 ` Lars Marowsky-Bree 2005-12-05 12:01 ` Willy Tarreau @ 2005-12-05 12:24 ` Bernd Eckenfels 2005-12-05 12:26 ` Arjan van de Ven 1 sibling, 1 reply; 639+ messages in thread From: Bernd Eckenfels @ 2005-12-05 12:24 UTC (permalink / raw) To: linux-kernel In article <20051205114028.GD5148@marowsky-bree.de> you wrote: > As I said, please, go on maintaining a release for a longer period of > time. On the other hand: Since it is such a ugly big boring and complicated task, why do you still think it can be done by volunteers? (or why will commercial distributions have the power to pay for this in the long run) I think this is exactly the reason why it cannot be done in parallel by volunteers without support from all of the contributors. With an official stable trunk it attrackts more backports. It is clear where the security fixes must happen, etc. We used to have this back in the 2.4 days with longer release cycles. However I am not sure if that was better or worse. Gruss Bernd ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 12:24 ` Bernd Eckenfels @ 2005-12-05 12:26 ` Arjan van de Ven 0 siblings, 0 replies; 639+ messages in thread From: Arjan van de Ven @ 2005-12-05 12:26 UTC (permalink / raw) To: Bernd Eckenfels; +Cc: linux-kernel > We used to have this back in the 2.4 days with longer release cycles. > However I am not sure if that was better or worse. 2.4 also had its fair share of regressions. Just that people by now have forgotten about those ;) ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 6:26 ` Willy Tarreau 2005-12-05 10:00 ` Matthias Andree 2005-12-05 10:55 ` Lars Marowsky-Bree @ 2005-12-06 17:54 ` Greg KH 2005-12-06 18:57 ` John Kelly 2 siblings, 1 reply; 639+ messages in thread From: Greg KH @ 2005-12-06 17:54 UTC (permalink / raw) To: Willy Tarreau; +Cc: linux-kernel, Adrian Bunk, Matthias Andree On Mon, Dec 05, 2005 at 07:26:09AM +0100, Willy Tarreau wrote: > Maybe you should just join forces, eg Chris and you to catch > new patches, and Adrian to merge them to older kernels ? Every > software maker always supports a few older releases for the > people who need to stay on something stable, and it is clearly > what is missing now in 2.6. I don't think people realize that the -stable series only contains patches that other people send us. For the most part, Chris and I aren't going out there and activly writing up fixes for some of these issues, as we both don't have the time and energy to do this. But if someone wants to start sending us more patches that do this, we will be glad to incorporate them. And as Chris said, we will be glad to release an extra release for the last kernel if we have pending patches. And also, anyone else can easily take over maintaining these kernel branches. The git trees are public, as is our stable patch queue. So if anyone wants to maintain older kernels, it is quite easy to start the process. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 17:54 ` Greg KH @ 2005-12-06 18:57 ` John Kelly 2005-12-06 21:55 ` Adrian Bunk 0 siblings, 1 reply; 639+ messages in thread From: John Kelly @ 2005-12-06 18:57 UTC (permalink / raw) To: linux-kernel On Tue, 06 Dec 2005 09:54:22 -0800, Greg KH <greg@kroah.com> wrote: >I don't think people realize that the -stable series only contains >patches that other people send us. For the most part, Chris and I >aren't going out there and activly writing up fixes for some of these >issues, as we both don't have the time and energy to do this. The -stable series was a good idea, and thanks for doing it. Adrian's idea is a natural extension of what you started. >And also, anyone else can easily take over maintaining these kernel >branches. The git trees are public, as is our stable patch queue. So >if anyone wants to maintain older kernels, it is quite easy to start the >process. So if Adrian wants to begin where -stable ends, there is no reason for people to oppose his efforts. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 18:57 ` John Kelly @ 2005-12-06 21:55 ` Adrian Bunk 2005-12-06 22:40 ` John Kelly 0 siblings, 1 reply; 639+ messages in thread From: Adrian Bunk @ 2005-12-06 21:55 UTC (permalink / raw) To: John Kelly; +Cc: linux-kernel On Tue, Dec 06, 2005 at 01:57:55PM -0500, John Kelly wrote: > On Tue, 06 Dec 2005 09:54:22 -0800, Greg KH <greg@kroah.com> wrote: > > >I don't think people realize that the -stable series only contains > >patches that other people send us. For the most part, Chris and I > >aren't going out there and activly writing up fixes for some of these > >issues, as we both don't have the time and energy to do this. > > The -stable series was a good idea, and thanks for doing it. Adrian's > idea is a natural extension of what you started. > > > >And also, anyone else can easily take over maintaining these kernel > >branches. The git trees are public, as is our stable patch queue. So > >if anyone wants to maintain older kernels, it is quite easy to start the > >process. > > So if Adrian wants to begin where -stable ends, there is no reason for > people to oppose his efforts. I've read the whole thread, and I haven't seen anyone opposing my idea. Most people in this thread who did or do maintain some kernel branch simply expressed that in their opinion my idea would be too much work for too few users... cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 21:55 ` Adrian Bunk @ 2005-12-06 22:40 ` John Kelly 0 siblings, 0 replies; 639+ messages in thread From: John Kelly @ 2005-12-06 22:40 UTC (permalink / raw) To: linux-kernel On Tue, 06 Dec 2005 22:55:26 +0100, Adrian Bunk <bunk@stusta.de> wrote: >> So if Adrian wants to begin where -stable ends, there is no reason for >> people to oppose his efforts. >I've read the whole thread, and I haven't seen anyone opposing my idea. >Most people in this thread who did or do maintain some kernel branch >simply expressed that in their opinion my idea would be too much work >for too few users... If you build it, will they come? No one really knows. There is only one way to find out. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 23:24 ` Greg KH 2005-12-05 6:26 ` Willy Tarreau @ 2005-12-05 18:51 ` Adrian Bunk 2005-12-06 17:50 ` Greg KH 2005-12-06 14:32 ` Florian Weimer 2 siblings, 1 reply; 639+ messages in thread From: Adrian Bunk @ 2005-12-05 18:51 UTC (permalink / raw) To: Greg KH; +Cc: linux-kernel On Sun, Dec 04, 2005 at 03:24:54PM -0800, Greg KH wrote: > On Sun, Dec 04, 2005 at 12:56:50PM +0100, Matthias Andree wrote: > > The problem is the upstream breaking backwards compatibility for no good > > reason. This can sometimes be a genuine unintended regression (aka. > > bug), but quite often this is deliberate breakage because someone wants > > to get rid of cruft. While the motivation is sound, breaking between > > 2.6.N and 2.6.M must stop. > > What are we breaking that people are complaining so much about? > Specifics please. > > And if you bring up udev, please see my previous comments in this thread > about that issue. > > It isn't userspace stuff that is breaking, as applications built on 2.2 > still work just fine here on 2.6 for me. > > Yes we break in-kernel apis, all the time, that's fine. See > Documentation/stable-api-nonsense.txt for details about why we do that. > > So again, specifics please? It's the kernel-related userspace that is the problem (besides regressions that are simply bugs). Be it the devfs removal, the requirement for a more recent wpa_supplicant package or my pending removal of the obsolete raw driver. > thanks, > > greg k-h cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 18:51 ` Adrian Bunk @ 2005-12-06 17:50 ` Greg KH 2005-12-06 22:50 ` Policy for reverting user ABI breaking patches was " Andi Kleen 0 siblings, 1 reply; 639+ messages in thread From: Greg KH @ 2005-12-06 17:50 UTC (permalink / raw) To: Adrian Bunk; +Cc: linux-kernel On Mon, Dec 05, 2005 at 07:51:10PM +0100, Adrian Bunk wrote: > On Sun, Dec 04, 2005 at 03:24:54PM -0800, Greg KH wrote: > > On Sun, Dec 04, 2005 at 12:56:50PM +0100, Matthias Andree wrote: > > > The problem is the upstream breaking backwards compatibility for no good > > > reason. This can sometimes be a genuine unintended regression (aka. > > > bug), but quite often this is deliberate breakage because someone wants > > > to get rid of cruft. While the motivation is sound, breaking between > > > 2.6.N and 2.6.M must stop. > > > > What are we breaking that people are complaining so much about? > > Specifics please. > > > > And if you bring up udev, please see my previous comments in this thread > > about that issue. > > > > It isn't userspace stuff that is breaking, as applications built on 2.2 > > still work just fine here on 2.6 for me. > > > > Yes we break in-kernel apis, all the time, that's fine. See > > Documentation/stable-api-nonsense.txt for details about why we do that. > > > > So again, specifics please? > > It's the kernel-related userspace that is the problem (besides > regressions that are simply bugs). Again, specifics please? > Be it the devfs removal, the requirement for a more recent > wpa_supplicant package or my pending removal of the obsolete raw > driver. devfs was documented for over a year that it would be removed. This after 2 year notice that it was going to be removed some time in the future. So for over 3 years people have known about this. You have also notified people about the raw driver going away, what more can we do about this? And there will always be a need for new package upgrades for some small subset of programs that are tightly tied to the kernel (like wpa_supplicant or alsa-libs, or even udev). But "normal" userspace applicatations should not break, and if they do, we want to know about it so we can fix it. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Policy for reverting user ABI breaking patches was Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 17:50 ` Greg KH @ 2005-12-06 22:50 ` Andi Kleen 2005-12-06 18:42 ` Dmitry Torokhov 0 siblings, 1 reply; 639+ messages in thread From: Andi Kleen @ 2005-12-06 22:50 UTC (permalink / raw) To: Greg KH; +Cc: linux-kernel Greg KH <greg@kroah.com> writes: > > And there will always be a need for new package upgrades for some small > subset of programs that are tightly tied to the kernel (like > wpa_supplicant or alsa-libs, or even udev). But "normal" userspace Actually I don't necessarily agree on that. It's best to avoid breakage even for them. It has actually worked for a long time. In the early days of Linux there was frequent breakage like this but then in recent times the kernel has been very good at this for a long time (one exception was the module rewrite, but that was a single flag day). I have been running modern kernels on old distributions for a long time and it generally worked. And if there is breakage of such kernel-near applications there should be an *extremly* good reason for this (and minor cleanup isn't such a reason). For example for the recent udev breakage imho the cleanup patch that caused this should have just been reverted. I know it's not possible to know such bad interactions in advance, but when they are known and there isn't an *extremly* good reason for it then the ABI breaking change should be reverted. It would be good to have a policy like this: if an important program breaks due to a new kernel [With important being fairly liberally defined as anything shipped in standard distros unless it's something exotic that does something stupid or is obviously broken. External kernel modules or /dev/mem access don't count.] then the breakage needs to have an *extremly* good rationale (fixing security bugs etc.) and if there isn't one from the person who submitted the patch then it should be reverted. -Andi ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Policy for reverting user ABI breaking patches was Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 22:50 ` Policy for reverting user ABI breaking patches was " Andi Kleen @ 2005-12-06 18:42 ` Dmitry Torokhov 0 siblings, 0 replies; 639+ messages in thread From: Dmitry Torokhov @ 2005-12-06 18:42 UTC (permalink / raw) To: Andi Kleen; +Cc: Greg KH, linux-kernel On 06 Dec 2005 15:50:55 -0700, Andi Kleen <ak@suse.de> wrote: > And if there is breakage of such kernel-near applications there should > be an *extremly* good reason for this (and minor cleanup isn't such a > reason). For example for the recent udev breakage imho the cleanup > patch that caused this should have just been reverted. It was not a cleanup patch, without it you could not get input events through netlink. I wonder, since udev is fairly closely tied to the kernel, meybe it woudl be beneficial just to fold it in. This way we could keep compatibility with older kernels and rapidly roll out never stuff. -- Dmitry ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 23:24 ` Greg KH 2005-12-05 6:26 ` Willy Tarreau 2005-12-05 18:51 ` Adrian Bunk @ 2005-12-06 14:32 ` Florian Weimer [not found] ` <6f6293f10512060855p79fb5e91ke6fca33f96cb1750@mail.gmail.com> 2005-12-06 19:01 ` Lee Revell 2 siblings, 2 replies; 639+ messages in thread From: Florian Weimer @ 2005-12-06 14:32 UTC (permalink / raw) To: Greg KH; +Cc: linux-kernel * Greg KH: > What are we breaking that people are complaining so much about? > Specifics please. Drastic performance changes in certain pipe usage patterns. This was probably too early in the 2.6 series to count, though. There might be some subtle changes in the netfilter/routing interaction which break user configurations, but this still being tracked down (and maybe the any behavior is fine because it's unspecified; hard to tell). ^ permalink raw reply [flat|nested] 639+ messages in thread
[parent not found: <6f6293f10512060855p79fb5e91ke6fca33f96cb1750@mail.gmail.com>]
* Re: RFC: Starting a stable kernel series off the 2.6 kernel [not found] ` <6f6293f10512060855p79fb5e91ke6fca33f96cb1750@mail.gmail.com> @ 2005-12-06 17:47 ` Greg KH 2005-12-06 23:27 ` David S. Miller 0 siblings, 1 reply; 639+ messages in thread From: Greg KH @ 2005-12-06 17:47 UTC (permalink / raw) To: Felipe Alfaro Solana; +Cc: Florian Weimer, linux-kernel On Tue, Dec 06, 2005 at 05:55:42PM +0100, Felipe Alfaro Solana wrote: > > There might be some subtle changes in the netfilter/routing > > interaction which break user configurations, but this still being > > tracked down (and maybe the any behavior is fine because it's > > unspecified; hard to tell). > > Yeah! For example, the first datagram triggering an IPSec SA is always > lost (instead of being queued until the IPSec SA has been > established). > > For example, try pinging the IPSec SA peer for the very first time and > the first ICMP datagram will always return "resource currently > unavailable" and, of course, will get lost. > > BTW this works perfectly under *BSD and Mac OS X. Do the network kernel developers know about this issue? And if so, what have they said about it? thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 17:47 ` Greg KH @ 2005-12-06 23:27 ` David S. Miller 0 siblings, 0 replies; 639+ messages in thread From: David S. Miller @ 2005-12-06 23:27 UTC (permalink / raw) To: greg; +Cc: felipe.alfaro, fw, linux-kernel From: Greg KH <greg@kroah.com> Date: Tue, 6 Dec 2005 09:47:14 -0800 > On Tue, Dec 06, 2005 at 05:55:42PM +0100, Felipe Alfaro Solana wrote: > > > There might be some subtle changes in the netfilter/routing > > > interaction which break user configurations, but this still being > > > tracked down (and maybe the any behavior is fine because it's > > > unspecified; hard to tell). > > > > Yeah! For example, the first datagram triggering an IPSec SA is always > > lost (instead of being queued until the IPSec SA has been > > established). > > > > For example, try pinging the IPSec SA peer for the very first time and > > the first ICMP datagram will always return "resource currently > > unavailable" and, of course, will get lost. > > > > BTW this works perfectly under *BSD and Mac OS X. > > Do the network kernel developers know about this issue? And if so, what > have they said about it? It's on the TODO list, known problem with not an easy solution. BTW, BSD doesn't do any better, the KAME BSD ipsec stack drops the initial datagram just like we do. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 14:32 ` Florian Weimer [not found] ` <6f6293f10512060855p79fb5e91ke6fca33f96cb1750@mail.gmail.com> @ 2005-12-06 19:01 ` Lee Revell 1 sibling, 0 replies; 639+ messages in thread From: Lee Revell @ 2005-12-06 19:01 UTC (permalink / raw) To: Florian Weimer; +Cc: Greg KH, linux-kernel On Tue, 2005-12-06 at 15:32 +0100, Florian Weimer wrote: > * Greg KH: > > > What are we breaking that people are complaining so much about? > > Specifics please. > > Drastic performance changes in certain pipe usage patterns. This was > probably too early in the 2.6 series to count, though. Where's the bug report? Lee ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 20:19 ` Greg KH ` (3 preceding siblings ...) 2005-12-04 3:48 ` Jesper Juhl @ 2005-12-04 17:00 ` Jakob Oestergaard 2005-12-04 22:39 ` Greg KH 2005-12-05 14:48 ` Florian Weimer 5 siblings, 1 reply; 639+ messages in thread From: Jakob Oestergaard @ 2005-12-04 17:00 UTC (permalink / raw) To: Greg KH; +Cc: Jesper Juhl, Adrian Bunk, linux-kernel On Sat, Dec 03, 2005 at 12:19:45PM -0800, Greg KH wrote: > On Sat, Dec 03, 2005 at 03:29:54PM +0100, Jesper Juhl wrote: > > > > Why can't this be done by distributors/vendors? > > It already is done by these people, look at the "enterprise" Linux > distributions and their 5 years of maintance (or whatever the number > is.) > > If people/customers want stability, they already have this option. If the kernel was stable (reliability wise - as in "not crashing") then you'd be perfectly right. In the real world, however, admins currently need to pick out specific versions of the kernel for specific workloads (try running a large fileserver on anything but 2.6.11.11 for example - any earlier or later kernel will barf reliably. For web serving it's another kernel that's golden, I forgot which). There are very very good reasons for offering a 'stable series' in plain source-tree form - lots of admins of real-world systems need this. Adrian, I like the idea :) -- / jakob ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 17:00 ` Jakob Oestergaard @ 2005-12-04 22:39 ` Greg KH 2005-12-05 15:17 ` Jakob Oestergaard 0 siblings, 1 reply; 639+ messages in thread From: Greg KH @ 2005-12-04 22:39 UTC (permalink / raw) To: Jakob Oestergaard, Jesper Juhl, Adrian Bunk, linux-kernel On Sun, Dec 04, 2005 at 06:00:49PM +0100, Jakob Oestergaard wrote: > > If the kernel was stable (reliability wise - as in "not crashing") then > you'd be perfectly right. But isn't it? :) > In the real world, however, admins currently need to pick out specific > versions of the kernel for specific workloads (try running a large > fileserver on anything but 2.6.11.11 for example - any earlier or later > kernel will barf reliably. Have you filed a but at bugzilla.kernel.org about this? If not, how do you expect it to get fixed? > For web serving it's another kernel that's golden, I forgot which). That sounds very strange, the same kernel version should work just as well for all workloads. If not, it's a bug and should be fixed. > There are very very good reasons for offering a 'stable series' in plain > source-tree form - lots of admins of real-world systems need this. But it sounds like you will want different stable series depending on what kind of server you are running. And that will be even more work... thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 22:39 ` Greg KH @ 2005-12-05 15:17 ` Jakob Oestergaard 2005-12-05 15:44 ` Pekka Enberg 2005-12-06 17:44 ` Greg KH 0 siblings, 2 replies; 639+ messages in thread From: Jakob Oestergaard @ 2005-12-05 15:17 UTC (permalink / raw) To: Greg KH; +Cc: Jesper Juhl, Adrian Bunk, linux-kernel On Sun, Dec 04, 2005 at 02:39:31PM -0800, Greg KH wrote: > On Sun, Dec 04, 2005 at 06:00:49PM +0100, Jakob Oestergaard wrote: > > > > If the kernel was stable (reliability wise - as in "not crashing") then > > you'd be perfectly right. > > But isn't it? :) I like your sense of humor :) > > In the real world, however, admins currently need to pick out specific > > versions of the kernel for specific workloads (try running a large > > fileserver on anything but 2.6.11.11 for example - any earlier or later > > kernel will barf reliably. > > Have you filed a but at bugzilla.kernel.org about this? If not, how do > you expect it to get fixed? I don't expect to get it fixed. It's futile. It can get fixed in one version and broken two days later, and it seems the attitude is that that is just fine. After a long long back-and-forth, 2.6.11 was fixed to the point where it could reliably serve files (at least on uniprocessor configurations - and in my setup I don't see problems on NUMA either, but as far as I know that's just me being lucky). Right after that, someone thought it was a great idea to pry out the PCI subsystem and shovel in something else. Find, that's great for a development kernel, but for a kernel that's supposed to be stable it's just not something you can realistically do and expect things to work afterwards. And things broke - try mounting 10-20 XFS filesystems simultaneously on 2.6.14. Boom - PCI errors. Now what? Do I as a user upgrade my production environment to the latest and greatest kernel experiment, hope that the problems can be fixed quickly, and hope that I don't lose too much data in the process? (remember I will have people unable to do their jobs whenever the file server is down). Or do I stay on 2.6.11.11 which works on this particular server? I think I stay. > > > For web serving it's another kernel that's golden, I forgot which). > > That sounds very strange, the same kernel version should work just as > well for all workloads. If not, it's a bug and should be fixed. Well... You have bugs in different places in different kernels. It's perfectly understandable that kernel A works for workload p and fails on workload q, where kernel B works for workload q and fails on p. > > > There are very very good reasons for offering a 'stable series' in plain > > source-tree form - lots of admins of real-world systems need this. > > But it sounds like you will want different stable series depending on > what kind of server you are running. And that will be even more work... The idea would be to fix the actual bugs. After a while, one could have a kernel of higher quality with fewer bugs, making it a lot more likely that the *same* kernel tree could be used for both workloads A and B. It's really very simple :) Now, I'm just giving my oppinion as a user, and my advise as a developer - I know how much it sucks to postpone new great cleanups or features, just because some policy says the current branch has to be 'stable'. But I also know how much it sucks to have users complain that a new feature broke their existing setup. That's not a problem for a kernel developer of course, because users don't pay for the service and the "if it breaks you get to keep the pieces" attitude can be defended. But as a user, it really really sucks, even if you get to keep the pieces. I don't mean to be entirely negative - sure there are great things about the new development model. But there is a very significant downside for at least a group of users too. My 0.02 Euro, for what it's worth. -- / jakob ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 15:17 ` Jakob Oestergaard @ 2005-12-05 15:44 ` Pekka Enberg 2005-12-05 17:17 ` Jakob Oestergaard 2005-12-06 17:44 ` Greg KH 1 sibling, 1 reply; 639+ messages in thread From: Pekka Enberg @ 2005-12-05 15:44 UTC (permalink / raw) To: Jakob Oestergaard, Greg KH, Jesper Juhl, Adrian Bunk, linux-kernel Hi, On Sun, Dec 04, 2005 at 02:39:31PM -0800, Greg KH wrote: > > Have you filed a but at bugzilla.kernel.org about this? If not, how do > > you expect it to get fixed? On 12/5/05, Jakob Oestergaard <jakob@unthought.net> wrote: > I don't expect to get it fixed. It's futile. It can get fixed in one > version and broken two days later, and it seems the attitude is that > that is just fine. I don't think anyone breaks things on purpose. Please feel free to report the bug as many times as necessary to get it fixed. You shouldn't be complaining if you're not doing your part. Pekka ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 15:44 ` Pekka Enberg @ 2005-12-05 17:17 ` Jakob Oestergaard 0 siblings, 0 replies; 639+ messages in thread From: Jakob Oestergaard @ 2005-12-05 17:17 UTC (permalink / raw) To: Pekka Enberg; +Cc: Greg KH, Jesper Juhl, Adrian Bunk, linux-kernel On Mon, Dec 05, 2005 at 05:44:08PM +0200, Pekka Enberg wrote: > Hi, > > On Sun, Dec 04, 2005 at 02:39:31PM -0800, Greg KH wrote: > > > Have you filed a but at bugzilla.kernel.org about this? If not, how do > > > you expect it to get fixed? > > On 12/5/05, Jakob Oestergaard <jakob@unthought.net> wrote: > > I don't expect to get it fixed. It's futile. It can get fixed in one > > version and broken two days later, and it seems the attitude is that > > that is just fine. > > I don't think anyone breaks things on purpose. Of course not, silly :) But there's a difference between having a tree where you fix bugs and having a tree where you are very lax about major changes. By including major changes you (knowingly or not) risk breaking things, and things do break regularly. > Please feel free to > report the bug as many times as necessary to get it fixed. Thank you :) If it did not occur to you, then I was never in doubt of this. I tried to point out a problem in this approach - namely, that you will never end up with a stable tree if major changes (read; new bugs) go in as fast as old bugs are squashed. You do get a tree that is evolving very quickly, and which is somewhat stable for most purposes. Whether or not that is good enough for everyone, was pretty much the topic of this thread as I understand it. > You > shouldn't be complaining if you're not doing your part. I'm not complaining. I'm voicing my oppinion in a thread that discusses whether or not it would be a good idea to try and produce a stable tree. I think it would be a good idea, and you're free to disagree. Please read the thread if you're in doubt of the context of my comments :) -- / jakob ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 15:17 ` Jakob Oestergaard 2005-12-05 15:44 ` Pekka Enberg @ 2005-12-06 17:44 ` Greg KH 2005-12-06 21:16 ` Bill Davidsen 2005-12-07 14:38 ` Massimiliano Hofer 1 sibling, 2 replies; 639+ messages in thread From: Greg KH @ 2005-12-06 17:44 UTC (permalink / raw) To: Jakob Oestergaard, Jesper Juhl, Adrian Bunk, linux-kernel On Mon, Dec 05, 2005 at 04:17:53PM +0100, Jakob Oestergaard wrote: > On Sun, Dec 04, 2005 at 02:39:31PM -0800, Greg KH wrote: > > On Sun, Dec 04, 2005 at 06:00:49PM +0100, Jakob Oestergaard wrote: > > > In the real world, however, admins currently need to pick out specific > > > versions of the kernel for specific workloads (try running a large > > > fileserver on anything but 2.6.11.11 for example - any earlier or later > > > kernel will barf reliably. > > > > Have you filed a but at bugzilla.kernel.org about this? If not, how do > > you expect it to get fixed? > > I don't expect to get it fixed. It's futile. It can get fixed in one > version and broken two days later, and it seems the attitude is that > that is just fine. Huh? That is just not true at all. Please give us a bit more credit than that. > After a long long back-and-forth, 2.6.11 was fixed to the point where it > could reliably serve files (at least on uniprocessor configurations - > and in my setup I don't see problems on NUMA either, but as far as I > know that's just me being lucky). > > Right after that, someone thought it was a great idea to pry out the PCI > subsystem and shovel in something else. Find, that's great for a > development kernel, but for a kernel that's supposed to be stable it's > just not something you can realistically do and expect things to work > afterwards. And things broke - try mounting 10-20 XFS filesystems > simultaneously on 2.6.14. Boom - PCI errors. What PCI errors are you speaking of? We did that PCI work to fix a lot of other machines that were having problems. And yes, this did break some working machines, and we are very sorry about this. But in the future, changes to this area will not cause this to happen due to the changes made. Please file a bug in bugzilla.kernel.org and let us know you are having problems. Otherwise we have no idea. > Now what? Do I as a user upgrade my production environment to the latest > and greatest kernel experiment, hope that the problems can be fixed > quickly, and hope that I don't lose too much data in the process? No, if you rely on a production environment for your stuff, stick with a disto kernel which has been tested and is backed up by a company that will maintain it over time. > (remember I will have people unable to do their jobs whenever the file > server is down). Or do I stay on 2.6.11.11 which works on this > particular server? > > I think I stay. As you wish. > > > There are very very good reasons for offering a 'stable series' in plain > > > source-tree form - lots of admins of real-world systems need this. > > > > But it sounds like you will want different stable series depending on > > what kind of server you are running. And that will be even more work... > > The idea would be to fix the actual bugs. After a while, one could have > a kernel of higher quality with fewer bugs, making it a lot more likely > that the *same* kernel tree could be used for both workloads A and B. > > It's really very simple :) If you can point out the "actual bugs" yes. It sounds so simple from a theoritical view, but we all know how well theory works in real life sometimes... We don't try to create new bugs they are the side affect of fixing other bugs, or adding new features that other people need. And again, if those bugs aren't reported, we have no idea they are present. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 17:44 ` Greg KH @ 2005-12-06 21:16 ` Bill Davidsen 2005-12-07 14:38 ` Massimiliano Hofer 1 sibling, 0 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-06 21:16 UTC (permalink / raw) To: Greg KH, Linux Kernel Mailing List Greg KH wrote: > On Mon, Dec 05, 2005 at 04:17:53PM +0100, Jakob Oestergaard wrote: > >>On Sun, Dec 04, 2005 at 02:39:31PM -0800, Greg KH wrote: >> >>>On Sun, Dec 04, 2005 at 06:00:49PM +0100, Jakob Oestergaard wrote: >>> >>>>In the real world, however, admins currently need to pick out specific >>>>versions of the kernel for specific workloads (try running a large >>>>fileserver on anything but 2.6.11.11 for example - any earlier or later >>>>kernel will barf reliably. >>> >>>Have you filed a but at bugzilla.kernel.org about this? If not, how do >>>you expect it to get fixed? >> >>I don't expect to get it fixed. It's futile. It can get fixed in one >>version and broken two days later, and it seems the attitude is that >>that is just fine. > > > Huh? That is just not true at all. Please give us a bit more credit > than that. > > >>After a long long back-and-forth, 2.6.11 was fixed to the point where it >>could reliably serve files (at least on uniprocessor configurations - >>and in my setup I don't see problems on NUMA either, but as far as I >>know that's just me being lucky). >> >>Right after that, someone thought it was a great idea to pry out the PCI >>subsystem and shovel in something else. Find, that's great for a >>development kernel, but for a kernel that's supposed to be stable it's >>just not something you can realistically do and expect things to work >>afterwards. And things broke - try mounting 10-20 XFS filesystems >>simultaneously on 2.6.14. Boom - PCI errors. > > > What PCI errors are you speaking of? We did that PCI work to fix a lot > of other machines that were having problems. And yes, this did break > some working machines, and we are very sorry about this. But in the > future, changes to this area will not cause this to happen due to the > changes made. I don't think it's reasonable to get overly upset about *accidental* breakages. People make mistakes, otherwise you don't get progress. Note that I haven't changed my mind about deliberately removing features for which there is no practical alternative. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 17:44 ` Greg KH 2005-12-06 21:16 ` Bill Davidsen @ 2005-12-07 14:38 ` Massimiliano Hofer 2005-12-07 16:05 ` Horst von Brand 1 sibling, 1 reply; 639+ messages in thread From: Massimiliano Hofer @ 2005-12-07 14:38 UTC (permalink / raw) To: linux-kernel On Tuesday 6 December 2005 6:44 pm, Greg KH wrote: > > Now what? Do I as a user upgrade my production environment to the latest > > and greatest kernel experiment, hope that the problems can be fixed > > quickly, and hope that I don't lose too much data in the process? > > No, if you rely on a production environment for your stuff, stick with a > disto kernel which has been tested and is backed up by a company that > will maintain it over time. If the purpose of not having a 2.7 branch or longer RCs is to have people test the latest vanilla, you can't simultaneously send users away. I maintain a number of servers and don't like to depend on a distro for the kernel. I do my tests before deployment and can live with some problems in a specific release (noone is perfect), but I'd like to have a plan B without creating my own branch. Having security patches in a 2.6.(x-1).y would allow me to test the latest vanilla AND have stable production servers without the rush that usually accompanies a new release followed by a vulnerability. -- Bye, Massimiliano Hofer ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-07 14:38 ` Massimiliano Hofer @ 2005-12-07 16:05 ` Horst von Brand 2005-12-07 16:29 ` Massimiliano Hofer 0 siblings, 1 reply; 639+ messages in thread From: Horst von Brand @ 2005-12-07 16:05 UTC (permalink / raw) To: Massimiliano Hofer; +Cc: linux-kernel Massimiliano Hofer <max@bbs.cc.uniud.it> wrote: [...] > I maintain a number of servers and don't like to depend on a distro for > the kernel. I do my tests before deployment and can live with some > problems in a specific release (noone is perfect), but I'd like to have a > plan B without creating my own branch. Reasonable. > Having security patches in a 2.6.(x-1).y would allow me to test the > latest vanilla AND have stable production servers without the rush that > usually accompanies a new release followed by a vulnerability. You can certainly keep 2.6.x.y for a while when 2.6.(x+1) shows up, and even wait for 2.6.(x+1).1. Note that the stable series maintainers are sypmathetic to the idea of doing a last 2.6.x.(y+1), flushing the queued patches when 2.6.(x+1) shows up. Is this enough for you? -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-07 16:05 ` Horst von Brand @ 2005-12-07 16:29 ` Massimiliano Hofer 0 siblings, 0 replies; 639+ messages in thread From: Massimiliano Hofer @ 2005-12-07 16:29 UTC (permalink / raw) To: linux-kernel On Wednesday 7 December 2005 5:05 pm, Horst von Brand wrote: > You can certainly keep 2.6.x.y for a while when 2.6.(x+1) shows up, and > even wait for 2.6.(x+1).1. Note that the stable series maintainers are > sypmathetic to the idea of doing a last 2.6.x.(y+1), flushing the queued > patches when 2.6.(x+1) shows up. Is this enough for you? If a 2.6.x.1 is released and a vulnerability is discovered with the wrong timing, this leaves us with a kernel that has had little or no testing. We already had a 2.6.x that didn't even boot on half my servers. When 2.6.x.1 is the first bootable version and a security patch arrives, this leaves me with an uncomfortable choice between an old, stable and vulnerable version and a new, shiny and untested one. Having 2.6.x-1.y and 2.6.x.y would avoid this situation. -- Bye, Massimiliano Hofer ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 20:19 ` Greg KH ` (4 preceding siblings ...) 2005-12-04 17:00 ` Jakob Oestergaard @ 2005-12-05 14:48 ` Florian Weimer 2005-12-06 17:46 ` Greg KH 5 siblings, 1 reply; 639+ messages in thread From: Florian Weimer @ 2005-12-05 14:48 UTC (permalink / raw) To: Greg KH; +Cc: Jesper Juhl, Adrian Bunk, linux-kernel * Greg KH: > On Sat, Dec 03, 2005 at 03:29:54PM +0100, Jesper Juhl wrote: >> >> Why can't this be done by distributors/vendors? > > It already is done by these people, look at the "enterprise" Linux > distributions and their 5 years of maintance (or whatever the number > is.) > > If people/customers want stability, they already have this option. It seems that vendor kernels lack most DoS-related fixes. I'm only aware of a single vendor which tracks them to the point that CVE names are assigned. Vendor kernels are not a panacea, either. With some of the basic support contracts (in the four-figure range per year and CPU), the vendor won't look extensively at random kernel crashes which could (in theory) be attributed to faulty hardware, *and* you don't get community support for these heavily patched kernel collages. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 14:48 ` Florian Weimer @ 2005-12-06 17:46 ` Greg KH 0 siblings, 0 replies; 639+ messages in thread From: Greg KH @ 2005-12-06 17:46 UTC (permalink / raw) To: Florian Weimer; +Cc: Jesper Juhl, Adrian Bunk, linux-kernel On Mon, Dec 05, 2005 at 03:48:06PM +0100, Florian Weimer wrote: > * Greg KH: > > > On Sat, Dec 03, 2005 at 03:29:54PM +0100, Jesper Juhl wrote: > >> > >> Why can't this be done by distributors/vendors? > > > > It already is done by these people, look at the "enterprise" Linux > > distributions and their 5 years of maintance (or whatever the number > > is.) > > > > If people/customers want stability, they already have this option. > > It seems that vendor kernels lack most DoS-related fixes. I'm only > aware of a single vendor which tracks them to the point that CVE names > are assigned. If those DoS-related problems are only related to local user DoS's, yes, that is understandable. If you have questions about this, ask the vendor, they usually have a good reason for not including those fixes. > Vendor kernels are not a panacea, either. With some of the basic > support contracts (in the four-figure range per year and CPU), the > vendor won't look extensively at random kernel crashes which could (in > theory) be attributed to faulty hardware, *and* you don't get > community support for these heavily patched kernel collages. Yes, the lack of community support is one thing you do give up with them. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 13:56 RFC: Starting a stable kernel series off the 2.6 kernel Adrian Bunk 2005-12-03 14:29 ` Jesper Juhl @ 2005-12-03 14:31 ` Ben Collins 2005-12-03 19:35 ` Adrian Bunk 2005-12-05 23:03 ` Bill Davidsen 2005-12-03 14:36 ` Arjan van de Ven ` (5 subsequent siblings) 7 siblings, 2 replies; 639+ messages in thread From: Ben Collins @ 2005-12-03 14:31 UTC (permalink / raw) To: Adrian Bunk; +Cc: linux-kernel On Sat, 2005-12-03 at 14:56 +0100, Adrian Bunk wrote: > The current kernel development model is pretty good for people who > always want to use or offer their costumers the maximum amount of the > latest bugs^Wfeatures without having to resort on additional patches for > them. > > Problems of the current development model from a user's point of view > are: > - many regressions in every new release > - kernel updates often require updates for the kernel-related userspace > (e.g. for udev or the pcmcia tools switch) > > One problem following from this is that people continue to use older > kernels with known security holes because the amount of work for kernel > upgrades is too high. What you're suggesting sounds just like going back to the old style of development where 2.<even>.x is stable, and 2.<odd>.x is development. You might as well just suggest that after 2.6.16, we fork to 2.7.0, and 2.6.17+ will be stable increments like we always used to do. You're just munging the version scheme :) -- Ben Collins <ben.collins@ubuntu.com> Developer Ubuntu Linux ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 14:31 ` Ben Collins @ 2005-12-03 19:35 ` Adrian Bunk 2005-12-03 19:57 ` Lee Revell 2005-12-05 23:03 ` Bill Davidsen 1 sibling, 1 reply; 639+ messages in thread From: Adrian Bunk @ 2005-12-03 19:35 UTC (permalink / raw) To: Ben Collins; +Cc: linux-kernel On Sat, Dec 03, 2005 at 09:31:03AM -0500, Ben Collins wrote: > On Sat, 2005-12-03 at 14:56 +0100, Adrian Bunk wrote: > > The current kernel development model is pretty good for people who > > always want to use or offer their costumers the maximum amount of the > > latest bugs^Wfeatures without having to resort on additional patches for > > them. > > > > Problems of the current development model from a user's point of view > > are: > > - many regressions in every new release > > - kernel updates often require updates for the kernel-related userspace > > (e.g. for udev or the pcmcia tools switch) > > > > One problem following from this is that people continue to use older > > kernels with known security holes because the amount of work for kernel > > upgrades is too high. > > What you're suggesting sounds just like going back to the old style of > development where 2.<even>.x is stable, and 2.<odd>.x is development. > You might as well just suggest that after 2.6.16, we fork to 2.7.0, and > 2.6.17+ will be stable increments like we always used to do. > > You're just munging the version scheme :) The 2.6.17+ development model is different from a traditional 2.7 development model in the sense that 2.6.17+ contains regular relatively stable releases. But yes, what I suggest is partially a step back in a way that it doesn't conflict with the current 2.6.17+ development model. Well, if taking the best from the old style development is improving things that isn't something bad. cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 19:35 ` Adrian Bunk @ 2005-12-03 19:57 ` Lee Revell 2005-12-03 21:04 ` M. 2005-12-03 22:58 ` Matthias Andree 0 siblings, 2 replies; 639+ messages in thread From: Lee Revell @ 2005-12-03 19:57 UTC (permalink / raw) To: Adrian Bunk; +Cc: Ben Collins, linux-kernel On Sat, 2005-12-03 at 20:35 +0100, Adrian Bunk wrote: > > But yes, what I suggest is partially a step back in a way that it > doesn't conflict with the current 2.6.17+ development model. > > Well, if taking the best from the old style development is improving > things that isn't something bad. You seem to be saying that the current development model is unacceptable for users for whom older kernel work just fine, and the main risk in upgrading is regression. But the new development model is clearly needed for those users whose needs are not met by the old kernel, say due to unacceptable soft RT performance or unsupported hardware. But it's wrong to try to evenly balance the needs of these two classes of users, because the first class has another option - they can stick with the old kernel that works for them. The second class of users has no other option, so their needs should be given greater weight. Lee ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 19:57 ` Lee Revell @ 2005-12-03 21:04 ` M. 2005-12-03 22:58 ` Matthias Andree 1 sibling, 0 replies; 639+ messages in thread From: M. @ 2005-12-03 21:04 UTC (permalink / raw) To: linux-kernel; +Cc: Adrian Bunk, Ben Collins On 12/3/05, Lee Revell <rlrevell@joe-job.com> wrote: > On Sat, 2005-12-03 at 20:35 +0100, Adrian Bunk wrote: > > > > But yes, what I suggest is partially a step back in a way that it > > doesn't conflict with the current 2.6.17+ development model. > > > > Well, if taking the best from the old style development is improving > > things that isn't something bad. > > You seem to be saying that the current development model is unacceptable > for users for whom older kernel work just fine, and the main risk in > upgrading is regression. But the new development model is clearly > needed for those users whose needs are not met by the old kernel, say > due to unacceptable soft RT performance or unsupported hardware. > > But it's wrong to try to evenly balance the needs of these two classes > of users, because the first class has another option - they can stick > with the old kernel that works for them. The second class of users has > no other option, so their needs should be given greater weight. > > Lee > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > Mhhh something much simpler: Use the current development model between 2.6.N and 2.6.N+1. This means the current dev model would be used like NOW, BUT changing the versioning scheme in a way which reflects the 6months release plans. TRYING TO EXPLAIN THINGS BETTER: You could skip the version scheme change and do something like: EXAMPLE: 2.6.14 - ok let's plan 3/4/whatever releases in the next 6 months with those major new features: - A - B - C then: .15, .16 like every 2.6 kernel, same dev model, same everything (erhm not every just like the latest releases, ok) .18 would be MAINLY stabilization: no device model changes, no networking change, and so on (but i think it would be ok new drivers since they cant be disabled or simply not used) and .19 would come out months after the .14 release it's not a stable/devel or 2.6/2.7 or better 2.4/2.5 scheme (even a 2.6/2.7 scheme almost exists with main tree/-mm tree atm) or old stuff under a different point of view BUT it's the SAME current dev model applied to every release EXCEPT for the last one BEFORE the 6months timeline. what does it means: no one says nothing is unacceptable, it means things could be better for distros/users/external kernel projects by doing something that could be called a two-layers model: -------------------- Linus & co. "hey, let's write down the major features for the next 6 months and plan things a little" -------------------- Other kernel developers "I would like to get in X, Y and Z. Let's do it 2.6.x style." -------------------- The changed versioning scheme just reflects the 6months release ciclye idea. fullstop. trying to clarify again: > What you're suggesting sounds just like going back to the old style of > development where 2.<even>.x is stable, and 2.<odd>.x is development. > You might as well just suggest that after 2.6.16, we fork to 2.7.0, and > 2.6.17+ will be stable increments like we always used to do. every 2.6.N.x release would be the SAME as a 2.6.x release. The entire 6months cycle puts in some time/features/stability boundaries on top of the existent model. Finally, those stability boundaries would be planned every 6months assuring the best fit of the model with developers, distros and users needs. hope I clarified, Michele ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 19:57 ` Lee Revell 2005-12-03 21:04 ` M. @ 2005-12-03 22:58 ` Matthias Andree 2005-12-03 23:49 ` Lee Revell 2005-12-05 21:00 ` Florian Weimer 1 sibling, 2 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-03 22:58 UTC (permalink / raw) To: linux-kernel On Sat, 03 Dec 2005, Lee Revell wrote: > You seem to be saying that the current development model is unacceptable > for users for whom older kernel work just fine, and the main risk in > upgrading is regression. But the new development model is clearly > needed for those users whose needs are not met by the old kernel, say > due to unacceptable soft RT performance or unsupported hardware. > > But it's wrong to try to evenly balance the needs of these two classes > of users, because the first class has another option - they can stick > with the old kernel that works for them. The second class of users has The point that just escaped you as the motivation for this thread was the availability of security (or other critical) fixes for older kernels. It would all be fine if, say, the fix for CVE-2004-2492 were available for those who find 2.6.8 works for them (the fix went into 2.6.14 BTW), and the concern is the development model isn't fit to accomodate needs like this. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:58 ` Matthias Andree @ 2005-12-03 23:49 ` Lee Revell 2005-12-05 21:05 ` Florian Weimer 2005-12-05 21:00 ` Florian Weimer 1 sibling, 1 reply; 639+ messages in thread From: Lee Revell @ 2005-12-03 23:49 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel On Sat, 2005-12-03 at 23:58 +0100, Matthias Andree wrote: > On Sat, 03 Dec 2005, Lee Revell wrote: > > > You seem to be saying that the current development model is unacceptable > > for users for whom older kernel work just fine, and the main risk in > > upgrading is regression. But the new development model is clearly > > needed for those users whose needs are not met by the old kernel, say > > due to unacceptable soft RT performance or unsupported hardware. > > > > But it's wrong to try to evenly balance the needs of these two classes > > of users, because the first class has another option - they can stick > > with the old kernel that works for them. The second class of users has > > The point that just escaped you as the motivation for this thread was > the availability of security (or other critical) fixes for older > kernels. It would all be fine if, say, the fix for CVE-2004-2492 were > available for those who find 2.6.8 works for them (the fix went into > 2.6.14 BTW), and the concern is the development model isn't fit to > accomodate needs like this. > If you want security fixes backported then you can get a distro kernel. Lee ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 23:49 ` Lee Revell @ 2005-12-05 21:05 ` Florian Weimer 2005-12-05 21:41 ` Lee Revell 0 siblings, 1 reply; 639+ messages in thread From: Florian Weimer @ 2005-12-05 21:05 UTC (permalink / raw) To: Lee Revell; +Cc: Matthias Andree, linux-kernel * Lee Revell: >> The point that just escaped you as the motivation for this thread was >> the availability of security (or other critical) fixes for older >> kernels. It would all be fine if, say, the fix for CVE-2004-2492 were >> available for those who find 2.6.8 works for them (the fix went into >> 2.6.14 BTW), and the concern is the development model isn't fit to >> accomodate needs like this. >> > > If you want security fixes backported then you can get a distro kernel. And these distro kernels appear magically from nowhere? ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 21:05 ` Florian Weimer @ 2005-12-05 21:41 ` Lee Revell 2005-12-05 23:00 ` Florian Weimer 0 siblings, 1 reply; 639+ messages in thread From: Lee Revell @ 2005-12-05 21:41 UTC (permalink / raw) To: Florian Weimer; +Cc: Matthias Andree, linux-kernel On Mon, 2005-12-05 at 22:05 +0100, Florian Weimer wrote: > * Lee Revell: > > >> The point that just escaped you as the motivation for this thread was > >> the availability of security (or other critical) fixes for older > >> kernels. It would all be fine if, say, the fix for CVE-2004-2492 were > >> available for those who find 2.6.8 works for them (the fix went into > >> 2.6.14 BTW), and the concern is the development model isn't fit to > >> accomodate needs like this. > >> > > > > If you want security fixes backported then you can get a distro kernel. > > And these distro kernels appear magically from nowhere? > No you get them from Red Hat or SuSE or whoever. One of the core assumptions of the new development model is that distros whose business model involves paying people to do QA and regression testing and have access to bug reports from zillions of users are better positioned than kernel developers to decide what a "stable" kernel is. Lee ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 21:41 ` Lee Revell @ 2005-12-05 23:00 ` Florian Weimer 2005-12-05 23:06 ` Bernd Petrovitsch 0 siblings, 1 reply; 639+ messages in thread From: Florian Weimer @ 2005-12-05 23:00 UTC (permalink / raw) To: Lee Revell; +Cc: Matthias Andree, linux-kernel * Lee Revell: > On Mon, 2005-12-05 at 22:05 +0100, Florian Weimer wrote: >> * Lee Revell: >> >> >> The point that just escaped you as the motivation for this thread was >> >> the availability of security (or other critical) fixes for older >> >> kernels. It would all be fine if, say, the fix for CVE-2004-2492 were >> >> available for those who find 2.6.8 works for them (the fix went into >> >> 2.6.14 BTW), and the concern is the development model isn't fit to >> >> accomodate needs like this. >> >> >> > >> > If you want security fixes backported then you can get a distro kernel. >> >> And these distro kernels appear magically from nowhere? >> > > No you get them from Red Hat or SuSE or whoever. "Whoever"? Debian? Slackware? Gentoo? Even companies like SGI might have difficulties providing security support for their custom kernels, not to speak of tons of embedded developers. Can I buy security support for my custom MIPS kernel, like I can buy GCC support for the platform? Is there a similar market? > One of the core assumptions of the new development model is that > distros whose business model involves paying people to do QA and > regression testing and have access to bug reports from zillions of > users are better positioned than kernel developers to decide what a > "stable" kernel is. But they aren't more qualified when it comes to extracting security fixes (and other critical bug fixes). For picking functionality, I agree, but critical bug fixes which basically affect everone are a different matter. It doesn't make sense to redo the same analysis over and over again, at each vendor. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 23:00 ` Florian Weimer @ 2005-12-05 23:06 ` Bernd Petrovitsch 2005-12-06 0:08 ` Florian Weimer 0 siblings, 1 reply; 639+ messages in thread From: Bernd Petrovitsch @ 2005-12-05 23:06 UTC (permalink / raw) To: Florian Weimer; +Cc: Lee Revell, Matthias Andree, linux-kernel On Tue, 2005-12-06 at 00:00 +0100, Florian Weimer wrote: [...] > fixes (and other critical bug fixes). For picking functionality, I > agree, but critical bug fixes which basically affect everone are a > different matter. It doesn't make sense to redo the same analysis > over and over again, at each vendor. Then vendors should cooperate/collaborate. Where's the problem? Bernd -- Firmix Software GmbH http://www.firmix.at/ mobil: +43 664 4416156 fax: +43 1 7890849-55 Embedded Linux Development and Services ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 23:06 ` Bernd Petrovitsch @ 2005-12-06 0:08 ` Florian Weimer 0 siblings, 0 replies; 639+ messages in thread From: Florian Weimer @ 2005-12-06 0:08 UTC (permalink / raw) To: Bernd Petrovitsch; +Cc: Lee Revell, Matthias Andree, linux-kernel * Bernd Petrovitsch: > On Tue, 2005-12-06 at 00:00 +0100, Florian Weimer wrote: > [...] >> fixes (and other critical bug fixes). For picking functionality, I >> agree, but critical bug fixes which basically affect everone are a >> different matter. It doesn't make sense to redo the same analysis >> over and over again, at each vendor. > > Then vendors should cooperate/collaborate. Where's the problem? Usually, publicly visisble security bug handling is not separated from the main development effort, especially if there is already a centralized team for that purpose. It's also a waste of resources if someone with no detailed knowledge of the first analysis (which was made when the bug was fixed) or the source code in question has to redo the whole analysis, just to pick up the correct patches and classify the vulnerability. If you duplicate the work just once, things are a bit better, but it's still a waste of resources, and people not familiar with the code tend to make more mistakes. It's not that there isn't any cooperation, either. As far as I can tell, it's possible to get most insider know-how on vulnerabilities once it is published. It's just more time-consuming than necessary. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:58 ` Matthias Andree 2005-12-03 23:49 ` Lee Revell @ 2005-12-05 21:00 ` Florian Weimer 2005-12-05 21:06 ` Arjan van de Ven 1 sibling, 1 reply; 639+ messages in thread From: Florian Weimer @ 2005-12-05 21:00 UTC (permalink / raw) To: linux-kernel * Matthias Andree: > The point that just escaped you as the motivation for this thread was > the availability of security (or other critical) fixes for older > kernels. It would all be fine if, say, the fix for CVE-2004-2492 were > available for those who find 2.6.8 works for them (the fix went into > 2.6.14 BTW), and the concern is the development model isn't fit to > accomodate needs like this. Well, if there's a CVE name, the proper patch isn't *that* far away (someone has already done a bit of work to isolate the fix). The real issue seems to be how to make sure that CVE names are assigned during the kernel development process (and not just as an afterthought by the security folks). ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 21:00 ` Florian Weimer @ 2005-12-05 21:06 ` Arjan van de Ven 2005-12-06 0:43 ` Florian Weimer 0 siblings, 1 reply; 639+ messages in thread From: Arjan van de Ven @ 2005-12-05 21:06 UTC (permalink / raw) To: Florian Weimer; +Cc: linux-kernel On Mon, 2005-12-05 at 22:00 +0100, Florian Weimer wrote: > * Matthias Andree: > > > The point that just escaped you as the motivation for this thread was > > the availability of security (or other critical) fixes for older > > kernels. It would all be fine if, say, the fix for CVE-2004-2492 were > > available for those who find 2.6.8 works for them (the fix went into > > 2.6.14 BTW), and the concern is the development model isn't fit to > > accomodate needs like this. > > Well, if there's a CVE name, the proper patch isn't *that* far away > (someone has already done a bit of work to isolate the fix). The real > issue seems to be how to make sure that CVE names are assigned during > the kernel development process (and not just as an afterthought by the > security folks). security@kernel.org works that way already in a way. Of course it'd be nice to add a cve name while coding the security hole into the kernel, but nobody is that clearvoyant ;) The hardest part is actually knowing which versions are affected, especially when the code no longer quite is the same, the locking rules got cleaned up in later versions (which means the older kernel was a mess, so you're always looking at more messy code than the "new" kernel). For some stuff this is easy. For other stuff it for sure isn't, especially if you want to keep api/abi compatibility, or at least low impact. Some security fixes just are invasive. Those are rare, maybe 2 or 3 times a year or so. But they do exist... unfortunate as it is. The irony is that doing a "hacky" less invasive risk actually may introduce more risk to stability than doing the full invasive fix. Nasty trade-offs there.... ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 21:06 ` Arjan van de Ven @ 2005-12-06 0:43 ` Florian Weimer 2005-12-06 11:21 ` Matthias Andree 2005-12-06 20:35 ` Alan Cox 0 siblings, 2 replies; 639+ messages in thread From: Florian Weimer @ 2005-12-06 0:43 UTC (permalink / raw) To: Arjan van de Ven; +Cc: linux-kernel * Arjan van de Ven: >> Well, if there's a CVE name, the proper patch isn't *that* far away >> (someone has already done a bit of work to isolate the fix). The real >> issue seems to be how to make sure that CVE names are assigned during >> the kernel development process (and not just as an afterthought by the >> security folks). > > security@kernel.org works that way already in a way. As far as I know, many of the recent CVE assignments for kernel vulnerabilities have been done by MITRE, requested by individuals which are neither known as kernel developers, nor vendor security folks (for "vendor" as in "we have our own legal department with real lawyers"). Maybe the source of CVE assignments paints a wrong picture. But if the CVE picture is correct, vendor-paid kernel developers help behind the scenes, but there is little interest in openly documenting security issues, so that users (and what kernel.org considers fringe distros) can apply the relevant patches if they use kernel.org kernels. >From a vendor POV, the lack of official kernel.org advisories may be a feature. I find it rather disturbing, and I'm puzzled that the kernel developer community doesn't view this a problem. I know I'm alone, and there are certainly part-time security guys who would be willing join forces to create something like a kernel.org security bug database. But the only answers we get is that everything is fine, vendors handle the situation, security@kernel.org actually does this already, etc. > The hardest part is actually knowing which versions are affected, First, you need to know that the patch plugs a security hole. 8-) This isn't always obvious based on the patch, even if the fact is known to the comitter. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 0:43 ` Florian Weimer @ 2005-12-06 11:21 ` Matthias Andree 2005-12-06 15:10 ` Florian Weimer ` (2 more replies) 2005-12-06 20:35 ` Alan Cox 1 sibling, 3 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-06 11:21 UTC (permalink / raw) To: linux-kernel On Tue, 06 Dec 2005, Florian Weimer wrote: > From a vendor POV, the lack of official kernel.org advisories may be a > feature. I find it rather disturbing, and I'm puzzled that the kernel > developer community doesn't view this a problem. I know I'm alone, You're not alone in viewing this as a problem, but QA is a burden kernel developers are not interested in. But it is necessary. QA has to happen at all levels if it is supposed to be affordable or scalable. The development process was scaled up, but QA wasn't. How about the Signed-off-by: lines? Those people who pass on the changes also pass on the bugs, and they are responsible for the code - not only license-wise, but also quality-wise. That's the latest point where regression tests MUST happen. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 11:21 ` Matthias Andree @ 2005-12-06 15:10 ` Florian Weimer 2005-12-06 16:45 ` Dmitry Torokhov 2005-12-10 0:22 ` Rob Landley 2 siblings, 0 replies; 639+ messages in thread From: Florian Weimer @ 2005-12-06 15:10 UTC (permalink / raw) To: linux-kernel * Matthias Andree: > On Tue, 06 Dec 2005, Florian Weimer wrote: > >> From a vendor POV, the lack of official kernel.org advisories may be a >> feature. I find it rather disturbing, and I'm puzzled that the kernel >> developer community doesn't view this a problem. I know I'm alone, > > You're not alone in viewing this as a problem, I know, it's a typo. > How about the Signed-off-by: lines? Those people who pass on the changes > also pass on the bugs, and they are responsible for the code - not only > license-wise, but also quality-wise. That's the latest point where > regression tests MUST happen. There are critical kernel parts for which automated regression testing is very hard. In some twisted sense, regression testing ist best done by those who run real applications, i.e. end users. The interesting thing is that you end up with reasonably stable software this way, except in a few corner cases. The main point of debate seems to be how relevant the corner cases are, and how much general kernel development should care about them. (And no, not everyone in such a corner has $$,$$$ to spend.) ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 11:21 ` Matthias Andree 2005-12-06 15:10 ` Florian Weimer @ 2005-12-06 16:45 ` Dmitry Torokhov 2005-12-07 11:29 ` Matthias Andree 2005-12-10 0:22 ` Rob Landley 2 siblings, 1 reply; 639+ messages in thread From: Dmitry Torokhov @ 2005-12-06 16:45 UTC (permalink / raw) To: linux-kernel On 12/6/05, Matthias Andree <matthias.andree@gmx.de> wrote: > > QA has to happen at all levels if it is supposed to be affordable or > scalable. The development process was scaled up, but QA wasn't. > > How about the Signed-off-by: lines? Those people who pass on the changes > also pass on the bugs, and they are responsible for the code - not only > license-wise, but also quality-wise. That's the latest point where > regression tests MUST happen. People who pass the changes can only test ones they have hardware for. For the rest they can try to validate the code by reading patches but have to rely on the submitter wrt to the patch actually working. -- Dmitry ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 16:45 ` Dmitry Torokhov @ 2005-12-07 11:29 ` Matthias Andree 2005-12-07 13:54 ` Horst von Brand 2005-12-08 3:29 ` Dmitry Torokhov 0 siblings, 2 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-07 11:29 UTC (permalink / raw) To: linux-kernel On Tue, 06 Dec 2005, Dmitry Torokhov wrote: > On 12/6/05, Matthias Andree <matthias.andree@gmx.de> wrote: > > > > QA has to happen at all levels if it is supposed to be affordable or > > scalable. The development process was scaled up, but QA wasn't. > > > > How about the Signed-off-by: lines? Those people who pass on the changes > > also pass on the bugs, and they are responsible for the code - not only > > license-wise, but also quality-wise. That's the latest point where > > regression tests MUST happen. > > People who pass the changes can only test ones they have hardware for. > For the rest they can try to validate the code by reading patches but > have to rely on the submitter wrt to the patch actually working. What I'm saying is that people (maintainer) should have a selected number of people (users) test the patches before they are merged. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-07 11:29 ` Matthias Andree @ 2005-12-07 13:54 ` Horst von Brand 2005-12-08 3:29 ` Dmitry Torokhov 1 sibling, 0 replies; 639+ messages in thread From: Horst von Brand @ 2005-12-07 13:54 UTC (permalink / raw) To: linux-kernel Matthias Andree <matthias.andree@gmx.de> wrote: > On Tue, 06 Dec 2005, Dmitry Torokhov wrote: > > On 12/6/05, Matthias Andree <matthias.andree@gmx.de> wrote: > > > QA has to happen at all levels if it is supposed to be affordable or > > > scalable. The development process was scaled up, but QA wasn't. > > > How about the Signed-off-by: lines? Those people who pass on the changes > > > also pass on the bugs, and they are responsible for the code - not only > > > license-wise, but also quality-wise. That's the latest point where > > > regression tests MUST happen. > > People who pass the changes can only test ones they have hardware for. > > For the rest they can try to validate the code by reading patches but > > have to rely on the submitter wrt to the patch actually working. > What I'm saying is that people (maintainer) should have a selected > number of people (users) test the patches before they are merged. Each one gets the patches from people who tested them on their machines (presumably), and the maintainer signs them off for code decency and (if he has the hardware) working on his machine too. It is very hard to get much more than that in terms of early testers. Again, one of the standard complaints here is that very few test the -rcX kernels, and then scream murder when the -final breaks. Now consider the case of proposed patches... Yet, the solution is /very/ simple: Instead of complaining, set up a machine with the hardware that particularly interests you, get on the relevant lists, find out where the proposed patches are kept and test them. You could even add a Signed-off-by: line to the patches you check out. And again, "somebody should do..." won't get you anywhere, "here I do..." has more of a chance of success. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-07 11:29 ` Matthias Andree 2005-12-07 13:54 ` Horst von Brand @ 2005-12-08 3:29 ` Dmitry Torokhov 2005-12-08 8:29 ` Matthias Andree 1 sibling, 1 reply; 639+ messages in thread From: Dmitry Torokhov @ 2005-12-08 3:29 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel On Wednesday 07 December 2005 06:29, Matthias Andree wrote: > On Tue, 06 Dec 2005, Dmitry Torokhov wrote: > > > On 12/6/05, Matthias Andree <matthias.andree@gmx.de> wrote: > > > > > > QA has to happen at all levels if it is supposed to be affordable or > > > scalable. The development process was scaled up, but QA wasn't. > > > > > > How about the Signed-off-by: lines? Those people who pass on the changes > > > also pass on the bugs, and they are responsible for the code - not only > > > license-wise, but also quality-wise. That's the latest point where > > > regression tests MUST happen. > > > > People who pass the changes can only test ones they have hardware for. > > For the rest they can try to validate the code by reading patches but > > have to rely on the submitter wrt to the patch actually working. > > What I'm saying is that people (maintainer) should have a selected > number of people (users) test the patches before they are merged. > And we try. Take for example psmouse_resync patch that is now in -mm. I got about 30 reports that it worked and fixed people's problems before I got it to Andrew. And still as soon as it got to -mm I got a complaint that it failed on one of boxes ;( -- Dmitry ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-08 3:29 ` Dmitry Torokhov @ 2005-12-08 8:29 ` Matthias Andree 0 siblings, 0 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-08 8:29 UTC (permalink / raw) To: Dmitry Torokhov; +Cc: Matthias Andree, linux-kernel On Wed, 07 Dec 2005, Dmitry Torokhov wrote: > On Wednesday 07 December 2005 06:29, Matthias Andree wrote: > > What I'm saying is that people (maintainer) should have a selected > > number of people (users) test the patches before they are merged. > > And we try. Take for example psmouse_resync patch that is now in -mm. > I got about 30 reports that it worked and fixed people's problems before > I got it to Andrew. And still as soon as it got to -mm I got a complaint > that it failed on one of boxes ;( The important thing is to get these failing boxes into the regular test set. I know that's not always easy, because end users tend to go away as soon as it works again for them. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 11:21 ` Matthias Andree 2005-12-06 15:10 ` Florian Weimer 2005-12-06 16:45 ` Dmitry Torokhov @ 2005-12-10 0:22 ` Rob Landley 2 siblings, 0 replies; 639+ messages in thread From: Rob Landley @ 2005-12-10 0:22 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel On Tuesday 06 December 2005 05:21, Matthias Andree wrote: > On Tue, 06 Dec 2005, Florian Weimer wrote: > > From a vendor POV, the lack of official kernel.org advisories may be a > > feature. I find it rather disturbing, and I'm puzzled that the kernel > > developer community doesn't view this a problem. I know I'm alone, > > You're not alone in viewing this as a problem, but QA is a burden kernel > developers are not interested in. But it is necessary. If you want to run a big automated regression test against the kernel, exercising the full API and immediately catching any regressions, go right ahead. Nobody's stopping you and you don't need our permission anyway. The Linux Test Project is working on something like this already, and ODSL does some of this to. (It's not like QA is being ignored.) The problem is that the bulk of the kernel code is device drivers, and nobody has all the strange and esoteric hardware that the drivers push. Nope, not even IBM. I doubt any one organization anywhere on the planet has _everything_ the kernel has been used to drive. . > QA has to happen at all levels if it is supposed to be affordable or > scalable. The development process was scaled up, but QA wasn't. > > How about the Signed-off-by: lines? Those people who pass on the changes > also pass on the bugs, and they are responsible for the code - not only > license-wise, but also quality-wise. That's the latest point where > regression tests MUST happen. I can't test your setup for you. I haven't got your setup. All I can tell you is that it worked for me. I spent most of a week last month fighting to get User Mode Linux 2.6.15-rc1 through rc4 to compile and run on both x86 ubuntu and x86-64 PLD. Different versions of GCC compiled the darn interface code differently (there's a section where it switches stacks and gcc kept trying to touch the stack in the middle of this, and segfaulting). Worked fine for Jeff Dike and Blaisorblade, because they weren't using a semi-obsolete version of ubuntu. Over on PLD, I had a fight just to get it to _compile_, because the header files were all different (PLD uses Mazur's cleaned up 2.6 headers which uClibc systems also use, while most things use the glibc package, and at one point they had userspace and kernel space headers reversed and it worked fine with the glibc kernel headers but Mazur's headers really are cleaned up and don't leak nearly so much kernel stuff into userspace). And then /lib wasn't a symlink to /lib64 (it is on Fedora and Debian, but on PLD they're separate directories) so the link path had to be adjusted (/lib64 was the correct directory for a 64-bit build and should be checked first). Then getting it to run had another half-dozen problems with various interface code: for some reason on PLD page_size was linked as a function call when they expected it to be a constant... Another fun little thing is just a performance issue: UML gets its "physical memory" from an mmap file (easy to share between processes), but if that file isn't on tmpfs then every page UML dirties gets scheduled for writeout, over and over again, keeping the hard drive constantly busy and slowing the system to a crawl. Of course it _works_, but so it's hard to pin down what the problem is. (UML isn't slowed down, the rest of the system is by the unnecessary I/O.) Again, on Jeff's system /tmp is a tmpfs mount. On most systems, /dev/shm is a tmpfs mount and /tmp inherits /. (Meaning on knoppix it's tmpfs, but on Fedora or Ubuntu or Gentoo, it isn't by default. Unless the sysadmin has changed it, which many sysadmins will.) And strangely, on the PLD system I'm borrowing /dev/shm isn't tmpfs, so changing the default is the right thing but it needed an improved error message. It all worked just _fine_ for the people who wrote it. (And continues to.) And all of this is why there was an -rc1, so people like me could try it and report that it didn't work the same way for us and spend a week figuring out all the various different _ways_ it didn't work. This isn't the full set of bugs I plowed through. I had a version at one point that ran fine, gave me a command shell (init=/bin/sh) and I reported success and then came back the next day with "nope, fork segfaults". (Actually it was exec segfaulting.) The shell _did_ come up fine. (And echo $USER hadn't actually had to exec anything...) But that wasn't the end of it. The thing is, me spending all this time making sure it worked _for_me_ was something that I did on my own time, voluntarily. I'm not really a UML developer, I have too much to do elsewhere. If I hadn't done this, would it work on ubuntu and PLD right now? Maybe. I don't know. But it already worked for Jeff Dike when he checked it in. Worked just fine. Because he didn't have the environment I had. He could find _none_ of these problems because the bugs only manifest in an environment he doesn't have. And all this is a _rounding_error_ compared to the kernel as a whole. This is just one little corner of it, in one little release, where one person spent one week debugging on just two systems. And this wasn't even hardware dependent! (Or an intermittent problem that you _think_ is fixed because you haven't seen it, or something requiring a particularly arduous reproduction sequence like a 40 hour calculation, or access to a machine that's only available thursdays from 2-4 am...) You seem to _deeply_ misunderstand nature of the problem. Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 0:43 ` Florian Weimer 2005-12-06 11:21 ` Matthias Andree @ 2005-12-06 20:35 ` Alan Cox 1 sibling, 0 replies; 639+ messages in thread From: Alan Cox @ 2005-12-06 20:35 UTC (permalink / raw) To: Florian Weimer; +Cc: Arjan van de Ven, linux-kernel On Maw, 2005-12-06 at 01:43 +0100, Florian Weimer wrote: > As far as I know, many of the recent CVE assignments for kernel > vulnerabilities have been done by MITRE, requested by individuals > which are neither known as kernel developers, nor vendor security > folks (for "vendor" as in "we have our own legal department with real > lawyers"). Most of them will be because vendors employ security professionals to handle security CVE work and do all the tedious and terribly important tracking of bugs v releases and what needs to be fixed by whom and when - and developers to write code. > Maybe the source of CVE assignments paints a wrong picture. But if > the CVE picture is correct, vendor-paid kernel developers help behind > the scenes, but there is little interest in openly documenting > security issues, so that users (and what kernel.org considers fringe > distros) can apply the relevant patches if they use kernel.org > kernels. The 2.6.x.y maintainers are directly involved in security@kernel.org last time I checked. > database. But the only answers we get is that everything is fine, > vendors handle the situation, security@kernel.org actually does this > already, etc. Having someone doing that on kernel.org sounds a good plan ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 14:31 ` Ben Collins 2005-12-03 19:35 ` Adrian Bunk @ 2005-12-05 23:03 ` Bill Davidsen 2005-12-06 1:48 ` Jeff Garzik 2005-12-06 1:56 ` Horst von Brand 1 sibling, 2 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-05 23:03 UTC (permalink / raw) To: Ben Collins; +Cc: linux-kernel Ben Collins wrote: > What you're suggesting sounds just like going back to the old style of > development where 2.<even>.x is stable, and 2.<odd>.x is development. > You might as well just suggest that after 2.6.16, we fork to 2.7.0, and > 2.6.17+ will be stable increments like we always used to do. I'll let him speak to what he intended, but my idea of stable is to keep the features of 2.6.0 in 2.6.N for any value of N. Adding new stuff rapidly hasn't been nearly the problem people feared, but that's largely due to the efforts of akpm to act as throttle, and somehow get more people to try his versions and knock the corners off the new code before it goes mainline. I do think the old model was better; by holding down major changes for six months or so after a new even release came out, people had a chance to polich the stable release, and developers had time to recharge their batteries so to speak, and to sit and think about what they wanted to do, without feeling the pressure to write code and submit it right away. Knowing that there's no place to send code for six months is a great aid to generating GOOD code. The other advantage of a development tree was that features could be added and removed without the argument that it would break this or that. It was development, no one was supposed to use it for production, no one could claim that there was even an implied promise of things working or even existing. ipchains could have gone out of 2.6 with no more fuss than xiafs departing. The people who really want it stay with the old kernel. To a large extent -mm has become the development kernel, and as neat as that is, a development model which depends on a small number of dedicated and talented people to make it work is fragile. Just my thoughts, I think we had it right before, I think it's less inherently stable now. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 23:03 ` Bill Davidsen @ 2005-12-06 1:48 ` Jeff Garzik 2005-12-06 11:23 ` Matthias Andree 2005-12-06 19:48 ` Bill Davidsen 2005-12-06 1:56 ` Horst von Brand 1 sibling, 2 replies; 639+ messages in thread From: Jeff Garzik @ 2005-12-06 1:48 UTC (permalink / raw) To: Bill Davidsen; +Cc: Ben Collins, linux-kernel Bill Davidsen wrote: > I do think the old model was better; by holding down major changes for > six months or so after a new even release came out, people had a chance > to polich the stable release, and developers had time to recharge their > batteries so to speak, and to sit and think about what they wanted to > do, without feeling the pressure to write code and submit it right away. > Knowing that there's no place to send code for six months is a great aid > to generating GOOD code. It never worked that way, which is why the model changed. Like it or not, developers would only focus on one release. In the old model, unstable things would get shoved into the stable kernel, because people didn't want to wait six months. And for the unstable kernel, it would often be so horribly broken that even developers couldn't use it for development (think 2.5.x IDE). Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 1:48 ` Jeff Garzik @ 2005-12-06 11:23 ` Matthias Andree 2005-12-06 19:48 ` Bill Davidsen 1 sibling, 0 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-06 11:23 UTC (permalink / raw) To: Jeff Garzik; +Cc: Bill Davidsen, Ben Collins, linux-kernel On Mon, 05 Dec 2005, Jeff Garzik wrote: > Bill Davidsen wrote: > >I do think the old model was better; by holding down major changes for > >six months or so after a new even release came out, people had a chance > >to polich the stable release, and developers had time to recharge their > >batteries so to speak, and to sit and think about what they wanted to > >do, without feeling the pressure to write code and submit it right away. > >Knowing that there's no place to send code for six months is a great aid > >to generating GOOD code. > > It never worked that way, which is why the model changed. > > Like it or not, developers would only focus on one release. In the old > model, unstable things would get shoved into the stable kernel, because > people didn't want to wait six months. And for the unstable kernel, it > would often be so horribly broken that even developers couldn't use it > for development (think 2.5.x IDE). So why haven't the broken patches (yes, TCQ and all that, too) been backed out at the time? -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 1:48 ` Jeff Garzik 2005-12-06 11:23 ` Matthias Andree @ 2005-12-06 19:48 ` Bill Davidsen 1 sibling, 0 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-06 19:48 UTC (permalink / raw) To: Jeff Garzik; +Cc: Ben Collins, linux-kernel Jeff Garzik wrote: > Bill Davidsen wrote: > >> I do think the old model was better; by holding down major changes for >> six months or so after a new even release came out, people had a >> chance to polich the stable release, and developers had time to >> recharge their batteries so to speak, and to sit and think about what >> they wanted to do, without feeling the pressure to write code and >> submit it right away. Knowing that there's no place to send code for >> six months is a great aid to generating GOOD code. > > > It never worked that way, which is why the model changed. > > Like it or not, developers would only focus on one release. In the old > model, unstable things would get shoved into the stable kernel, because > people didn't want to wait six months. And for the unstable kernel, it > would often be so horribly broken that even developers couldn't use it > for development (think 2.5.x IDE). I was actually thinking of Rusty's module code... I do every time I have to build an initrd file by hand "Although the syntax is similar to the older /etc/modules.conf, there are many features missing." -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 23:03 ` Bill Davidsen 2005-12-06 1:48 ` Jeff Garzik @ 2005-12-06 1:56 ` Horst von Brand 1 sibling, 0 replies; 639+ messages in thread From: Horst von Brand @ 2005-12-06 1:56 UTC (permalink / raw) To: Bill Davidsen; +Cc: Ben Collins, linux-kernel Bill Davidsen <davidsen@tmr.com> wrote: > Ben Collins wrote: > > What you're suggesting sounds just like going back to the old style of > > development where 2.<even>.x is stable, and 2.<odd>.x is development. > > You might as well just suggest that after 2.6.16, we fork to 2.7.0, and > > 2.6.17+ will be stable increments like we always used to do. > I'll let him speak to what he intended, but my idea of stable is to > keep the features of 2.6.0 in 2.6.N for any value of N. That works iff N == 0. > Adding new > stuff rapidly hasn't been nearly the problem people feared, ... because they had the leeway to change broken/unsuitable things to fit, and because the tools today are so much better... > but that's > largely due to the efforts of akpm to act as throttle, and somehow get > more people to try his versions and knock the corners off the new code > before it goes mainline. Heroic efforts, sure. > I do think the old model was better; It just /didn't work/. You don't remember the pain of jumping from 2.2 to 2.4, do you? Sure, while 2.2 lasted it was stable, but everybody screamed that the latest&greatest whatever-card didn't work, and either jumped to 2.3.x du jour (good luck! had to futz around with lots of matching userland changes /without/ distro support for that) or choose a distro which shipped a patched 2.3 kernel (which was totally incompatible when 2.4 showed up) or (tried to) backport features to 2.2 (ditto, resulting from a /huge/ amount of wasted effort). > by holding down major changes for > six months or so after a new even release came out, people had a > chance to polich the stable release, No chance. The people who would have been doing so just got bored and looked elsewhere for challenges. Do enough of that, and you'll be left without any volunteers at all. > and developers had time to > recharge their batteries so to speak, and to sit and think about what > they wanted to do, without feeling the pressure to write code and > submit it right away. This assumes kernel development is uniform movement. Far from it, at any moment there are pieces that haven't been touched in ages, others in active turnover, others just finished being worked over. > Knowing that there's no place to send code for > six months is a great aid to generating GOOD code. For something else, sure. > The other advantage of a development tree was that features could be > added and removed without the argument that it would break this or > that. It was development, no one was supposed to use it for > production, no one could claim that there was even an implied promise > of things working or even existing. With the current tools, that development can be done outside the vanilla tree, and integrated with not too much pain. The /reason/ for "wild development" phases is not there anymore. > ipchains could have gone out of > 2.6 with no more fuss than xiafs departing. The people who really want > it stay with the old kernel. Come on, it has been announced for a /long/ time. It is not like anybody could have been caught unaware. More like people thinking it would /never/ be done as it was called so long before... -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 13:56 RFC: Starting a stable kernel series off the 2.6 kernel Adrian Bunk 2005-12-03 14:29 ` Jesper Juhl 2005-12-03 14:31 ` Ben Collins @ 2005-12-03 14:36 ` Arjan van de Ven 2005-12-03 15:23 ` Adrian Bunk 2005-12-03 18:39 ` Dr. David Alan Gilbert ` (4 subsequent siblings) 7 siblings, 1 reply; 639+ messages in thread From: Arjan van de Ven @ 2005-12-03 14:36 UTC (permalink / raw) To: Adrian Bunk; +Cc: linux-kernel > ase for a stable series. > > After 2.6.16, there will be a 2.6.16.y series with the usual stable > rules. > > After the release of 2.6.17, this 2.6.16.y series will be continued with > more relaxed rules similar to the rules in kernel 2.4 since the release > of kernel 2.6.0 (e.g. driver updates will be allowed). > this is a contradiction. You can't eat a cake and have it; either you're really low churn (like existing -stable) or you start adding new features like hardware support. the problem with hardware support is that it's not just a tiny driver update. If involves midlayer updates as well usually, and especially if those midlayers diverge between your stable and mainline, the "backports" are getting increasingly unsafe and hard. If the current model doesn't work as you claim it doesn't, then maybe the model needs finetuning. Right now the biggest pain is the userland ABI changes that need new packages; sometimes (often) for no real hard reason. Maybe we should just stop doing those bits, they're not in any fundamental way blocking general progress (sure there's some code bloat due to it, but I guess we'll just have to live with that). ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 14:36 ` Arjan van de Ven @ 2005-12-03 15:23 ` Adrian Bunk 2005-12-03 15:39 ` Arjan van de Ven ` (4 more replies) 0 siblings, 5 replies; 639+ messages in thread From: Adrian Bunk @ 2005-12-03 15:23 UTC (permalink / raw) To: Arjan van de Ven; +Cc: linux-kernel On Sat, Dec 03, 2005 at 03:36:38PM +0100, Arjan van de Ven wrote: > > ase for a stable series. > > > > After 2.6.16, there will be a 2.6.16.y series with the usual stable > > rules. > > > > After the release of 2.6.17, this 2.6.16.y series will be continued with > > more relaxed rules similar to the rules in kernel 2.4 since the release > > of kernel 2.6.0 (e.g. driver updates will be allowed). > > > > > this is a contradiction. You can't eat a cake and have it; either you're > really low churn (like existing -stable) or you start adding new > features like hardware support. the problem with hardware support is > that it's not just a tiny driver update. If involves midlayer updates as > well usually, and especially if those midlayers diverge between your > stable and mainline, the "backports" are getting increasingly unsafe and > hard. In the beginning, backporting hardware support is relatively easy, and therefore cherry-picking from mainline 2.6 should be relatively safe. Things will change as time passes by, but then there's the possibility to open the next branch and bring the older branch into a security-fixes only mode. > If the current model doesn't work as you claim it doesn't, then maybe > the model needs finetuning. Right now the biggest pain is the userland > ABI changes that need new packages; sometimes (often) for no real hard > reason. Maybe we should just stop doing those bits, they're not in any > fundamental way blocking general progress (sure there's some code bloat > due to it, but I guess we'll just have to live with that). IOW, we should e.g. ensure that today's udev will still work flawlessly with kernel 2.6.30 (sic)? This could work, but it should be officially announced that e.g. a userspace running kernel 2.6.15 must work flawlessly with _any_ future 2.6 kernel. For how many years do you think we will be able to ensure that this will stay true? cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 15:23 ` Adrian Bunk @ 2005-12-03 15:39 ` Arjan van de Ven 2005-12-04 13:53 ` Denis Vlasenko 2005-12-05 9:47 ` Michael Frank 2005-12-03 16:27 ` Matthias Andree ` (3 subsequent siblings) 4 siblings, 2 replies; 639+ messages in thread From: Arjan van de Ven @ 2005-12-03 15:39 UTC (permalink / raw) To: Adrian Bunk; +Cc: linux-kernel > > this is a contradiction. You can't eat a cake and have it; either you're > > really low churn (like existing -stable) or you start adding new > > features like hardware support. the problem with hardware support is > > that it's not just a tiny driver update. If involves midlayer updates as > > well usually, and especially if those midlayers diverge between your > > stable and mainline, the "backports" are getting increasingly unsafe and > > hard. > > In the beginning, backporting hardware support is relatively easy, and > therefore cherry-picking from mainline 2.6 should be relatively safe. and then there's reality. At least in my experience as distro kernel maintainer... you can do this for a few months, but it gets exponentially more expensive after 4 to 5 months. And "safe" is just not true. API, midlayer and locking changes in the newer kernels just void that concept entirely. And then there's the testing dillema; the people who'd run such a branch are EXACTLY the ones who wouldn't test prereleases of such branch (and yes 2.4 suffers from this as well). I doubt many distros would go for it as well for longer than a few months, simply because the hardware support and other features are going to be needed for them. > Things will change as time passes by, but then there's the possibility > to open the next branch and bring the older branch into a security-fixes > only mode. if you end up with 5 such branches it's no longer fun, trust me on that. Especially if the security fix is in a tricky area or a high flux area, then it's just not a matter of a simple backport anymore, even knowing if you're vulnerable or not is going to be a pain. And then there are the holes that happened to have gone away by later changes... what are you going to do then... put those changes in? that won't work longer term. > > > If the current model doesn't work as you claim it doesn't, then maybe > > the model needs finetuning. Right now the biggest pain is the userland > > ABI changes that need new packages; sometimes (often) for no real hard > > reason. Maybe we should just stop doing those bits, they're not in any > > fundamental way blocking general progress (sure there's some code bloat > > due to it, but I guess we'll just have to live with that). > > IOW, we should e.g. ensure that today's udev will still work flawlessly > with kernel 2.6.30 (sic)? I'd say yes. It doesn't need to support all new functionality, but at least what it does today it should be able to do then. If that really isn't possible maybe udev should be part of the kernel build and per kernel version. > This could work, but it should be officially announced that e.g. a > userspace running kernel 2.6.15 must work flawlessly with _any_ future > 2.6 kernel. I would argue that this in theory already is the current policy. Now "any" is pretty wide, but still. Maybe any such changes need to be scheduled to specific kernel releases only. Eg only do it every 4th or 5th kernel release. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 15:39 ` Arjan van de Ven @ 2005-12-04 13:53 ` Denis Vlasenko 2005-12-05 9:47 ` Michael Frank 1 sibling, 0 replies; 639+ messages in thread From: Denis Vlasenko @ 2005-12-04 13:53 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Adrian Bunk, linux-kernel On Saturday 03 December 2005 17:39, Arjan van de Ven wrote: > > IOW, we should e.g. ensure that today's udev will still work flawlessly > > with kernel 2.6.30 (sic)? > > I'd say yes. It doesn't need to support all new functionality, but at > least what it does today it should be able to do then. If that really > isn't possible maybe udev should be part of the kernel build and per > kernel version. I agree with this idea. -- vda ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 15:39 ` Arjan van de Ven 2005-12-04 13:53 ` Denis Vlasenko @ 2005-12-05 9:47 ` Michael Frank 2005-12-06 0:54 ` Horst von Brand 1 sibling, 1 reply; 639+ messages in thread From: Michael Frank @ 2005-12-05 9:47 UTC (permalink / raw) To: Arjan van de Ven, Adrian Bunk; +Cc: linux-kernel On Saturday 03 December 2005 16:39, Arjan van de Ven wrote: > > > this is a contradiction. You can't eat a cake and > > > have it; either you're really low churn (like > > > existing -stable) or you start adding new features > > > like hardware support. the problem with hardware > > > support is that it's not just a tiny driver update. > > > If involves midlayer updates as well usually, and > > > especially if those midlayers diverge between your > > > stable and mainline, the "backports" are getting > > > increasingly unsafe and hard. > > > > In the beginning, backporting hardware support is > > relatively easy, and therefore cherry-picking from > > mainline 2.6 should be relatively safe. > > and then there's reality. At least in my experience as > distro kernel maintainer... you can do this for a few > months, but it gets exponentially more expensive after 4 > to 5 months. And "safe" is just not true. API, midlayer > and locking changes in the newer kernels just void that > concept entirely. And then there's the testing dillema; > the people who'd run such a branch are EXACTLY the ones > who wouldn't test prereleases of such branch (and yes 2.4 > suffers from this as well). > > I doubt many distros would go for it as well for longer > than a few months, simply because the hardware support > and other features are going to be needed for them. > > > Things will change as time passes by, but then there's > > the possibility to open the next branch and bring the > > older branch into a security-fixes only mode. > > if you end up with 5 such branches it's no longer fun, > trust me on that. Especially if the security fix is in a > tricky area or a high flux area, then it's just not a > matter of a simple backport anymore, even knowing if > you're vulnerable or not is going to be a pain. And then > there are the holes that happened to have gone away by > later changes... what are you going to do then... put > those changes in? that won't work longer term. > > > > If the current model doesn't work as you claim it > > > doesn't, then maybe the model needs finetuning. Right > > > now the biggest pain is the userland ABI changes that > > > need new packages; sometimes (often) for no real hard > > > reason. Maybe we should just stop doing those bits, > > > they're not in any fundamental way blocking general > > > progress (sure there's some code bloat due to it, but > > > I guess we'll just have to live with that). > > > > IOW, we should e.g. ensure that today's udev will still > > work flawlessly with kernel 2.6.30 (sic)? > > I'd say yes. It doesn't need to support all new > functionality, but at least what it does today it should > be able to do then. If that really isn't possible maybe > udev should be part of the kernel build and per kernel > version. Most problems are avoided when packages closely linked to the kernel like udev and pcmcia will be updated by the distro together with the kernel by way of package version dependencies matching for example 2.6.14 to udev 065-069 and kernel 2.6.15 to udev 070. udev 070 and later could block kernels <=2.6.14. udev bit me recently when a scanner stopped working after updating to udev 070. In that way I had a chance to figure out how udev works and how to make some rules. Neat! ;) Perhaps extra care should be taken by the distro to not break the 50-udev.rules configuration file. > > > This could work, but it should be officially announced > > that e.g. a userspace running kernel 2.6.15 must work > > flawlessly with _any_ future 2.6 kernel. > > I would argue that this in theory already is the current > policy. Now "any" is pretty wide, but still. Maybe any > such changes need to be scheduled to specific kernel > releases only. Eg only do it every 4th or 5th kernel > release. My 2 cents: Test drive some rc's (2.6.15-rc) Use current -stable kernel as much as possible (2.6.14.3) Critical apps use -stable -1 or -2 (2.6.13.x or 2.6.12.x) Using -stable to -stable -2 one is about 3 months behind the latest and greatest instability ;). As to security, most vulnerabilities are hard to exploit remotely and practical security can be much more improved by hiding detailed software versions from clients. Apache 2 on linux 2.6 will do instead of providing full vendor specific package versions! As to drivers, in case 3 month driver delay matters, HW vendor can improve situation substantially by not waiting 6+ months before (if at all) releasing drivers/docs for linux! Michael ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 9:47 ` Michael Frank @ 2005-12-06 0:54 ` Horst von Brand 2005-12-06 17:08 ` Michael Frank 0 siblings, 1 reply; 639+ messages in thread From: Horst von Brand @ 2005-12-06 0:54 UTC (permalink / raw) To: mhf; +Cc: Arjan van de Ven, Adrian Bunk, linux-kernel Michael Frank <mhf@users.berlios.de> wrote: [...] > As to security, most vulnerabilities are hard to exploit > remotely Right. > and practical security can be much more improved > by hiding detailed software versions from clients. Ever heard of nmap <http://www.nmap.org>? Or perhaps noticed all kinds of attacks against Linux using old exploits or Windows specific ones? Hiding versions is /not/ secure. At most marginally so, and the pain for whoever needs the version for legitimate reasons just isn't worth it. > Apache > 2 on linux 2.6 will do instead of providing full vendor > specific package versions! > > As to drivers, in case 3 month driver delay matters, HW > vendor can improve situation substantially by not waiting > 6+ months before (if at all) releasing drivers/docs for > linux! For /server/ type workloads, where you /need/ stability, you carefully pick the hardware and then run a selected "enterprise" distro on it. The distro people do the hard work of keeping your kernel up to date and secure. And even worry about a smooth upgrade to the next version. For a price, sure. But either you really need it (and gladly pay the price) or you don't (in which case you have nothing to complain about). -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 0:54 ` Horst von Brand @ 2005-12-06 17:08 ` Michael Frank 0 siblings, 0 replies; 639+ messages in thread From: Michael Frank @ 2005-12-06 17:08 UTC (permalink / raw) To: Horst von Brand; +Cc: Arjan van de Ven, Adrian Bunk, linux-kernel On Tuesday 06 December 2005 01:54, Horst von Brand wrote: > Michael Frank <mhf@users.berlios.de> wrote: > > [...] > > > As to security, most vulnerabilities are hard to > > exploit remotely > > Right. > > > and practical security can be much more > > improved by hiding detailed software versions from > > clients. > > Ever heard of nmap <http://www.nmap.org>? I wrote my original post with nmap in mind. I use nmap all the time, it is a excellent tool. At times I used nmap to scan those IP's who scan my IP and have unleashed at times a barrage of counter-scans to the point of bringing my connection pretty much down. Some of those scanners must have big pipes. > Or perhaps > noticed all kinds of attacks against Linux using old > exploits or Windows specific ones? 100s to 1000s of unsolicited packets mainly to windows specific service ports day and also several burst attacks a day. These days I just leave the firewall logging off in order to play less with nmap ;) > Hiding versions is > /not/ secure. At most marginally so, Sorry, do not concur. Unlike windows and such, linux is a _fast_ moving target. The best bet to crack it a _inside_ job by a trusted perpetrator (for example the debian case or the corruption of the kernels bk derived cvs repo ), which still ring bells... Remote exploitation of the odd random vulnerability in the absence of detailed version info is as likely as winning a jackpot. > and the pain for > whoever needs the version for legitimate reasons just > isn't worth it. Oh well, If I have a legitimate requirement for the detailed versions someone is running, I can ask. Again, most violations are made possible by: a) access to hardware or admin/root passwords (what was the kernel.org case tracked to?) b) access to version info and utilizing a exploit short term (the debian case) IMHO, to have good security, 1) use open source and 2) sound implementation of common sense security procedures beginning with the basic doctrine who does not have to know won't know. Yes, some things seem never to change :-( > > > > Apache 2 on linux 2.6 will do instead of providing full > > vendor specific package versions! > > > > As to drivers, in case 3 month driver delay matters, HW > > vendor can improve situation substantially by not > > waiting 6+ months before (if at all) releasing > > drivers/docs for linux! > > For /server/ type workloads, where you /need/ stability, > you carefully pick the hardware and then run a selected > "enterprise" distro on it. The distro people do the hard > work of keeping your kernel up to date and secure. And > even worry about a smooth upgrade to the next version. > For a price, sure. But either you really need it (and > gladly pay the price) sure > or you don't (in which case you > have nothing to complain about). kernel.org kernels and gentoo linux will do for me. Thank you Michael ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 15:23 ` Adrian Bunk 2005-12-03 15:39 ` Arjan van de Ven @ 2005-12-03 16:27 ` Matthias Andree 2005-12-03 16:40 ` Otavio Salvador ` (2 more replies) 2005-12-03 18:43 ` Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) Jeff Garzik ` (2 subsequent siblings) 4 siblings, 3 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-03 16:27 UTC (permalink / raw) To: linux-kernel On Sat, 03 Dec 2005, Adrian Bunk wrote: > Things will change as time passes by, but then there's the possibility > to open the next branch and bring the older branch into a security-fixes > only mode. > > > If the current model doesn't work as you claim it doesn't, then maybe > > the model needs finetuning. Right now the biggest pain is the userland > > ABI changes that need new packages; sometimes (often) for no real hard > > reason. Maybe we should just stop doing those bits, they're not in any > > fundamental way blocking general progress (sure there's some code bloat > > due to it, but I guess we'll just have to live with that). > > IOW, we should e.g. ensure that today's udev will still work flawlessly > with kernel 2.6.30 (sic)? Exactly that, and kernel interfaces going away just to annoy binary module providers also hurts third-party OSS modules, such as Fujitsu-Siemens's ServerView agents. I found myself chasing some genuine bugs in that code to please GCC 4 (which doesn't tolerate extern blah declarations before static blah definitions), and some idiotic breakage when inter_module_get was dropped somewhen between 2.6.8 (where the modules compiled just fine) and 2.6.13 (where they didn't as functions had been removed). And there's no point arguing about deprecation warnings being around, interfaces can be removed between 2.6 and 2.8 but not between 2.6.foo.bar and 2.6.foo.baz. Why not use #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,8,0) or similar to automatically disable such interfaces at the given version? > This could work, but it should be officially announced that e.g. a > userspace running kernel 2.6.15 must work flawlessly with _any_ future > 2.6 kernel. Right. This effectively means to call the beast 2.8.0 if you feel you must break the applications. > For how many years do you think we will be able to ensure that this will > stay true? Well, the current model, since it has been in effect, is just "let's break the interfaces at will, but let's not hint anyone, so let's always just bump the patchlevel no matter how intrusive the change is", and the bandaid 2.6.M.N releases cannot cover the fact that the M -> M+1 transition causes breakage. Effectively, 2.6.X behaves like a bleeding edge development ("everything changes") kernel such as 2.3 or 2.5; it isn't stable because some code monkeys claim it is. You cannot declare the can of pea soup in front of you "open" either. Linux is in fact light years away from being "stable", and while it was relatively easy to move forward between 2.4.X releases and even 2.4.X and some early 2.6.X release, moving between 2.6.X and 2.6.Y means switching user-space, too, and chances are the new user-space doesn't work with the old kernel. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 16:27 ` Matthias Andree @ 2005-12-03 16:40 ` Otavio Salvador 2005-12-03 16:58 ` David Ranson 2005-12-03 17:22 ` Arjan van de Ven 2 siblings, 0 replies; 639+ messages in thread From: Otavio Salvador @ 2005-12-03 16:40 UTC (permalink / raw) To: linux-kernel Matthias Andree <matthias.andree@gmx.de> writes: >> This could work, but it should be officially announced that e.g. a >> userspace running kernel 2.6.15 must work flawlessly with _any_ future >> 2.6 kernel. > > Right. This effectively means to call the beast 2.8.0 if you feel you > must break the applications. Looks like things are leaving clear the need of 2.7 for those developments. This is my point of view. -- O T A V I O S A L V A D O R --------------------------------------------- E-mail: otavio@debian.org UIN: 5906116 GNU/Linux User: 239058 GPG ID: 49A5F855 Home Page: http://www.freedom.ind.br/otavio --------------------------------------------- "Microsoft gives you Windows ... Linux gives you the whole house." ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 16:27 ` Matthias Andree 2005-12-03 16:40 ` Otavio Salvador @ 2005-12-03 16:58 ` David Ranson 2005-12-03 17:13 ` Steven Rostedt 2005-12-03 17:22 ` Arjan van de Ven 2 siblings, 1 reply; 639+ messages in thread From: David Ranson @ 2005-12-03 16:58 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel [-- Attachment #1: Type: text/plain, Size: 2099 bytes --] Matthias Andree wrote: >>>the model needs finetuning. Right now the biggest pain is the userland >>>ABI changes that need new packages; sometimes (often) for no real hard >>>reason. Maybe we should just stop doing those bits, they're not in any >>>fundamental way blocking general progress (sure there's some code bloat >>>due to it, but I guess we'll just have to live with that) >>> >>> Well as a mildly technical 'user' who's been tracking the 2.6 series I can't recall having to update _any_ userland packages due to kernel changes. An example of this would be helpful. >Exactly that, and kernel interfaces going away just to annoy binary >module providers also hurts third-party OSS modules, such as >Fujitsu-Siemens's ServerView agents. > > As many here say. Too bad, we really don't care. Hardware providers should have the requisite relationships with the distros and target their management utilities to those. Surely no business using this sort of high end hardware is running anything other than RHEL or SLES etc ? >I found myself chasing some genuine bugs in that code to please GCC 4 >(which doesn't tolerate extern blah declarations before static blah >definitions), and some idiotic breakage when inter_module_get was >dropped somewhen between 2.6.8 (where the modules compiled just fine) >and 2.6.13 (where they didn't as functions had been removed). And >there's no point arguing about deprecation warnings being around, >interfaces can be removed between 2.6 and 2.8 but not between > > See above. >Well, the current model, since it has been in effect, is just "let's >break the interfaces at will, but let's not hint anyone, so let's always > > Only kernel interfaces. And that's too bad for out of kernel modules. >Linux is in fact light years away from being "stable", and while it was > > If you want stable you want a distro 'enterprise' version. >and some early 2.6.X release, moving between 2.6.X and 2.6.Y means >switching user-space, too, and chances are the new user-space doesn't >work with the old kernel. > > In my experience this isn't the case. Cheers David [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 187 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 16:58 ` David Ranson @ 2005-12-03 17:13 ` Steven Rostedt 2005-12-03 17:17 ` David Ranson 0 siblings, 1 reply; 639+ messages in thread From: Steven Rostedt @ 2005-12-03 17:13 UTC (permalink / raw) To: David Ranson; +Cc: linux-kernel, Matthias Andree On Sat, 2005-12-03 at 16:58 +0000, David Ranson wrote: > Matthias Andree wrote: > > >>>the model needs finetuning. Right now the biggest pain is the userland > >>>ABI changes that need new packages; sometimes (often) for no real hard > >>>reason. Maybe we should just stop doing those bits, they're not in any > >>>fundamental way blocking general progress (sure there's some code bloat > >>>due to it, but I guess we'll just have to live with that) > >>> > >>> > Well as a mildly technical 'user' who's been tracking the 2.6 series I > can't recall having to update _any_ userland packages due to kernel > changes. An example of this would be helpful. udev ;) http://seclists.org/lists/linux-kernel/2005/Dec/0180.html -- Steve ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 17:13 ` Steven Rostedt @ 2005-12-03 17:17 ` David Ranson 2005-12-03 17:53 ` Adrian Bunk ` (2 more replies) 0 siblings, 3 replies; 639+ messages in thread From: David Ranson @ 2005-12-03 17:17 UTC (permalink / raw) To: Steven Rostedt; +Cc: linux-kernel, Matthias Andree [-- Attachment #1: Type: text/plain, Size: 246 bytes --] Steven Rostedt wrote: >udev ;) > >http://seclists.org/lists/linux-kernel/2005/Dec/0180.html > > Ahh OK .. I don't use it, so wouldn't have been affected. That's one userspace interface broken during the series, does anyone have any more? David [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 187 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 17:17 ` David Ranson @ 2005-12-03 17:53 ` Adrian Bunk 2005-12-03 18:17 ` newbie - mdadm create raid1 mirrors on large drives Larry Bates ` (2 more replies) 2005-12-03 18:21 ` Mark Lord 2005-12-03 22:21 ` Matthias Andree 2 siblings, 3 replies; 639+ messages in thread From: Adrian Bunk @ 2005-12-03 17:53 UTC (permalink / raw) To: David Ranson; +Cc: Steven Rostedt, linux-kernel, Matthias Andree On Sat, Dec 03, 2005 at 05:17:41PM +0000, David Ranson wrote: > Steven Rostedt wrote: > > >udev ;) > > > >http://seclists.org/lists/linux-kernel/2005/Dec/0180.html > > > > > Ahh OK .. I don't use it, so wouldn't have been affected. That's one > userspace interface broken during the series, does anyone have any more? - support for ipfwadm and ipchains was removed during 2.6 - devfs support was removed during 2.6 - removal of kernel support for pcmcia-cs is pending - ip{,6}_queue removal is pending - removal of the RAW driver is pending > David cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* newbie - mdadm create raid1 mirrors on large drives 2005-12-03 17:53 ` Adrian Bunk @ 2005-12-03 18:17 ` Larry Bates 2005-12-03 22:23 ` Matthias Andree 2005-12-04 22:13 ` Neil Brown 2005-12-03 18:34 ` RFC: Starting a stable kernel series off the 2.6 kernel David Ranson 2005-12-05 3:31 ` Rob Landley 2 siblings, 2 replies; 639+ messages in thread From: Larry Bates @ 2005-12-03 18:17 UTC (permalink / raw) To: linux-kernel I hope this is the correct list for this question. I've just recently begun using mdadm to set up some arrays using large drives (300-400Gb). One of the things I don't understand is this: when you first create a raid1 (mirrored) array from two drives mdadm insists on mirroring the contents of the first drive to the second even though the drives are entirely blank (e.g. new drives don't have anything on them). In one configuration I have, this takes about 16 hours on a 400Gb drive. When I do 5 of them simultaneously this takes 2+ days to complete. Is there some way to tell mdadm that you want to create a mirrored set but skip this rather long initial mirroring process? I don't really see that it actually accomplishes anything. Thanks in advance for your assistance. -Larry Bates ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: newbie - mdadm create raid1 mirrors on large drives 2005-12-03 18:17 ` newbie - mdadm create raid1 mirrors on large drives Larry Bates @ 2005-12-03 22:23 ` Matthias Andree 2005-12-04 22:13 ` Neil Brown 1 sibling, 0 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-03 22:23 UTC (permalink / raw) To: Larry Bates; +Cc: linux-kernel On Sat, 03 Dec 2005, Larry Bates wrote: > I hope this is the correct list for this question. No list is right for hijacking threads. Don't do that again. Please use "New Mail" or however it's named for a new different topic. Your message has nothing to do with stable kernel fork discussion. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: newbie - mdadm create raid1 mirrors on large drives 2005-12-03 18:17 ` newbie - mdadm create raid1 mirrors on large drives Larry Bates @ 2005-12-04 22:13 ` Neil Brown 2005-12-04 22:13 ` Neil Brown 1 sibling, 0 replies; 639+ messages in thread From: Neil Brown @ 2005-12-04 22:13 UTC (permalink / raw) To: Larry Bates; +Cc: linux-kernel, linux-raid On Saturday December 3, lbates@syscononline.com wrote: > I hope this is the correct list for this question. linux-raid@vger.kernel.org is possibly better, but linux-kernel is probably a suitable catch-all. > > I've just recently begun using mdadm to set up some > arrays using large drives (300-400Gb). One of the > things I don't understand is this: when you first > create a raid1 (mirrored) array from two drives > mdadm insists on mirroring the contents of the first > drive to the second even though the drives are > entirely blank (e.g. new drives don't have anything > on them). Well... they do have something one them - lots of zeros and ones, or maybe just zeros, or maybe just ones. Sure, you may not be interested in that data, but it is there. > In one configuration I have, this takes > about 16 hours on a 400Gb drive. When I do 5 of them > simultaneously this takes 2+ days to complete. Is > there some way to tell mdadm that you want to create > a mirrored set but skip this rather long initial > mirroring process? I don't really see that it actually > accomplishes anything. No, there is no way to tell mdadm to skip the initial copying process. It is not clear to me that you really want to do this(*) (though on the "enough rope" principle I'm probably going to extend the "--assume-clean" option to work in --create mode). I suggest you simply ignore the fact that it is doing the copy. Just keep using the array as though it wasn't. If this seems to be impacting over-all system performance, tune /proc/sys/dev/raid/speed_* to slow it down even more. If you reboot, it should remember where it was up to and restart from the same place (providing you are using a 2.6 kernel). If you have 5 of these 400Gb raid1's, then I suspect you really want to avoid the slow resync that happens after a crash. You should look into adding a bitmap write-intent log. This requires 2.6.14, and mdadm 2.1, and is as easy as mdadm --grow --bitmap=internal /dev/md3 while the array is running. This should dramatically reduce resync time, at a possible small cost in write throughput. Some limited measurements I have done suggest up-to 10% slowdown, though usually less. Possibly some tuning can make it much better. (*) A raid array can suffer from sleeping bad blocks. i.e. blocks that you cannot read, but normally you never do (because they haven't been allocated to a file yet). When a drive fails, and you are recovering the data onto a spare, hitting that sleeper can kill your array. For this reason it is good to regularly (daily, or weekly, maybe monthly) read through the entire array making sure everything is OK. In 2.6.16 (with complete functionality in 2.6.17) you will be able to trigger a background read-test of the whole array: echo check > /sys/block/mdX/md/sync_action If you were to create an array with --assume-clean, then whenever you run this it will report lots of errors, though you can fix them with echo repair > /sys/block/mdX/md/sync_action If you are going to be doing that (and I would recommend it) then you may as well allow the initial sync, especially as you can quite happily ignore the fact that it is happening. NeilBrown ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: newbie - mdadm create raid1 mirrors on large drives @ 2005-12-04 22:13 ` Neil Brown 0 siblings, 0 replies; 639+ messages in thread From: Neil Brown @ 2005-12-04 22:13 UTC (permalink / raw) To: Larry Bates; +Cc: linux-kernel, linux-raid On Saturday December 3, lbates@syscononline.com wrote: > I hope this is the correct list for this question. linux-raid@vger.kernel.org is possibly better, but linux-kernel is probably a suitable catch-all. > > I've just recently begun using mdadm to set up some > arrays using large drives (300-400Gb). One of the > things I don't understand is this: when you first > create a raid1 (mirrored) array from two drives > mdadm insists on mirroring the contents of the first > drive to the second even though the drives are > entirely blank (e.g. new drives don't have anything > on them). Well... they do have something one them - lots of zeros and ones, or maybe just zeros, or maybe just ones. Sure, you may not be interested in that data, but it is there. > In one configuration I have, this takes > about 16 hours on a 400Gb drive. When I do 5 of them > simultaneously this takes 2+ days to complete. Is > there some way to tell mdadm that you want to create > a mirrored set but skip this rather long initial > mirroring process? I don't really see that it actually > accomplishes anything. No, there is no way to tell mdadm to skip the initial copying process. It is not clear to me that you really want to do this(*) (though on the "enough rope" principle I'm probably going to extend the "--assume-clean" option to work in --create mode). I suggest you simply ignore the fact that it is doing the copy. Just keep using the array as though it wasn't. If this seems to be impacting over-all system performance, tune /proc/sys/dev/raid/speed_* to slow it down even more. If you reboot, it should remember where it was up to and restart from the same place (providing you are using a 2.6 kernel). If you have 5 of these 400Gb raid1's, then I suspect you really want to avoid the slow resync that happens after a crash. You should look into adding a bitmap write-intent log. This requires 2.6.14, and mdadm 2.1, and is as easy as mdadm --grow --bitmap=internal /dev/md3 while the array is running. This should dramatically reduce resync time, at a possible small cost in write throughput. Some limited measurements I have done suggest up-to 10% slowdown, though usually less. Possibly some tuning can make it much better. (*) A raid array can suffer from sleeping bad blocks. i.e. blocks that you cannot read, but normally you never do (because they haven't been allocated to a file yet). When a drive fails, and you are recovering the data onto a spare, hitting that sleeper can kill your array. For this reason it is good to regularly (daily, or weekly, maybe monthly) read through the entire array making sure everything is OK. In 2.6.16 (with complete functionality in 2.6.17) you will be able to trigger a background read-test of the whole array: echo check > /sys/block/mdX/md/sync_action If you were to create an array with --assume-clean, then whenever you run this it will report lots of errors, though you can fix them with echo repair > /sys/block/mdX/md/sync_action If you are going to be doing that (and I would recommend it) then you may as well allow the initial sync, especially as you can quite happily ignore the fact that it is happening. NeilBrown ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 17:53 ` Adrian Bunk 2005-12-03 18:17 ` newbie - mdadm create raid1 mirrors on large drives Larry Bates @ 2005-12-03 18:34 ` David Ranson 2005-12-03 22:27 ` Matthias Andree 2005-12-05 20:43 ` Bill Davidsen 2005-12-05 3:31 ` Rob Landley 2 siblings, 2 replies; 639+ messages in thread From: David Ranson @ 2005-12-03 18:34 UTC (permalink / raw) To: Adrian Bunk; +Cc: Steven Rostedt, linux-kernel, Matthias Andree [-- Attachment #1: Type: text/plain, Size: 765 bytes --] Adrian Bunk wrote: >- support for ipfwadm and ipchains was removed during 2.6 > > Surely this one had loads of notice though? I was using iptables with 2.4 kernels. >- devfs support was removed during 2.6 > > Did this affect many 'real' users? >- removal of kernel support for pcmcia-cs is pending >- ip{,6}_queue removal is pending >- removal of the RAW driver is pending > > I don't use any of these. I guess pcmcia-cs may be disruptive for laptop users. So far I don't see evidence to suggest huge repeated userspace breakages between Kernel versions that were implied earlier in this thread. Whatever, we aren't going to see any more stable branches without volunteers to do the spadework. As has been pointed out, this won't always be an easy task. David [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 187 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 18:34 ` RFC: Starting a stable kernel series off the 2.6 kernel David Ranson @ 2005-12-03 22:27 ` Matthias Andree 2005-12-03 22:34 ` Lee Revell ` (2 more replies) 2005-12-05 20:43 ` Bill Davidsen 1 sibling, 3 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-03 22:27 UTC (permalink / raw) To: linux-kernel On Sat, 03 Dec 2005, David Ranson wrote: > Adrian Bunk wrote: > > >- support for ipfwadm and ipchains was removed during 2.6 > > > > > Surely this one had loads of notice though? I was using iptables with > 2.4 kernels. So was I. And now what? ipfwadm and ipchains should have been removed from 2.6.0 if 2.6.0 was not to support these. That opportunity was missed, the removal wasn't made up for in 2.6.1, so the stuff has to stick until 2.8.0. > >- devfs support was removed during 2.6 > > Did this affect many 'real' users? This doesn't matter. A kernel that calls itself stable CAN NOT remove features unless they had been critically broken from the beginning. And this level of breakage is a moot point, so removal is not justified. > >- removal of kernel support for pcmcia-cs is pending > >- ip{,6}_queue removal is pending > >- removal of the RAW driver is pending > I don't use any of these. I guess pcmcia-cs may be disruptive for laptop > users. Who cares what you or I use? It's a commonly acknowledged policy that "stable" releases do not remove features that are good enough for some. Linux 2.6 is not "stable" in this regard. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:27 ` Matthias Andree @ 2005-12-03 22:34 ` Lee Revell 2005-12-03 22:50 ` Matthias Andree 2005-12-03 22:36 ` David Ranson 2005-12-04 1:04 ` Horst von Brand 2 siblings, 1 reply; 639+ messages in thread From: Lee Revell @ 2005-12-03 22:34 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel On Sat, 2005-12-03 at 23:27 +0100, Matthias Andree wrote: > A kernel that calls itself stable CAN NOT remove > features unless they had been critically broken from the beginning. So in your opinion we can't add support for new hardware to a stable kernel either because there's a chance of breaking something that worked before, which brings us right back to "stable" meaning "no progress allowed", which begs the question of why you want to upgrade at all. Lee ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:34 ` Lee Revell @ 2005-12-03 22:50 ` Matthias Andree 2005-12-04 0:20 ` Greg KH 0 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2005-12-03 22:50 UTC (permalink / raw) To: linux-kernel On Sat, 03 Dec 2005, Lee Revell wrote: > On Sat, 2005-12-03 at 23:27 +0100, Matthias Andree wrote: > > A kernel that calls itself stable CAN NOT remove > > features unless they had been critically broken from the beginning. > > So in your opinion we can't add support for new hardware to a stable > kernel either because there's a chance of breaking something that worked > before, which brings us right back to "stable" meaning "no progress > allowed", which begs the question of why you want to upgrade at all. Perhaps I did not word not clearly enough, please bear with me as I'm not a native speaker. There's a gray area between these two extremes. I don't mind if new drivers are added, not even if they touch other parts of the code if these changes are thoroughly (and I mean thoroughly, not a smoke test of the "works for me" kind) examined. If devfs had been marked "DEPRECATED, WILL BE REMOVED FROM 2.6.0", all would have been fine. (I don't recall the exact version, 2.6.12? It wasn't known beforehand), I certainly do not expect new drivers that are added to support it. First step, make a note "this driver has been added in Linux 2.6.14" so that everyone is aware the driver doesn't need to support devfs as that one was already deprecated then the driver moved in. Even better, mark which deprecated subsystems are unsupported by the driver. Second, schedule for removal such subsystems well ahead of time, for instance, "DEPRECATED, WILL BE REMOVED JUST BEFORE 2.8.0", and only use minor releases to that extent. The point is, removing something that has worked well enough that some people had a reason to use it, is not "stable". Third, IF udev is so sexy but OTOH a real kernel-space devfs can be done in 200 LoC as has been claimed so often, why in hell is this not happening? Switch "broken bloaty bulky devfs" to "lean & clean devfs"? This ship would have been flying the "clean-up nation" flags. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:50 ` Matthias Andree @ 2005-12-04 0:20 ` Greg KH 2005-12-04 4:46 ` Luke-Jr 2005-12-05 22:47 ` Rob Landley 0 siblings, 2 replies; 639+ messages in thread From: Greg KH @ 2005-12-04 0:20 UTC (permalink / raw) To: linux-kernel On Sat, Dec 03, 2005 at 11:50:20PM +0100, Matthias Andree wrote: > The point is, removing something that has worked well enough that some > people had a reason to use it, is not "stable". Please remember, no one is calling 2.6 "stable" anymore than they are calling it "development". The current development model is different than what we used to do pre 2.6. See the archives for details about this if you want more information. > Third, IF udev is so sexy but OTOH a real kernel-space devfs can be done > in 200 LoC as has been claimed so often, 282 LoC: drivers/base/class.c | 7 + fs/Kconfig | 3 fs/Makefile | 1 fs/ndevfs/Makefile | 4 fs/ndevfs/inode.c | 249 +++++++++++++++++++++++++++++++++++++++++++++++++ fs/partitions/check.c | 6 + include/linux/ndevfs.h | 13 ++ 7 files changed, 282 insertions(+), 1 deletion(-) > why in hell is this not happening? Because it's not the correct solution. > Switch "broken bloaty bulky devfs" to "lean & clean devfs"? This ship > would have been flying the "clean-up nation" flags. Again, because an in-kernel devfs is not the correct thing to do. devfs has been disabled for a few months now, and I don't think anyone has missed it yet :) thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 0:20 ` Greg KH @ 2005-12-04 4:46 ` Luke-Jr 2005-12-04 15:06 ` Michael Frank 2005-12-04 23:22 ` Greg KH 2005-12-05 22:47 ` Rob Landley 1 sibling, 2 replies; 639+ messages in thread From: Luke-Jr @ 2005-12-04 4:46 UTC (permalink / raw) To: Linux Kernel Mailing List On Sunday 04 December 2005 00:20, Greg KH wrote: > > Switch "broken bloaty bulky devfs" to "lean & clean devfs"? This ship > > would have been flying the "clean-up nation" flags. > > Again, because an in-kernel devfs is not the correct thing to do. devfs > has been disabled for a few months now, and I don't think anyone has > missed it yet :) Well, devfs does have some abilities udev doesn't: hotplug/udev doesn't detect everything, and can result in rarer or non-PnP devices not being automatically available; devfs has the effect of trying to load a module when a program looks for the devices it provides-- while it can cause problems, it does have a possibility to work better. Interesting effects of switching my desktop from devfs to udev: 1. my DVD burners are left uninitialized until I manually modprobe ide-cd or (more recently) ide-scsi 2. my sound card is autodetected and the drivers loaded, but the OSS emulation modules are omitted; with devfs, they would be autoloaded when an app tried to use OSS devfs also has the advantage of keeping the module info all in one place-- the kernel or the module. In particular, with udev the detection and /dev info is scattered into different locations of the filesystem. This can probably be fixed easily simply by having udev read such info from modules or via a /sys entry, though. -- Luke-Jr Developer, Utopios http://utopios.org/ ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 4:46 ` Luke-Jr @ 2005-12-04 15:06 ` Michael Frank 2005-12-04 23:22 ` Greg KH 1 sibling, 0 replies; 639+ messages in thread From: Michael Frank @ 2005-12-04 15:06 UTC (permalink / raw) To: Luke-Jr; +Cc: Linux Kernel Mailing List On Sunday 04 December 2005 05:46, Luke-Jr wrote: > On Sunday 04 December 2005 00:20, Greg KH wrote: > > > Switch "broken bloaty bulky devfs" to "lean & clean > > > devfs"? This ship would have been flying the > > > "clean-up nation" flags. > > > > Again, because an in-kernel devfs is not the correct > > thing to do. devfs has been disabled for a few months > > now, and I don't think anyone has missed it yet :) > > Well, devfs does have some abilities udev doesn't: > hotplug/udev doesn't detect everything, and can result in > rarer or non-PnP devices not being automatically > available; devfs has the effect of trying to load a > module when a program looks for the devices it provides-- > while it can cause problems, it does have a possibility > to work better. Interesting effects of switching my > desktop from devfs to udev: > 1. my DVD burners are left uninitialized until I manually > modprobe ide-cd or (more recently) ide-scsi > 2. my sound card is autodetected and the drivers loaded, > but the OSS emulation modules are omitted; with devfs, > they would be autoloaded when an app tried to use OSS > devfs also has the advantage of keeping the module info > all in one place-- the kernel or the module. In > particular, with udev the detection and /dev info is > scattered into different locations of the filesystem. > This can probably be fixed easily simply by having udev > read such info from modules or via a /sys entry, though. Seems your complaints are related to missing/broken configuration (of your distribution). Here are some references which may be of help: http://webpages.charter.net/decibelshelp/LinuxHelp_UDEVPrimer.html http://www.gentoo.org/doc/en/udev-guide.xml http://gentoo-wiki.com/HOWTO_Customizing_UDEV http://ubuntuforums.org/archive/index.php/t-11718.html IME, udev is easier to manage than devfs because there are tools such as udevtest. Please see also manuals for udevstart udevtest, udevinfo, udevcontrol Also strace udevstart is interesting... ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 4:46 ` Luke-Jr 2005-12-04 15:06 ` Michael Frank @ 2005-12-04 23:22 ` Greg KH 2005-12-05 5:59 ` Luke-Jr 2005-12-06 14:58 ` Bill Davidsen 1 sibling, 2 replies; 639+ messages in thread From: Greg KH @ 2005-12-04 23:22 UTC (permalink / raw) To: Luke-Jr; +Cc: Linux Kernel Mailing List On Sun, Dec 04, 2005 at 04:46:31AM +0000, Luke-Jr wrote: > Well, devfs does have some abilities udev doesn't: hotplug/udev > doesn't detect everything, and can result in rarer or non-PnP devices > not being automatically available; Are you sure about that today? And udev wasn't created to do everything that devfs does. And devfs can't do everything that udev can (by far...) > devfs has the effect of trying to load a module when a program looks > for the devices it provides-- while it can cause problems, it does > have a possibility to work better. Sorry, but that model of loading modules is very broken and it is good that we don't do that anymore (as you allude to.) > Interesting effects of switching my desktop from devfs to udev: > 1. my DVD burners are left uninitialized until I manually modprobe ide-cd or > (more recently) ide-scsi Sounds like a broken distro configuration :) > 2. my sound card is autodetected and the drivers loaded, but the OSS emulation > modules are omitted; with devfs, they would be autoloaded when an app tried > to use OSS Again, broken distro configuration :) > devfs also has the advantage of keeping the module info all in one place-- the > kernel or the module. What? > In particular, with udev the detection and /dev info is scattered into > different locations of the filesystem. This can probably be fixed > easily simply by having udev read such info from modules or via a /sys > entry, though. What information are you talking about here? thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 23:22 ` Greg KH @ 2005-12-05 5:59 ` Luke-Jr 2005-12-06 0:34 ` Rob Landley 2005-12-06 17:38 ` Greg KH 2005-12-06 14:58 ` Bill Davidsen 1 sibling, 2 replies; 639+ messages in thread From: Luke-Jr @ 2005-12-05 5:59 UTC (permalink / raw) To: Greg KH; +Cc: Linux Kernel Mailing List On Sunday 04 December 2005 23:22, Greg KH wrote: > On Sun, Dec 04, 2005 at 04:46:31AM +0000, Luke-Jr wrote: > > Well, devfs does have some abilities udev doesn't: hotplug/udev > > doesn't detect everything, and can result in rarer or non-PnP devices > > not being automatically available; > > Are you sure about that today? Nope, but I don't see how udev can possibly detect something that doesn't let the OS know it's there-- except, of course, loading the driver for it and seeing if it works. > And udev wasn't created to do everything that devfs does. Which might be a case for leaving devfs in. *shrug* > And devfs can't do everything that udev can (by far...) Didn't say it could... > > Interesting effects of switching my desktop from devfs to udev: > > 1. my DVD burners are left uninitialized until I manually modprobe ide-cd > > or (more recently) ide-scsi > > Sounds like a broken distro configuration :) Well, I was assuming you kept Gentoo's udev packages up to date. ;) [ebuild R ] sys-fs/udev-070-r1 (-selinux) -static 429 kB > > devfs also has the advantage of keeping the module info all in one > > place-- the kernel or the module. > > In particular, with udev the detection and /dev info is scattered into > > different locations of the filesystem. This can probably be fixed > > easily simply by having udev read such info from modules or via a /sys > > entry, though. > > What information are you talking about here? I'm assuming everything in /etc/udev/rules.d/50-udev.rules used to be in the kernel for devfs-- perhaps it was PAM though, I'm not sure. Other than that, I don't expect that simply installing a new kernel module will allow the device to be detected automatically, but that some hotplug or udev configurations will need to be updated also. -- Luke-Jr Developer, Utopios http://utopios.org/ ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 5:59 ` Luke-Jr @ 2005-12-06 0:34 ` Rob Landley 2005-12-06 10:34 ` Luke-Jr 2005-12-06 17:38 ` Greg KH 1 sibling, 1 reply; 639+ messages in thread From: Rob Landley @ 2005-12-06 0:34 UTC (permalink / raw) To: Luke-Jr; +Cc: Greg KH, Linux Kernel Mailing List On Sunday 04 December 2005 23:59, Luke-Jr wrote: > On Sunday 04 December 2005 23:22, Greg KH wrote: > > On Sun, Dec 04, 2005 at 04:46:31AM +0000, Luke-Jr wrote: > > > Well, devfs does have some abilities udev doesn't: hotplug/udev > > > doesn't detect everything, and can result in rarer or non-PnP devices > > > not being automatically available; > > > > Are you sure about that today? > > Nope, but I don't see how udev can possibly detect something that doesn't > let the OS know it's there-- except, of course, loading the driver for it > and seeing if it works. Stuff shows up in /sys whether or not Linux has a driver loaded for it. ls -l /sys/bus/*/devices Hotplug insertion events are for when the _device_ shows up, not for when the driver is loaded. > > And udev wasn't created to do everything that devfs does. > > Which might be a case for leaving devfs in. *shrug* I think it was a polite way of saying "udev doesn't suck, have unfixable races, randomly crash your system..." > > What information are you talking about here? > > I'm assuming everything in /etc/udev/rules.d/50-udev.rules used to be in > the kernel for devfs-- perhaps it was PAM though, I'm not sure. > Other than that, I don't expect that simply installing a new kernel module > will allow the device to be detected automatically, but that some hotplug > or udev configurations will need to be updated also. On an unrelated note, the proposed file format for busybox's mdev looks something like: hd[a-z] 0:3 700 hd[a-z][0-9]* 0:3 740 .* 0:0 700 There's a little more to it than that, but really, specifying ownership and permissions is all we _really_ care about. (We're not trying to obsolete udev.) The point is, 90% of the complexity of udev is optional. This _can_ be a lot simpler if you're not trying to tackle strange persistent naming issues resulting in dynamically generated symlinks, and similar fun. (Which we may add the ability to do as compile-time config options later, but perhaps not until somebody actually misses them...) We really don't forsee having to update mdev for deal with new kernel versions... ever, if we can help it. And a dynamic module loader hanging off of /sbin/hotplug is probably (conceptually) a different tool from mdev... Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 0:34 ` Rob Landley @ 2005-12-06 10:34 ` Luke-Jr 2005-12-06 19:17 ` Rob Landley 0 siblings, 1 reply; 639+ messages in thread From: Luke-Jr @ 2005-12-06 10:34 UTC (permalink / raw) To: Linux Kernel Mailing List; +Cc: Rob Landley, Greg KH On Tuesday 06 December 2005 00:34, Rob Landley wrote: > On Sunday 04 December 2005 23:59, Luke-Jr wrote: > > On Sunday 04 December 2005 23:22, Greg KH wrote: > > > On Sun, Dec 04, 2005 at 04:46:31AM +0000, Luke-Jr wrote: > > > > Well, devfs does have some abilities udev doesn't: hotplug/udev > > > > doesn't detect everything, and can result in rarer or non-PnP devices > > > > not being automatically available; > > > > > > Are you sure about that today? > > > > Nope, but I don't see how udev can possibly detect something that doesn't > > let the OS know it's there-- except, of course, loading the driver for it > > and seeing if it works. > > Stuff shows up in /sys whether or not Linux has a driver loaded for it. Only if Linux is aware it exists. I'm thinking of those old ISA cards and such. -- Luke-Jr Developer, Utopios http://utopios.org/ ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 10:34 ` Luke-Jr @ 2005-12-06 19:17 ` Rob Landley 0 siblings, 0 replies; 639+ messages in thread From: Rob Landley @ 2005-12-06 19:17 UTC (permalink / raw) To: Luke-Jr; +Cc: Linux Kernel Mailing List, Greg KH On Tuesday 06 December 2005 04:34, Luke-Jr wrote: > > > Nope, but I don't see how udev can possibly detect something that > > > doesn't let the OS know it's there-- except, of course, loading the > > > driver for it and seeing if it works. > > > > Stuff shows up in /sys whether or not Linux has a driver loaded for it. > > Only if Linux is aware it exists. I'm thinking of those old ISA cards and > such. A) This is only true for obsolete hardware. Can you name an example that's currently being manufactured? (I'm trying to figure out if serial mice count, not that you really need kernel support to detect those.) B) You can insmod the module from userspace to actively probe for stuff. If the kernel doesn't know either until it probes, and you can trigger the probe at will, what additional kernel support do you need? Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 5:59 ` Luke-Jr 2005-12-06 0:34 ` Rob Landley @ 2005-12-06 17:38 ` Greg KH 1 sibling, 0 replies; 639+ messages in thread From: Greg KH @ 2005-12-06 17:38 UTC (permalink / raw) To: Luke-Jr; +Cc: Linux Kernel Mailing List On Mon, Dec 05, 2005 at 05:59:33AM +0000, Luke-Jr wrote: > On Sunday 04 December 2005 23:22, Greg KH wrote: > > On Sun, Dec 04, 2005 at 04:46:31AM +0000, Luke-Jr wrote: > > > Well, devfs does have some abilities udev doesn't: hotplug/udev > > > doesn't detect everything, and can result in rarer or non-PnP devices > > > not being automatically available; > > > > Are you sure about that today? > > Nope, but I don't see how udev can possibly detect something that > doesn't let the OS know it's there-- except, of course, loading the > driver for it and seeing if it works. > > > And udev wasn't created to do everything that devfs does. > > Which might be a case for leaving devfs in. *shrug* Heh, no. Please go look up why devfs was deleted, this one broken feature is not a good enough reason to keep it around. > > And devfs can't do everything that udev can (by far...) > > Didn't say it could... > > > > Interesting effects of switching my desktop from devfs to udev: > > > 1. my DVD burners are left uninitialized until I manually modprobe ide-cd > > > or (more recently) ide-scsi > > > > Sounds like a broken distro configuration :) > > Well, I was assuming you kept Gentoo's udev packages up to date. ;) > [ebuild R ] sys-fs/udev-070-r1 (-selinux) -static 429 kB That's the latest stable udev release in Gentoo, yes. Do you have problems with that one? If so, please file them in the proper Gentoo bugzilla. And yes, Gentoo is lagging a bit on the latest udev support (073 is in the tree, but 076 has been released.) That's totally my fault, and you can blame my lack of free time to do what is needed to fully intregrate it (due to some very good snow in our local mountains...) > > > devfs also has the advantage of keeping the module info all in one > > > place-- the kernel or the module. > > > In particular, with udev the detection and /dev info is scattered into > > > different locations of the filesystem. This can probably be fixed > > > easily simply by having udev read such info from modules or via a /sys > > > entry, though. > > > > What information are you talking about here? > > I'm assuming everything in /etc/udev/rules.d/50-udev.rules used to be > in the kernel for devfs-- perhaps it was PAM though, I'm not sure. That is not true, it was not. > Other than that, I don't expect that simply installing a new kernel > module will allow the device to be detected automatically, but that > some hotplug or udev configurations will need to be updated also. Have you tried this? What "new kernel module" are you speaking of? thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 23:22 ` Greg KH 2005-12-05 5:59 ` Luke-Jr @ 2005-12-06 14:58 ` Bill Davidsen 2005-12-06 17:59 ` Greg KH 2005-12-10 2:16 ` Rob Landley 1 sibling, 2 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-06 14:58 UTC (permalink / raw) To: Greg KH; +Cc: Linux Kernel Mailing List Greg KH wrote: > On Sun, Dec 04, 2005 at 04:46:31AM +0000, Luke-Jr wrote: > >>Well, devfs does have some abilities udev doesn't: hotplug/udev >>doesn't detect everything, and can result in rarer or non-PnP devices >>not being automatically available; > > > Are you sure about that today? And udev wasn't created to do everything > that devfs does. And devfs can't do everything that udev can (by > far...) > > >>devfs has the effect of trying to load a module when a program looks >>for the devices it provides-- while it can cause problems, it does >>have a possibility to work better. > > > Sorry, but that model of loading modules is very broken and it is good > that we don't do that anymore (as you allude to.) > > >>Interesting effects of switching my desktop from devfs to udev: >>1. my DVD burners are left uninitialized until I manually modprobe ide-cd or >>(more recently) ide-scsi > > > Sounds like a broken distro configuration :) > > >>2. my sound card is autodetected and the drivers loaded, but the OSS emulation >>modules are omitted; with devfs, they would be autoloaded when an app tried >>to use OSS > > > Again, broken distro configuration :) If a new udev config is needed with every new kernel, why isn't it in the kernel tarball? Is that what you mean by "broken distro configuration?" The info should be in /proc or /sys and not in an external config file, particularly if a different versions per-kernel is needed and people are trying new kernels and perhaps falling back to the old. Have "make install" drop the udev config in /boot like the initrd file. The the boot could create an slink to "/boot/$(uname -r)-udev" or some such. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 14:58 ` Bill Davidsen @ 2005-12-06 17:59 ` Greg KH 2005-12-06 21:10 ` Bill Davidsen 2005-12-10 2:16 ` Rob Landley 1 sibling, 1 reply; 639+ messages in thread From: Greg KH @ 2005-12-06 17:59 UTC (permalink / raw) To: Bill Davidsen; +Cc: Linux Kernel Mailing List On Tue, Dec 06, 2005 at 09:58:54AM -0500, Bill Davidsen wrote: > > If a new udev config is needed with every new kernel, why isn't it in > the kernel tarball? Is that what you mean by "broken distro > configuration?" The info should be in /proc or /sys and not in an > external config file, particularly if a different versions per-kernel is > needed and people are trying new kernels and perhaps falling back to the > old. Every distro has different needs for its device naming and groups and other intergration into the boot process. To force all of them to unify on one-grand-way-of-doing-things would just not work out at all. Look at all of the variations in the udev tarball between the different vendor configurations (we put them in there for other people to base their distro off of, if they want to.) So providing this config in the kernel will just not work, sorry. greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 17:59 ` Greg KH @ 2005-12-06 21:10 ` Bill Davidsen 2005-12-06 21:51 ` Kay Sievers 0 siblings, 1 reply; 639+ messages in thread From: Bill Davidsen @ 2005-12-06 21:10 UTC (permalink / raw) To: Greg KH; +Cc: Linux Kernel Mailing List Greg KH wrote: > On Tue, Dec 06, 2005 at 09:58:54AM -0500, Bill Davidsen wrote: > >>If a new udev config is needed with every new kernel, why isn't it in >>the kernel tarball? Is that what you mean by "broken distro >>configuration?" The info should be in /proc or /sys and not in an >>external config file, particularly if a different versions per-kernel is >>needed and people are trying new kernels and perhaps falling back to the >>old. > > > Every distro has different needs for its device naming and groups and > other intergration into the boot process. To force all of them to unify > on one-grand-way-of-doing-things would just not work out at all. Did I say that. No, I said it would be desirable to provide a working config with the kernel, to which something could be symlinked. This no more "forces" distributions to do anything than LSB. It would provide a default, it would provide something working, and if I didn't like it I could change it. But I wouldn't have to try and change thing way up in initrd so I can boot one kernel or another... > > Look at all of the variations in the udev tarball between the different > vendor configurations (we put them in there for other people to base > their distro off of, if they want to.) > > So providing this config in the kernel will just not work, sorry. We have standard libraries, header files, system calls, why is a standard in this case a bad thing? Actually not even a standard, perhaps, a default. It wouldn't make it one bit harder to have custom names, for those who believe different is better. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 21:10 ` Bill Davidsen @ 2005-12-06 21:51 ` Kay Sievers 0 siblings, 0 replies; 639+ messages in thread From: Kay Sievers @ 2005-12-06 21:51 UTC (permalink / raw) To: Bill Davidsen; +Cc: Greg KH, Linux Kernel Mailing List On Tue, Dec 06, 2005 at 04:10:12PM -0500, Bill Davidsen wrote: > Greg KH wrote: > >On Tue, Dec 06, 2005 at 09:58:54AM -0500, Bill Davidsen wrote: > > > >>If a new udev config is needed with every new kernel, why isn't it in > >>the kernel tarball? Is that what you mean by "broken distro > >>configuration?" The info should be in /proc or /sys and not in an > >>external config file, particularly if a different versions per-kernel is > >>needed and people are trying new kernels and perhaps falling back to the > >>old. > > > >Every distro has different needs for its device naming and groups and > >other intergration into the boot process. To force all of them to unify > >on one-grand-way-of-doing-things would just not work out at all. > > Did I say that. No, I said it would be desirable to provide a working > config with the kernel, to which something could be symlinked. This no > more "forces" distributions to do anything than LSB. It would provide a > default, it would provide something working, and if I didn't like it I > could change it. But I wouldn't have to try and change thing way up in > initrd so I can boot one kernel or another... That already works today. All distros I know are capable to run kernel.org kernels, if you care yourself, that the rootfs is accessible. > >Look at all of the variations in the udev tarball between the different > >vendor configurations (we put them in there for other people to base > >their distro off of, if they want to.) > > > >So providing this config in the kernel will just not work, sorry. > > We have standard libraries, header files, system calls, why is a > standard in this case a bad thing? Actually not even a standard, > perhaps, a default. It wouldn't make it one bit harder to have custom > names, for those who believe different is better. Just give it its time. It will happen without anybody claiming to have or to be a standard. We are already much more similar than we've ever been across the current devel releases of all major distros. The complete replacement of hotplug by udev rules, kernel uevents and kernel event replay triggers made the situation so much simpler and better than it ever was. It's, as in most cases, not about someone defining some standard, talk a lot about it, create an interest group, write a noisy paper... - it's the whole lot of real work to come up with a solution that is convincing enough for the involved parties, and to manage all the different interests people have and still get things done at the same time. It has almost nothing to do with a "standard". Convergence will just happen, cause it makes sense and there is a reasonable way to do it. And there was no such thing in that area in the past that made this kind of sense. And yeah, the never ending discussion about stupid options like devfs does not help anything here. It should have been removed a long time ago, so that the the confused people start to walk in the right direction instead of delaying the needed convergence progress again and again. Thanks, Kay ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 14:58 ` Bill Davidsen 2005-12-06 17:59 ` Greg KH @ 2005-12-10 2:16 ` Rob Landley 2005-12-10 4:04 ` Greg KH 1 sibling, 1 reply; 639+ messages in thread From: Rob Landley @ 2005-12-10 2:16 UTC (permalink / raw) To: Bill Davidsen; +Cc: Greg KH, Linux Kernel Mailing List On Tuesday 06 December 2005 08:58, Bill Davidsen wrote: > Greg KH wrote: > > On Sun, Dec 04, 2005 at 04:46:31AM +0000, Luke-Jr wrote: > >>Well, devfs does have some abilities udev doesn't: hotplug/udev > >>doesn't detect everything, and can result in rarer or non-PnP devices > >>not being automatically available; > > > > Are you sure about that today? And udev wasn't created to do everything > > that devfs does. And devfs can't do everything that udev can (by > > far...) > > > >>devfs has the effect of trying to load a module when a program looks > >>for the devices it provides-- while it can cause problems, it does > >>have a possibility to work better. > > > > Sorry, but that model of loading modules is very broken and it is good > > that we don't do that anymore (as you allude to.) > > > >>Interesting effects of switching my desktop from devfs to udev: > >>1. my DVD burners are left uninitialized until I manually modprobe ide-cd > >> or (more recently) ide-scsi > > > > Sounds like a broken distro configuration :) > > > >>2. my sound card is autodetected and the drivers loaded, but the OSS > >> emulation modules are omitted; with devfs, they would be autoloaded when > >> an app tried to use OSS > > > > Again, broken distro configuration :) > > If a new udev config is needed with every new kernel, why isn't it in > the kernel tarball? Why isn't inittab in the kernel tarball? I have a shell script that initializes /dev. (I've posted it here a few times, somebody ported it to C, and a micro-udev replacement will go into busybox in 1.2.) Why isn't there a command shell in the kernel tarball? Kinda hard to use your system without a shell... As far as I can tell, what broke with udev was their embedded version of "libsysfs", which is an abstraction layer I've _never_ understood the point of. (Because opening single value files in /sys is just too hard. Nobody needed a "libproc", the parsing of which is actual work, but they felt a need a libsysfs. Uh-huh...) Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-10 2:16 ` Rob Landley @ 2005-12-10 4:04 ` Greg KH 0 siblings, 0 replies; 639+ messages in thread From: Greg KH @ 2005-12-10 4:04 UTC (permalink / raw) To: Rob Landley; +Cc: Bill Davidsen, Linux Kernel Mailing List On Fri, Dec 09, 2005 at 08:16:42PM -0600, Rob Landley wrote: > As far as I can tell, what broke with udev was their embedded version of > "libsysfs", which is an abstraction layer I've _never_ understood the point > of. (Because opening single value files in /sys is just too hard. Nobody > needed a "libproc", the parsing of which is actual work, but they felt a need > a libsysfs. Uh-huh...) The original goal of libsysfs was to have a library that handled all of the direct sysfs calls, and have it create structures that looked something like what sysfs exported (devices, busses, etc.) Then, if things changed in the sysfs layout or structure, only libsysfs would need to be changed, and all apps that used it would continue to work just fine. But in reality, it was only used by udev, and was very fragile. So fragile udev is thinking of ripping it out as it has had a lot of problems in the past. Hope this explains things better. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 0:20 ` Greg KH 2005-12-04 4:46 ` Luke-Jr @ 2005-12-05 22:47 ` Rob Landley 2005-12-05 23:05 ` Benjamin LaHaise 2005-12-06 3:15 ` Greg KH 1 sibling, 2 replies; 639+ messages in thread From: Rob Landley @ 2005-12-05 22:47 UTC (permalink / raw) To: Greg KH; +Cc: linux-kernel On Saturday 03 December 2005 18:20, Greg KH wrote: > On Sat, Dec 03, 2005 at 11:50:20PM +0100, Matthias Andree wrote: > > The point is, removing something that has worked well enough that some > > people had a reason to use it, is not "stable". > > Please remember, no one is calling 2.6 "stable" anymore than they are > calling it "development". The current development model is different > than what we used to do pre 2.6. See the archives for details about > this if you want more information. > > > Third, IF udev is so sexy but OTOH a real kernel-space devfs can be done > > in 200 LoC as has been claimed so often, > > 282 LoC: ... > > why in hell is this not happening? > > Because it's not the correct solution. More detail on this: On the busybox list we're currently working out a design for mdev, the micro-udev that'll go into busybox 1.2. So we're thinking about this issue pretty carefully, as we speak. What's the minimal amount of work we can't get away with not doing? And much as we'd like to, we can't eliminate the config file. In mdev we can accept the kernel's suggested names for devices, throw everything into a single directory with no subdirectories, even configure out hotplugging support (since not all embedded devices need that). But nowhere in sys is there any hint about the correct ownership and permissions for a device, and you can't create a device node without specifying that. The fundamental problem is that the kernel _can't_ tell us this through /sys because the kernel has no idea what users and groups are on a given system. It can't, and it shouldn't. That's not it's job. (It deals with uid and gid and never looks at /etc/passwd or /etc/groups. And if it doesn't know who should own what, it can't know what permissions they should have either.) So no in-kernel filesystem can get this right without help from userspace (even devfs had devfsd), and as soon as you've got a userspace daemon to tell the kernel who is who you might as well do the whole thing there, now that the kernel is exporting everyting _else_ we need to know via /sys and /sbin/hotplug. Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 22:47 ` Rob Landley @ 2005-12-05 23:05 ` Benjamin LaHaise 2005-12-06 3:19 ` Rob Landley 2005-12-06 10:51 ` Matthias Andree 2005-12-06 3:15 ` Greg KH 1 sibling, 2 replies; 639+ messages in thread From: Benjamin LaHaise @ 2005-12-05 23:05 UTC (permalink / raw) To: Rob Landley; +Cc: Greg KH, linux-kernel On Mon, Dec 05, 2005 at 04:47:55PM -0600, Rob Landley wrote: > So no in-kernel filesystem can get this right without help from userspace > (even devfs had devfsd), and as soon as you've got a userspace daemon to tell > the kernel who is who you might as well do the whole thing there, now that > the kernel is exporting everyting _else_ we need to know via /sys > and /sbin/hotplug. /sbin/hotplug is suboptimal. Even a pretty fast machine is slowed down pretty significantly by the ~thousand fork and exec that take place during startup. For the most common devices -- common tty, pty, floppy, etc that every system has, this is a plain waste of resources -- otherwise known as bloat. -ben -- "You know, I've seen some crystals do some pretty trippy shit, man." Don't Email: <dont@kvack.org>. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 23:05 ` Benjamin LaHaise @ 2005-12-06 3:19 ` Rob Landley 2005-12-06 3:32 ` Benjamin LaHaise 2005-12-06 10:51 ` Matthias Andree 1 sibling, 1 reply; 639+ messages in thread From: Rob Landley @ 2005-12-06 3:19 UTC (permalink / raw) To: Benjamin LaHaise; +Cc: Greg KH, linux-kernel On Monday 05 December 2005 17:05, Benjamin LaHaise wrote: > On Mon, Dec 05, 2005 at 04:47:55PM -0600, Rob Landley wrote: > > So no in-kernel filesystem can get this right without help from userspace > > (even devfs had devfsd), and as soon as you've got a userspace daemon to > > tell the kernel who is who you might as well do the whole thing there, > > now that the kernel is exporting everyting _else_ we need to know via > > /sys and /sbin/hotplug. > > /sbin/hotplug is suboptimal. Even a pretty fast machine is slowed down > pretty significantly by the ~thousand fork and exec that take place during > startup. Why do you need hotplug events on startup? Can't you just scan /sys for "dev" entries do the initial populate of /dev from that? > For the most common devices -- common tty, pty, floppy, etc that > every system has, this is a plain waste of resources -- otherwise known as > bloat. I get those from a scan of /sys, and only care about hotplug events that come in after that. (Could just be me...) > -ben Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 3:19 ` Rob Landley @ 2005-12-06 3:32 ` Benjamin LaHaise 2005-12-06 5:49 ` Rob Landley 0 siblings, 1 reply; 639+ messages in thread From: Benjamin LaHaise @ 2005-12-06 3:32 UTC (permalink / raw) To: Rob Landley; +Cc: Greg KH, linux-kernel On Mon, Dec 05, 2005 at 09:19:28PM -0600, Rob Landley wrote: > > /sbin/hotplug is suboptimal. Even a pretty fast machine is slowed down > > pretty significantly by the ~thousand fork and exec that take place during > > startup. > > Why do you need hotplug events on startup? Can't you just scan /sys for "dev" > entries do the initial populate of /dev from that? That's my point: I don't. Yet the kernel tries to exec /sbin/hotplug on startup around 1000 times. -ben -- "You know, I've seen some crystals do some pretty trippy shit, man." Don't Email: <dont@kvack.org>. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 3:32 ` Benjamin LaHaise @ 2005-12-06 5:49 ` Rob Landley 0 siblings, 0 replies; 639+ messages in thread From: Rob Landley @ 2005-12-06 5:49 UTC (permalink / raw) To: Benjamin LaHaise; +Cc: Greg KH, linux-kernel On Monday 05 December 2005 21:32, Benjamin LaHaise wrote: > On Mon, Dec 05, 2005 at 09:19:28PM -0600, Rob Landley wrote: > > > /sbin/hotplug is suboptimal. Even a pretty fast machine is slowed down > > > pretty significantly by the ~thousand fork and exec that take place > > > during startup. > > > > Why do you need hotplug events on startup? Can't you just scan /sys for > > "dev" entries do the initial populate of /dev from that? > > That's my point: I don't. Yet the kernel tries to exec /sbin/hotplug on > startup around 1000 times. > > -ben At what stage? If it's initramfs, then don't have one on initramfs. (Not by default anyway, add a symlink when you're ready to start caring, or write the correct path to /proc/sys/heeeeeeere's_hotplog.) Failure to exec 1000 times shouldn't take too long. I have shell scripts that fork and exec 1000 times in under a second, and they're actually doing something. Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 23:05 ` Benjamin LaHaise 2005-12-06 3:19 ` Rob Landley @ 2005-12-06 10:51 ` Matthias Andree 1 sibling, 0 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-06 10:51 UTC (permalink / raw) To: linux-kernel On Mon, 05 Dec 2005, Benjamin LaHaise wrote: > On Mon, Dec 05, 2005 at 04:47:55PM -0600, Rob Landley wrote: > > So no in-kernel filesystem can get this right without help from userspace > > (even devfs had devfsd), and as soon as you've got a userspace daemon to tell > > the kernel who is who you might as well do the whole thing there, now that > > the kernel is exporting everyting _else_ we need to know via /sys > > and /sbin/hotplug. > > /sbin/hotplug is suboptimal. Even a pretty fast machine is slowed down > pretty significantly by the ~thousand fork and exec that take place during > startup. For the most common devices -- common tty, pty, floppy, etc that > every system has, this is a plain waste of resources -- otherwise known as > bloat. You mean that distro boot scripts now need to wait for 20 seconds until hotplug has finally handled all the coldplug events so the script can finally slap the IPv4 address onto the interface or start dhcpcd. :-) -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 22:47 ` Rob Landley 2005-12-05 23:05 ` Benjamin LaHaise @ 2005-12-06 3:15 ` Greg KH 2005-12-06 3:23 ` Rob Landley 1 sibling, 1 reply; 639+ messages in thread From: Greg KH @ 2005-12-06 3:15 UTC (permalink / raw) To: Rob Landley; +Cc: linux-kernel On Mon, Dec 05, 2005 at 04:47:55PM -0600, Rob Landley wrote: > On the busybox list we're currently working out a design for mdev, the > micro-udev that'll go into busybox 1.2. So we're thinking about this issue > pretty carefully, as we speak. What's the minimal amount of work we can't > get away with not doing? I suggest you take this discussion to the linux-hotplug-devel mailing list, which is the proper place for it. Not this thread about stable kernel series :) thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 3:15 ` Greg KH @ 2005-12-06 3:23 ` Rob Landley 0 siblings, 0 replies; 639+ messages in thread From: Rob Landley @ 2005-12-06 3:23 UTC (permalink / raw) To: Greg KH; +Cc: linux-kernel On Monday 05 December 2005 21:15, Greg KH wrote: > On Mon, Dec 05, 2005 at 04:47:55PM -0600, Rob Landley wrote: > > On the busybox list we're currently working out a design for mdev, the > > micro-udev that'll go into busybox 1.2. So we're thinking about this > > issue pretty carefully, as we speak. What's the minimal amount of work > > we can't get away with not doing? > > I suggest you take this discussion to the linux-hotplug-devel mailing > list, which is the proper place for it. Not this thread about stable > kernel series :) Didn't know there was one. Will do, but probably not today... > thanks, > > greg k-h Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:27 ` Matthias Andree 2005-12-03 22:34 ` Lee Revell @ 2005-12-03 22:36 ` David Ranson 2005-12-03 22:50 ` Matthias Andree 2005-12-06 14:59 ` Bill Davidsen 2005-12-04 1:04 ` Horst von Brand 2 siblings, 2 replies; 639+ messages in thread From: David Ranson @ 2005-12-03 22:36 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel [-- Attachment #1: Type: text/plain, Size: 822 bytes --] Matthias Andree wrote: >So was I. And now what? ipfwadm and ipchains should have been removed >from 2.6.0 if 2.6.0 was not to support these. That opportunity was >missed, the removal wasn't made up for in 2.6.1, so the stuff has to >stick until 2.8.0. > > I'm not aware of that policy... maybe I overlooked something? >This doesn't matter. A kernel that calls itself stable CAN NOT remove >features unless they had been critically broken from the beginning. And >this level of breakage is a moot point, so removal is not justified. > > >Who cares what you or I use? It's a commonly acknowledged policy that >"stable" releases do not remove features that are good enough for some. >Linux 2.6 is not "stable" in this regard. > > I guess our definitions of stable (and the degree of stability acceptable) differ. David [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 187 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:36 ` David Ranson @ 2005-12-03 22:50 ` Matthias Andree 2005-12-06 14:59 ` Bill Davidsen 1 sibling, 0 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-03 22:50 UTC (permalink / raw) To: linux-kernel On Sat, 03 Dec 2005, David Ranson wrote: > Matthias Andree wrote: > > >So was I. And now what? ipfwadm and ipchains should have been removed > >from 2.6.0 if 2.6.0 was not to support these. That opportunity was > >missed, the removal wasn't made up for in 2.6.1, so the stuff has to > >stick until 2.8.0. > > > > > I'm not aware of that policy... maybe I overlooked something? Everyone does it, except Linux and GDBM. > I guess our definitions of stable (and the degree of stability > acceptable) differ. No need to guess, this is quite obvious. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:36 ` David Ranson 2005-12-03 22:50 ` Matthias Andree @ 2005-12-06 14:59 ` Bill Davidsen 1 sibling, 0 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-06 14:59 UTC (permalink / raw) To: David Ranson; +Cc: linux-kernel David Ranson wrote: > Matthias Andree wrote: > > >>So was I. And now what? ipfwadm and ipchains should have been removed > >>from 2.6.0 if 2.6.0 was not to support these. That opportunity was > >>missed, the removal wasn't made up for in 2.6.1, so the stuff has to >>stick until 2.8.0. >> >> > > I'm not aware of that policy... maybe I overlooked something? Until 2.6, a stable series did not remove existing features, so someone building an rpm or deb package could release it for "2.4.12 or later" and expect it to work. As a for instance there are people who went to 2.6 and kept their old firwall rules written in ipchains, because they still worked. Now if ipchains are deleted a full rewrite of firewall rules is needed, and that just shouldn't be don't in haste. My personal opinion is that ipfwadm and ipchains should have followed some other features into the night before 2.6.0 ever came out. No one would have gotten a nasty surprise later. I also think that reiser4 is 2.7 material, if there were a 2.7. I didn't see all that much wrong with the old odd/even model to tell the truth, it wasn't perfect but you knew what you got. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:27 ` Matthias Andree 2005-12-03 22:34 ` Lee Revell 2005-12-03 22:36 ` David Ranson @ 2005-12-04 1:04 ` Horst von Brand 2005-12-04 12:07 ` Matthias Andree 2005-12-06 20:01 ` Bill Davidsen 2 siblings, 2 replies; 639+ messages in thread From: Horst von Brand @ 2005-12-04 1:04 UTC (permalink / raw) To: linux-kernel Matthias Andree <matthias.andree@gmx.de> wrote: > On Sat, 03 Dec 2005, David Ranson wrote: > > Adrian Bunk wrote: > > > > >- support for ipfwadm and ipchains was removed during 2.6 > > Surely this one had loads of notice though? I was using iptables with > > 2.4 kernels. Sure had. They were scheduled for removal in march, 2005 a long time ago. > So was I. And now what? ipfwadm and ipchains should have been removed > from 2.6.0 if 2.6.0 was not to support these. Or in 2.6.10, or 2.6.27, or whatever. > That opportunity was > missed, the removal wasn't made up for in 2.6.1, so the stuff has to > stick until 2.8.0. Sorry, but the new development model is that there is no "uneven" series anymore. Sure, it /might/ open for worldshattering changes, but nothing of that sort is remotely in sight right now, so... > > >- devfs support was removed during 2.6 > > > > Did this affect many 'real' users? > This doesn't matter. A kernel that calls itself stable CAN NOT remove > features unless they had been critically broken from the beginning. And > this level of breakage is a moot point, so removal is not justified. devfs was broken, and very little used. > > >- removal of kernel support for pcmcia-cs is pending > > >- ip{,6}_queue removal is pending > > >- removal of the RAW driver is pending > > > I don't use any of these. I guess pcmcia-cs may be disruptive for laptop > > users. > Who cares what you or I use? It's a commonly acknowledged policy that > "stable" releases do not remove features that are good enough for some. Right, for a suitably large set of "some". If the set is too small... > Linux 2.6 is not "stable" in this regard. Right. The idea of "stable series" had to go. And went. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 1:04 ` Horst von Brand @ 2005-12-04 12:07 ` Matthias Andree 2005-12-04 19:29 ` Horst von Brand 2005-12-06 20:01 ` Bill Davidsen 1 sibling, 1 reply; 639+ messages in thread From: Matthias Andree @ 2005-12-04 12:07 UTC (permalink / raw) To: linux-kernel On Sat, 03 Dec 2005, Horst von Brand wrote: > Matthias Andree <matthias.andree@gmx.de> wrote: > > On Sat, 03 Dec 2005, David Ranson wrote: > > > Adrian Bunk wrote: > > > > > > >- support for ipfwadm and ipchains was removed during 2.6 > > > > Surely this one had loads of notice though? I was using iptables with > > > 2.4 kernels. > > Sure had. They were scheduled for removal in march, 2005 a long time ago. > > > So was I. And now what? ipfwadm and ipchains should have been removed > > from 2.6.0 if 2.6.0 was not to support these. > > Or in 2.6.10, or 2.6.27, or whatever. No. If you need to remove major components, it is only diligent to bump the minor revision and call the beast 2.7.0. At that time, not only one or two subsystems, but all that were marked deprecated for 6 months or so, should be dropped. > > This doesn't matter. A kernel that calls itself stable CAN NOT remove > > features unless they had been critically broken from the beginning. And > > this level of breakage is a moot point, so removal is not justified. > > devfs was broken, and very little used. OK. This however doesn't hold for ipfwadm (which should probably never have made it into 2.6.0 in the first place) or ipchains. > > Linux 2.6 is not "stable" in this regard. > > Right. The idea of "stable series" had to go. And went. So what is the point in using Linux anyhow if the kernel developers don't care for the outside world, one might ask? What is in the way of reflecting feature removals in the minor version of the project, say, remove devfs, ipfwadm, ipchains and whatnot in one go and call the new release without this legacies 2.7.0? -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 12:07 ` Matthias Andree @ 2005-12-04 19:29 ` Horst von Brand 0 siblings, 0 replies; 639+ messages in thread From: Horst von Brand @ 2005-12-04 19:29 UTC (permalink / raw) To: linux-kernel Matthias Andree <matthias.andree@gmx.de> wrote: > On Sat, 03 Dec 2005, Horst von Brand wrote: > > Matthias Andree <matthias.andree@gmx.de> wrote: > > > On Sat, 03 Dec 2005, David Ranson wrote: > > > > Adrian Bunk wrote: > > > > > > > > >- support for ipfwadm and ipchains was removed during 2.6 > > > > > > Surely this one had loads of notice though? I was using iptables with > > > > 2.4 kernels. > > Sure had. They were scheduled for removal in march, 2005 a long time ago. > > > > > So was I. And now what? ipfwadm and ipchains should have been removed > > > from 2.6.0 if 2.6.0 was not to support these. > > > > Or in 2.6.10, or 2.6.27, or whatever. > No. If you need to remove major components, it is only diligent to bump > the minor revision and call the beast 2.7.0. In the case of ipfwadm and ipchains, that was precisely the changeover to 2.6... ipfwadm is from 2.2, obsoleted by ipchains in 2.4, and both axed in the timespan promised. devfs was never widely used, and was also announced to be killed by 2.6, IIRC. > At that time, not only one > or two subsystems, but all that were marked deprecated for 6 months or > so, should be dropped. That /is/ what is happening, modulo needless changes in version number. > > > This doesn't matter. A kernel that calls itself stable CAN NOT remove > > > features unless they had been critically broken from the beginning. And > > > this level of breakage is a moot point, so removal is not justified. > > > > devfs was broken, and very little used. > > OK. This however doesn't hold for ipfwadm (which should probably never > have made it into 2.6.0 in the first place) or ipchains. They were carried along exactly to keep the promise of killing them off by a certain date. It seems you are going by what is called around here "Palos porque bogas, palos porque no bogas" (roughly translated, "get whipped for rowing, and get whipped for not rowing"). > > > Linux 2.6 is not "stable" in this regard. > > Right. The idea of "stable series" had to go. And went. > So what is the point in using Linux anyhow if the kernel developers > don't care for the outside world, one might ask? Sorry, but again: - Userland API is remarkably stable, only some stuff with intimate kernel connection have ever changed - In-kernel API has /never/ been stable. At all. Sure, changes are much faster now (the community is larger, the tools are better, ....). > What is in the way of > reflecting feature removals in the minor version of the project, say, > remove devfs, ipfwadm, ipchains and whatnot in one go and call the new > release without this legacies 2.7.0? It /was/ called 2.6... -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 1:04 ` Horst von Brand 2005-12-04 12:07 ` Matthias Andree @ 2005-12-06 20:01 ` Bill Davidsen 1 sibling, 0 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-06 20:01 UTC (permalink / raw) To: Horst von Brand, Linux Kernel Mailing List Horst von Brand wrote: > Matthias Andree <matthias.andree@gmx.de> wrote: > >>On Sat, 03 Dec 2005, David Ranson wrote: >> >>>Adrian Bunk wrote: >>> >>> >>>>- support for ipfwadm and ipchains was removed during 2.6 > > >>>Surely this one had loads of notice though? I was using iptables with >>>2.4 kernels. > > > Sure had. They were scheduled for removal in march, 2005 a long time ago. > > >>So was I. And now what? ipfwadm and ipchains should have been removed >>from 2.6.0 if 2.6.0 was not to support these. > > > Or in 2.6.10, or 2.6.27, or whatever. > > >> That opportunity was >>missed, the removal wasn't made up for in 2.6.1, so the stuff has to >>stick until 2.8.0. > > > Sorry, but the new development model is that there is no "uneven" series > anymore. Sure, it /might/ open for worldshattering changes, but nothing of > that sort is remotely in sight right now, so... > > >>>>- devfs support was removed during 2.6 >>> >>>Did this affect many 'real' users? > > >>This doesn't matter. A kernel that calls itself stable CAN NOT remove >>features unless they had been critically broken from the beginning. And >>this level of breakage is a moot point, so removal is not justified. > > > devfs was broken, and very little used. Perhaps there is a cause and effect relationship? If devfs worked I don't see the need for every distro to have it's own udev (or mdev or sdev or whatever the flavor is this month). -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 18:34 ` RFC: Starting a stable kernel series off the 2.6 kernel David Ranson 2005-12-03 22:27 ` Matthias Andree @ 2005-12-05 20:43 ` Bill Davidsen 1 sibling, 0 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-05 20:43 UTC (permalink / raw) To: David Ranson; +Cc: Steven Rostedt, linux-kernel, Matthias Andree David Ranson wrote: > Adrian Bunk wrote: > > >>- support for ipfwadm and ipchains was removed during 2.6 >> >> > > Surely this one had loads of notice though? I was using iptables with > 2.4 kernels. > > >>- devfs support was removed during 2.6 >> >> > > Did this affect many 'real' users? > > >>- removal of kernel support for pcmcia-cs is pending >>- ip{,6}_queue removal is pending >>- removal of the RAW driver is pending >> >> > > I don't use any of these. I guess pcmcia-cs may be disruptive for laptop > users. You don't seem to grasp that thousands of people DO use these features, and by removing the features those users are blocked from security, reliability, and performance related changes. And there are a number of other features mentioned > > So far I don't see evidence to suggest huge repeated userspace breakages > between Kernel versions that were implied earlier in this thread. > Whatever, we aren't going to see any more stable branches without > volunteers to do the spadework. As has been pointed out, this won't > always be an easy task. To a large extent I don't think it's a needed task. If new stuff doesn't work that doesn't hurt established uses, it's only when changes like the PCI rethink go in that existing users are impacted. As long as things aren't taken OUT, the current kernel is usefully stable. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 17:53 ` Adrian Bunk 2005-12-03 18:17 ` newbie - mdadm create raid1 mirrors on large drives Larry Bates 2005-12-03 18:34 ` RFC: Starting a stable kernel series off the 2.6 kernel David Ranson @ 2005-12-05 3:31 ` Rob Landley 2005-12-05 16:17 ` Mark Lord 2005-12-05 18:44 ` Adrian Bunk 2 siblings, 2 replies; 639+ messages in thread From: Rob Landley @ 2005-12-05 3:31 UTC (permalink / raw) To: Adrian Bunk; +Cc: David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Saturday 03 December 2005 11:53, Adrian Bunk wrote: > On Sat, Dec 03, 2005 at 05:17:41PM +0000, David Ranson wrote: > > Steven Rostedt wrote: > > >udev ;) > > > > > >http://seclists.org/lists/linux-kernel/2005/Dec/0180.html > > > > Ahh OK .. I don't use it, so wouldn't have been affected. That's one > > userspace interface broken during the series, does anyone have any more? > > - support for ipfwadm and ipchains was removed during 2.6 > - devfs support was removed during 2.6 > - removal of kernel support for pcmcia-cs is pending > - ip{,6}_queue removal is pending > - removal of the RAW driver is pending So what you're upset about is the feature removal scheduling mechanism, which usually gives a full year's warning, and the removal patch can be reversed into a feature addition patch you can maintain outside the tree if you really care? Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 3:31 ` Rob Landley @ 2005-12-05 16:17 ` Mark Lord 2005-12-05 16:28 ` Lee Revell 2005-12-05 18:44 ` Adrian Bunk 1 sibling, 1 reply; 639+ messages in thread From: Mark Lord @ 2005-12-05 16:17 UTC (permalink / raw) To: Rob Landley Cc: Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree >>>Ahh OK .. I don't use it, so wouldn't have been affected. That's one >>>userspace interface broken during the series, does anyone have any more? Ah.. another one, that I was just reminded of again by the umpteenth person posting that their wireless no longer is WPA capable after upgrading from 2.6.12. Of course, the known solution for that issue is to upgrade to the recently "fixed" latest wpa_supplicant daemon in userspace, since the old one no longer works. Things like this are all too regular an occurance. Cheers ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 16:17 ` Mark Lord @ 2005-12-05 16:28 ` Lee Revell 2005-12-05 16:44 ` Matthias Andree ` (3 more replies) 0 siblings, 4 replies; 639+ messages in thread From: Lee Revell @ 2005-12-05 16:28 UTC (permalink / raw) To: Mark Lord Cc: Rob Landley, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Mon, 2005-12-05 at 11:17 -0500, Mark Lord wrote: > >>>Ahh OK .. I don't use it, so wouldn't have been affected. That's one > >>>userspace interface broken during the series, does anyone have any more? > > Ah.. another one, that I was just reminded of again > by the umpteenth person posting that their wireless > no longer is WPA capable after upgrading from 2.6.12. > > Of course, the known solution for that issue is to > upgrade to the recently "fixed" latest wpa_supplicant > daemon in userspace, since the old one no longer works. > > Things like this are all too regular an occurance. The distro should have solved this problem by making sure that the kernel upgrade depends on a new wpa_supplicant package. Don't they bother to test this stuff before they ship it?!? Lee ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 16:28 ` Lee Revell @ 2005-12-05 16:44 ` Matthias Andree 2005-12-05 17:17 ` Lee Revell 2005-12-05 17:58 ` Rob Landley ` (2 subsequent siblings) 3 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2005-12-05 16:44 UTC (permalink / raw) To: Lee Revell Cc: Mark Lord, Rob Landley, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Mon, 05 Dec 2005, Lee Revell wrote: > On Mon, 2005-12-05 at 11:17 -0500, Mark Lord wrote: > > >>>Ahh OK .. I don't use it, so wouldn't have been affected. That's one > > >>>userspace interface broken during the series, does anyone have any more? > > > > Ah.. another one, that I was just reminded of again > > by the umpteenth person posting that their wireless > > no longer is WPA capable after upgrading from 2.6.12. > > > > Of course, the known solution for that issue is to > > upgrade to the recently "fixed" latest wpa_supplicant > > daemon in userspace, since the old one no longer works. > > > > Things like this are all too regular an occurance. > > The distro should have solved this problem by making sure that the > kernel upgrade depends on a new wpa_supplicant package. Don't they > bother to test this stuff before they ship it?!? This constant shifting the blame on someone else is becoming offensive. Diligent maintainers put "INCOMPATIBLE CHANGES" sections up front in their release announcements or notes, that is the upstream maintainer's chance to state "wpa_supplicant version >= 1.2.3 required" and really pass the buck on to the distros. Without such upgrade-required-for: notes, it's just rude. "We break everything but you need to find out for yourself which..." Let's not mention that section 2, 7 and 9 manual pages should be maintained by the kernel developers rather than an external maintainer. If you need a luminous example how release management works, look at Postfix. I don't suggest taking Postfix's development model of printing the diffs and using text marker and pencil, but the "Incompatible changes", "Major changes" in Release Notes, with all the details in a separate changelog, works rather well for distros and direct users. My suggestion is to build upon the signed-off-by: stuff and require that every incompatible change carry such a RFC2822-header-like line if it is to be merged into baseline, and unconditionally back out all incompatibilities that are later found but not documented. Perhaps just making people actually write such notes can cut the number of these shipwrecks. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 16:44 ` Matthias Andree @ 2005-12-05 17:17 ` Lee Revell 2005-12-05 17:55 ` Matthias Andree 0 siblings, 1 reply; 639+ messages in thread From: Lee Revell @ 2005-12-05 17:17 UTC (permalink / raw) To: Matthias Andree Cc: Mark Lord, Rob Landley, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel On Mon, 2005-12-05 at 17:44 +0100, Matthias Andree wrote: > This constant shifting the blame on someone else is becoming > offensive. > > Diligent maintainers put "INCOMPATIBLE CHANGES" sections up front in > their release announcements or notes, that is the upstream > maintainer's chance to state "wpa_supplicant version >= 1.2.3 > required" and really pass the buck on to the distros. Without such > upgrade-required-for: notes, it's just rude. "We break everything but > you need to find out for > yourself which..." > I'm not trying to shift blame, I am just saying that with their access to a larger hardware and user base the distros are in a much better position to regression test changes than the kernel developers. And I didn't even mention the cases where the distros just don't do their homework. For example in order to insulate users from internal changes ALSA has a kernel and userspace (alsa-lib) component and both must be upgraded in sync to properly support new hardware. This is common knowledge. But many distros keep shipping kernel upgrades that introduce new ALSA drivers but don't bother to make the kernel upgrade depend on an alsa-lib upgrade, or even to make a newer alsa-lib available. Lee ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 17:17 ` Lee Revell @ 2005-12-05 17:55 ` Matthias Andree 2005-12-05 20:52 ` Florian Weimer 0 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2005-12-05 17:55 UTC (permalink / raw) To: Lee Revell Cc: Matthias Andree, Mark Lord, Rob Landley, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel On Mon, 05 Dec 2005, Lee Revell wrote: > On Mon, 2005-12-05 at 17:44 +0100, Matthias Andree wrote: > > This constant shifting the blame on someone else is becoming > > offensive. > > > > Diligent maintainers put "INCOMPATIBLE CHANGES" sections up front in > > their release announcements or notes, that is the upstream > > maintainer's chance to state "wpa_supplicant version >= 1.2.3 > > required" and really pass the buck on to the distros. Without such > > upgrade-required-for: notes, it's just rude. "We break everything but > > you need to find out for > > yourself which..." > > > > I'm not trying to shift blame, I am just saying that with their access > to a larger hardware and user base the distros are in a much better > position to regression test changes than the kernel developers. You just described what shifting burden or blame means. Are you seriously saying it's the distributors' fault for not trying the random monkey patch on end users machines? Heck, SUSE 9.2 ate a complete server because SUSE (they take the blame) didn't manage to (1) notice in time, (2) therefore package a CRITICAL (as in causes data corruption) MegaRAID bugfix. Do you really want such things to happen as intrinsic part of the kernel development? Do the upstream gurus such as Linus and Andrew want that? If so, they can say so and we'll see the companies running for their sheer lives and putting their money into other kernels. BSD makes it only easier to provide binary modules, because you don't even have to discuss with anyone if it's derived work or not, you can just embrace, extend and lock the beast up and everyone in. > And I didn't even mention the cases where the distros just don't do > their homework. For example in order to insulate users from internal > changes ALSA has a kernel and userspace (alsa-lib) component and both > must be upgraded in sync to properly support new hardware. This is > common knowledge. But many distros keep shipping kernel upgrades that > introduce new ALSA drivers but don't bother to make the kernel upgrade > depend on an alsa-lib upgrade, or even to make a newer alsa-lib > available. Major distros usually aim for small and well-audited changes in order not to make things worse, at least where end-user support is concerned. Given the development pace and the ridiculous policy which is effectively "you may break everything in the two weeks after release, and we'll collect those fixes that come in until Linus's machine works and ship without the rest". Basically, no-one should have permission to touch any core parts, except for fixes, until 2.7. Yes, that means going back to older models. Yes, that means that the discussions will start all over. And yes, that means that the cool stuff has to wait. Solution: release more often. I'm so bold as to claim that a new minor release every 6 months with a tight "only fixes allowed as core changes" policy would satisfy many. Of course you cannot break the binary driver of the day that way, but it also means fewer chances to break NFS, XFS, MegaRAID and whatnot. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 17:55 ` Matthias Andree @ 2005-12-05 20:52 ` Florian Weimer 2005-12-05 21:21 ` Steven Rostedt 2005-12-06 11:09 ` Matthias Andree 0 siblings, 2 replies; 639+ messages in thread From: Florian Weimer @ 2005-12-05 20:52 UTC (permalink / raw) To: Lee Revell Cc: Mark Lord, Rob Landley, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel * Matthias Andree: > Basically, no-one should have permission to touch any core parts, except > for fixes, until 2.7. Yes, that means going back to older models. Yes, > that means that the discussions will start all over. And yes, that means > that the cool stuff has to wait. Solution: release more often. Would this alone change much? I think what we really want is that our favorite branch (whatever it is) gets critical fixes forever (well, maybe one or two years, but this is forever). This is a bit unrealistic because everyone has a slightly different branchpoint. Releasing more often doesn't change that, really. In the security area, I think there is enough experience out there to collect data which would help each local branch maintainer to install the relevant fixes. But for general development, this seems to be infeasible, unless you focus your software architecture on this purpose (which is probably a terrible idea to do for kernel development). ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 20:52 ` Florian Weimer @ 2005-12-05 21:21 ` Steven Rostedt 2005-12-05 23:09 ` Rob Landley 2005-12-06 1:06 ` Florian Weimer 2005-12-06 11:09 ` Matthias Andree 1 sibling, 2 replies; 639+ messages in thread From: Steven Rostedt @ 2005-12-05 21:21 UTC (permalink / raw) To: Florian Weimer Cc: Lee Revell, Mark Lord, Rob Landley, Adrian Bunk, David Ranson, linux-kernel On Mon, 2005-12-05 at 21:52 +0100, Florian Weimer wrote: > * Matthias Andree: > > > Basically, no-one should have permission to touch any core parts, except > > for fixes, until 2.7. Yes, that means going back to older models. Yes, > > that means that the discussions will start all over. And yes, that means > > that the cool stuff has to wait. Solution: release more often. > > Would this alone change much? I think what we really want is that our > favorite branch (whatever it is) gets critical fixes forever (well, > maybe one or two years, but this is forever). This is a bit > unrealistic because everyone has a slightly different branchpoint. > Releasing more often doesn't change that, really. Maybe that is what is needed. A branch that all can use. Have every 5 or so 2.6.x become a "stable" branch. Where distributions and users can work together on keeping it stable. The rules to modifying such a branch would pretty much stay with what it already takes to modify the current 2.6.x.y branch. If you want a feature, you must either take the latest "unstable" 2.6.x branch or wait for the next "stable" 2.6.x branch to merge. Now who should chose which version the "stable" branch should be? Well, we could just say ever 5 branches will become one, or if we have a "F*cked up" branch (really bad bug made it in), then we can skip it and go to the 6th to branch. Perhaps, we could start out having Greg and Chris just concentrate on every fifth branch instead of every one, and that way the stability will last much longer. Again, if you want the latest functionality, you go with the latest "unstable" release, or wait for the next stable. Since these releases come out about every month or two, waiting 5 releases will last for almost a year. For this to work, the normal releases would just continue like normal. And just the marked branch will become stable. This may be similar to what Linus formally proposed. Where he had every odd revision be unstable, and every even stable. What I'm suggesting would not make the stable branch stable by what goes into it. It's just that those are the branches that would have the .y version. And then we could ignore the other branches instead. This idea combines pretty much the idea of the 2.7 with the current 2.6.x.y. Actually it is more like the 2.7 approach, but it's hidden :-) The problem with 2.7 is that nobody tests it, and it takes too long to go from 2.6 to 2.8. My method here hides that fact. You just basically say "here's the stable version" and let it fork. Continue on with the 2.6.x and when you think too many people are using the last stable version, and are not testing the current branch, just release the new "stable", and pull everyone back in. -- Steve ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 21:21 ` Steven Rostedt @ 2005-12-05 23:09 ` Rob Landley 2005-12-06 0:54 ` Steven Rostedt 2005-12-06 1:06 ` Florian Weimer 1 sibling, 1 reply; 639+ messages in thread From: Rob Landley @ 2005-12-05 23:09 UTC (permalink / raw) To: Steven Rostedt Cc: Florian Weimer, Lee Revell, Mark Lord, Adrian Bunk, David Ranson, linux-kernel On Monday 05 December 2005 15:21, Steven Rostedt wrote: > Perhaps, we could start out having Greg and Chris just concentrate on > every fifth branch instead of every one, and that way the stability will > last much longer. Ah, belling the cat. Hint: Any plan in a volunteer community that starts with "$BUSY_PEOPLE should do $THIS" fails. Any plan that starts with "I could do $THIS" at least has a chance. This is not limited to open source, by the way... -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 23:09 ` Rob Landley @ 2005-12-06 0:54 ` Steven Rostedt 2005-12-06 1:10 ` Florian Weimer 2005-12-06 3:22 ` Rob Landley 0 siblings, 2 replies; 639+ messages in thread From: Steven Rostedt @ 2005-12-06 0:54 UTC (permalink / raw) To: Rob Landley Cc: Florian Weimer, Lee Revell, Mark Lord, Adrian Bunk, David Ranson, linux-kernel On Mon, 5 Dec 2005, Rob Landley wrote: > On Monday 05 December 2005 15:21, Steven Rostedt wrote: > > Perhaps, we could start out having Greg and Chris just concentrate on > > every fifth branch instead of every one, and that way the stability will > > last much longer. > > Ah, belling the cat. :) > > Hint: Any plan in a volunteer community that starts with "$BUSY_PEOPLE should > do $THIS" fails. Any plan that starts with "I could do $THIS" at least has a > chance. Actually, they are already maintaining 2.6.x.y, (x => 11, 12, ...) I was trying to get them to only maintain 2.6.x.y (x => 11, 12, 13, 14, 20, 25, ...) So maybe it would actually be easier. But I'm sure they wouldn't be fooled, since the longer you maintain a fork, the harder it becomes. I was just making a suggestion, so that if someone else thought it was a good idea, they could do it. I personally don't need such a beast, since I would just stay with the latest 2.6.x anyway. Since I have that luxury. > > This is not limited to open source, by the way... Yep, I know that. -- Steve ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 0:54 ` Steven Rostedt @ 2005-12-06 1:10 ` Florian Weimer 2005-12-06 1:26 ` Steven Rostedt 2005-12-06 3:22 ` Rob Landley 1 sibling, 1 reply; 639+ messages in thread From: Florian Weimer @ 2005-12-06 1:10 UTC (permalink / raw) To: Steven Rostedt Cc: Rob Landley, Lee Revell, Mark Lord, Adrian Bunk, David Ranson, linux-kernel * Steven Rostedt: >> Hint: Any plan in a volunteer community that starts with "$BUSY_PEOPLE should >> do $THIS" fails. Any plan that starts with "I could do $THIS" at least has a >> chance. > > Actually, they are already maintaining 2.6.x.y, (x => 11, 12, ...) I think the 2.6.x.y series is no longer maintained once 2.6.x+1 has been released for some time (surely after 2.6.x+2). ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 1:10 ` Florian Weimer @ 2005-12-06 1:26 ` Steven Rostedt 2005-12-06 18:06 ` Horst von Brand 0 siblings, 1 reply; 639+ messages in thread From: Steven Rostedt @ 2005-12-06 1:26 UTC (permalink / raw) To: Florian Weimer Cc: Rob Landley, Lee Revell, Mark Lord, Adrian Bunk, David Ranson, linux-kernel On Tue, 6 Dec 2005, Florian Weimer wrote: > > > > Actually, they are already maintaining 2.6.x.y, (x => 11, 12, ...) > > I think the 2.6.x.y series is no longer maintained once 2.6.x+1 has > been released for some time (surely after 2.6.x+2). > The same can still go for this, but instead of stopping at 2.6.x+2 we could stop at 2.6.x+6 (or +5), and just not care about 2.6.x+[1-4]. But that would be strong enough for those that would like the stable branch to maintain it themselves. Currently it'l hard to pick a 2.6.x that you want to stay with since the 2.6.x.y is stopped right after 2.6.x+1 is out. But if not all 2.6.x has a .y, then that would focus more distrobutions or whatever to pick the same one to support. Oh well, I'm just spitting out a bunch of lip service here. It actually seems interesting to try, and if I actually had a need to do this, I would. But right now my focus is elsewhere. Cheers, -- Steve ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 1:26 ` Steven Rostedt @ 2005-12-06 18:06 ` Horst von Brand 0 siblings, 0 replies; 639+ messages in thread From: Horst von Brand @ 2005-12-06 18:06 UTC (permalink / raw) To: Steven Rostedt Cc: Florian Weimer, Rob Landley, Lee Revell, Mark Lord, Adrian Bunk, David Ranson, linux-kernel [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain, Size: 3767 bytes --] Steven Rostedt <rostedt@goodmis.org> wrote: > On Tue, 6 Dec 2005, Florian Weimer wrote: > > > Actually, they are already maintaining 2.6.x.y, (x => 11, 12, ...) > > I think the 2.6.x.y series is no longer maintained once 2.6.x+1 has > > been released for some time (surely after 2.6.x+2). > The same can still go for this, but instead of stopping at 2.6.x+2 we could > stop at 2.6.x+6 (or +5), and just not care about 2.6.x+[1-4]. But that > would be strong enough for those that would like the stable branch to > maintain it themselves. Currently it'l hard to pick a 2.6.x that you want > to stay with since the 2.6.x.y is stopped right after 2.6.x+1 is out. But > if not all 2.6.x has a .y, then that would focus more distrobutions or > whatever to pick the same one to support. OK, let's step back and... - People work on Linux (or whatever other stuff) because it is /fun/ and /exciting/. - People who don't actively work on a piece of code won't know it intimately, so they'll make mistakes when looking for bugs/fixes - There is little excitement in just fixing bugs in frozen code, developers will just migrate elsewhere if there is no fun here - "Experimental versions" are only run by masochists and bored people who need fireworks now and then to know they are still alive. End users don't even consider them, when the software is stable enough for the crazier ones to unleash on the world as "stable" is when the bugs surface (Remember the disaster of the early 2.4s? Ever heard of "Let's wait for X.<even>.10 or so, by then it will be stable enough for everyday use"?). - End users /don't/ test "prereleases", they deem them too risky... so the "releases" usually ship with lethal bugs for some people. Decreeing that each 5th (or whatever) release will be "golden" will just get people to skip all the others, resulting in /much more/ serious bugs in the end - Backporting new features into a different setting is almost as hard, or perhaps much harder, than developing said features in the first place. Backpòrting bug fixes is a thankless job. - Distributions /do/ have the infrastructure in place to collect bug reports and correlate them with hardware and software configurations, moreover they work with /one/ (or at most a few) kernel configuration, not with the almost random assortment of kernel configurations you'll find with self-built ones. They also have the (paid) manpower to extract conclusions from the above data. - If all distributions work from approximately the same base, much duplicate/wasted work (yes, backporting is mostly wasted effort IMHO) is saved. - Tools for kernel work are /much/ better now, it is reasonable to maintain patches out-of-vanilla and keep the base syncronized with the standard kernel source. Lots of people do so now, when it previously was a full-time job for the incredibly productive gnome community known collectively as "Alan Cox" to do so. There is thus much less need for "odd" series in which to integrate new stuff. All of the above led to the current kernel development model. Users might not be too happy about it, but the key developers are. And the important people, the ones you /have/ to keep happy in the OSS development model, aren't the users. Besides, everybody moaning about the new development model want something impossible: Fast development, timely integration of support for newest hardware, while bug free all the time. Won't ever happen. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 0:54 ` Steven Rostedt 2005-12-06 1:10 ` Florian Weimer @ 2005-12-06 3:22 ` Rob Landley 1 sibling, 0 replies; 639+ messages in thread From: Rob Landley @ 2005-12-06 3:22 UTC (permalink / raw) To: Steven Rostedt Cc: Florian Weimer, Lee Revell, Mark Lord, Adrian Bunk, David Ranson, linux-kernel On Monday 05 December 2005 18:54, Steven Rostedt wrote: > > Hint: Any plan in a volunteer community that starts with "$BUSY_PEOPLE > > should do $THIS" fails. Any plan that starts with "I could do $THIS" at > > least has a chance. > > Actually, they are already maintaining 2.6.x.y, (x => 11, 12, ...) I was > trying to get them to only maintain 2.6.x.y (x => 11, 12, 13, 14, 20, 25, > ...) That's still "trying to get them" rather than "I could"... > So maybe it would actually be easier. But I'm sure they wouldn't be > fooled, since the longer you maintain a fork, the harder it becomes. And the number exponentially increases (2.6.x+1.y, 2.6.x+2.y, all at the same time...) No, I pestered them a while back about possibly doing a 2.6.x.y+1 to flush their patch queue before doing a 2.6.x+1.1, and they seem more receptive to the idea now. But then backporting 2.6.x+1.y to 2.6.x becomes your job... Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 21:21 ` Steven Rostedt 2005-12-05 23:09 ` Rob Landley @ 2005-12-06 1:06 ` Florian Weimer 1 sibling, 0 replies; 639+ messages in thread From: Florian Weimer @ 2005-12-06 1:06 UTC (permalink / raw) To: Steven Rostedt Cc: Lee Revell, Mark Lord, Rob Landley, Adrian Bunk, David Ranson, linux-kernel * Steven Rostedt: >> Would this alone change much? I think what we really want is that our >> favorite branch (whatever it is) gets critical fixes forever (well, >> maybe one or two years, but this is forever). This is a bit >> unrealistic because everyone has a slightly different branchpoint. >> Releasing more often doesn't change that, really. > > Maybe that is what is needed. A branch that all can use. There isn't a single one. Even for Debian, it was a hard struggle to get sown to just two (or three?). Now try that across distributions, or for people who own choosy hardware. (I once had to deal with a box which didn't like anything else except 2.6.0-test9. I believe it's still running this version, maybe slightly patched.) > Have every 5 or so 2.6.x become a "stable" branch. Where > distributions and users can work together on keeping it stable. The > rules to modifying such a branch would pretty much stay with what it > already takes to modify the current 2.6.x.y branch. If you want a > feature, you must either take the latest "unstable" 2.6.x branch or > wait for the next "stable" 2.6.x branch to merge. In essence, this is just a slower version of the current model. It won't change that much, unless the speed of the development cycle (and its phase) matches your needs, which is unlikely. Security bugs would still be discovered at about the same rate. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 20:52 ` Florian Weimer 2005-12-05 21:21 ` Steven Rostedt @ 2005-12-06 11:09 ` Matthias Andree 1 sibling, 0 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-06 11:09 UTC (permalink / raw) To: linux-kernel On Mon, 05 Dec 2005, Florian Weimer wrote: > * Matthias Andree: > > > Basically, no-one should have permission to touch any core parts, except > > for fixes, until 2.7. Yes, that means going back to older models. Yes, > > that means that the discussions will start all over. And yes, that means > > that the cool stuff has to wait. Solution: release more often. > > Would this alone change much? I think what we really want is that our > favorite branch (whatever it is) gets critical fixes forever (well, > maybe one or two years, but this is forever). This is a bit > unrealistic because everyone has a slightly different branchpoint. > Releasing more often doesn't change that, really. Releasing minor releases more often and enforcing "don't touch unless you must" policy would create such synchronization point and a branch where everyone could safely hop between releases. > In the security area, I think there is enough experience out there to > collect data which would help each local branch maintainer to install > the relevant fixes. But for general development, this seems to be > infeasible, unless you focus your software architecture on this > purpose (which is probably a terrible idea to do for kernel > development). I don't think focusing the kernel on code quality and security is wrong though. The actual problem we've seen from postings by Lee and others is that the burden of test is shifted to the distros and their QA teams so that effectively everyone is free to break things at will, downstream QA will fix it anyways. This however doesn't work, and the problem here is the propagation delay. At the time the end user sees a problem with his kernel, the upstream has already abandoned the 2.6.X.Y stable branch the distro was based on, and upstream is at 2.6.X+2 or even farther ahead. What is actually needed is to enclose this end user system in the tests run before further changes in the same area. And as udev etc. need to change, a simple test if the current kernel works means updating some user space packages, hotplug, modutils (OK this was 2.5), udev, whatever, and what's even worse, if that doesn't help or breaks other things. Going back may not even work through the packaging system because the old kernel version may not have a "udev <= N" dependency either... So before this can work, the actual package maintenance systems such as yum, yast, dpkg, apt and rpm will need to support what Emacs Lisp calls excursions. It means, snapshot the packages and revert to the same set later. Even if this were solved and excursions were cheap, it would still not solve the time skew bug report and upstream fixes... -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 16:28 ` Lee Revell 2005-12-05 16:44 ` Matthias Andree @ 2005-12-05 17:58 ` Rob Landley 2005-12-06 18:51 ` Bill Davidsen 2005-12-05 21:22 ` Bill Davidsen 2005-12-06 14:59 ` Bill Davidsen 3 siblings, 1 reply; 639+ messages in thread From: Rob Landley @ 2005-12-05 17:58 UTC (permalink / raw) To: Lee Revell Cc: Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Monday 05 December 2005 10:28, Lee Revell wrote: > > > > Things like this are all too regular an occurance. > > The distro should have solved this problem by making sure that the > kernel upgrade depends on a new wpa_supplicant package. Don't they > bother to test this stuff before they ship it?!? I've broken stuff by upgrading glibc, upgrading X, upgrading KDE... Upgrading the kernel is safer? (Anybody remember 2.4.11-dontuse?) Yay, modular component-based design. We have standard interfaces so that stuff mostly works when you swap out different versions (or entire different components). This is cool. But if such interfaces were actually sufficient to specify all the functionality you actually want to use, why would you ever need to upgrade a component implementing that interface to a new version? The real problem people are seeing is that the rate of change has increased. Linus used to be a hell of a bottleneck, and this stopped being the case when source control systems took over a lot of the patch tracking, ordering, batching, and integration burden. Automating the patch flow allowed entire subsystems to be effectively delegated (and thus the "lieutenants" layer formed and was formalized), and each of _them_ is now doing as much work as Linus used to do. And _that_ is why there is no longer any point in -devel forks, because Linus is now fielding as many patches in a month as he used to in a year. That means in every 3 months the Linux kernel undergoes as much development (in terms of patches merged and lines of code changed) as an entire -devel series used to do. (Somebody confirm the numbers, these are approximations.) There's no point in launching a fork that's only expected to last three months. Hence no -devel fork. Also, forks are cheaper now. The new source control tools (not just git but quilt and ketchup and so on) allow multiple parallel trees to be trivially integrated. It used to take somebody like Alan Cox all his spare time to maintain a tree and merge with linus, and back then the -ac tree was very special. Now Con Colivas can maintain a tree in his spare time with a day job as an anesthesiologist, and this is _normal_. There are dozens of trees out there feeding into each other, and anybody who wants to can grab the relevant subsystem tree and try it out to make sure that the issue they care about is fixed, and be assured that it'll all flow in to Linus's tree. What's special about Linus's tree is that it's the point to the wedge. This is the farthest we've advanced, this is your best bet. There's always something wrong with any piece of software, but half the complaints have always been that something is fixed but not merged yet. (Orinoco scanning, ISDN, ALSA, examples are legion.) That's getting way better. Now the _new_ class of complaints is that the IPW2200 driver that first got merged was too old. (I noticed this because that's the wireless card in my laptop.) Stop and think about that for a bit. People were used to IPW2200 not being there at all, so they could easily add an external patch. Then 2.6.14 grew partial IPW2200 support, but with an older driver, and people were mad because the external patch they had to add support only applied when the driver wasn't there at all, and it didn't apply over the older version. They were mad that _insufficient_ support showed up. This is being fixed in 2.6.15. It didn't last long. The new model is that if the kernel has half what you need, you need to come up with an incremental patch to get it the rest of the way, and submit that. And the up-side is that it'll go in pretty fast now. Yes, the kernel is changing rapidly enough that external patches probably need to be fixed up with every new version. (And if you're using the nvidia driver, this sucks rocks. You were warned.) The other thing people are complaining about is the deprecation schedule. Notice is posted prominently up to a YEAR before a feature gets yanked. Apparently, they want to upgrade to the new version when it comes out, but don't want to read the instructions for this version (X went away), or the warnings about upcoming versions... Possibly if we had a CHANGES file at the root level of the source mentioning A) What's new in this version. B) What's slated for next version (DEPRECATION COMING). C) What was new in the last version, in case you missed it. A combination of Documentation/feature-removal-schedule.txt and http://wiki.kernelnewbies.org/LinuxChanges, with emphasis on userspace tools known to be impacted. > Lee Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 17:58 ` Rob Landley @ 2005-12-06 18:51 ` Bill Davidsen 2005-12-07 15:48 ` Arjan van de Ven 2005-12-07 18:14 ` Rob Landley 0 siblings, 2 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-06 18:51 UTC (permalink / raw) To: Rob Landley Cc: Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree Rob Landley wrote: > On Monday 05 December 2005 10:28, Lee Revell wrote: > >>>Things like this are all too regular an occurance. >> >>The distro should have solved this problem by making sure that the >>kernel upgrade depends on a new wpa_supplicant package. Don't they >>bother to test this stuff before they ship it?!? > > > I've broken stuff by upgrading glibc, upgrading X, upgrading KDE... > > Upgrading the kernel is safer? (Anybody remember 2.4.11-dontuse?) > > Yay, modular component-based design. We have standard interfaces so that > stuff mostly works when you swap out different versions (or entire different > components). This is cool. > > But if such interfaces were actually sufficient to specify all the > functionality you actually want to use, why would you ever need to upgrade a > component implementing that interface to a new version? > > The real problem people are seeing is that the rate of change has increased. > Linus used to be a hell of a bottleneck, and this stopped being the case when > source control systems took over a lot of the patch tracking, ordering, > batching, and integration burden. > > Automating the patch flow allowed entire subsystems to be effectively > delegated (and thus the "lieutenants" layer formed and was formalized), and > each of _them_ is now doing as much work as Linus used to do. > > And _that_ is why there is no longer any point in -devel forks, because Linus > is now fielding as many patches in a month as he used to in a year. That > means in every 3 months the Linux kernel undergoes as much development (in > terms of patches merged and lines of code changed) as an entire -devel series > used to do. (Somebody confirm the numbers, these are approximations.) > > There's no point in launching a fork that's only expected to last three > months. Hence no -devel fork. Just so we're all on the same page, I think there are two sets of unhappy people here... one is the group who want new stuff fast and stable. For the most part that's not me, although I was in the "if you're going to add ipw2200 support, why not something that works?" group. But new stuff is going in faster than most people can assimilate it if they have a real job, so I don't see too much problem there. The other group is the people who use and depend on some feature, be it cryptoloop, 8k stacks, ndiswrapper, ipchains, whatever... which is scheduled for extinction. That's a departure from the way 2.{0,2,4} were done, where adds happened regularly, but features were only deleted in development trees. Deleting features leaves anyone who can't keep their own tree without security fixes. I see that as bed, and a far more important departure from the old model than the speed of new adds. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 18:51 ` Bill Davidsen @ 2005-12-07 15:48 ` Arjan van de Ven 2005-12-07 18:40 ` Horst von Brand 2005-12-07 18:14 ` Rob Landley 1 sibling, 1 reply; 639+ messages in thread From: Arjan van de Ven @ 2005-12-07 15:48 UTC (permalink / raw) To: Bill Davidsen Cc: Rob Landley, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree > The other group is the people who use and depend on some feature, be it > cryptoloop, 8k stacks, ndiswrapper, ipchains, whatever... which is > scheduled for extinction. these are actually 2 groups 1) people who depend on an in-kernel features 2) people who depend on out of kernel / binary modules treating them as one is not correct or fair to this thread. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-07 15:48 ` Arjan van de Ven @ 2005-12-07 18:40 ` Horst von Brand 0 siblings, 0 replies; 639+ messages in thread From: Horst von Brand @ 2005-12-07 18:40 UTC (permalink / raw) To: Arjan van de Ven Cc: Bill Davidsen, Rob Landley, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree Arjan van de Ven <arjan@infradead.org> wrote: > > The other group is the people who use and depend on some feature, be it > > cryptoloop, 8k stacks, ndiswrapper, ipchains, whatever... which is > > scheduled for extinction. Come on, most of those were scheduled for deletion a /long/ time ago. > these are actually 2 groups > > 1) people who depend on an in-kernel features > > 2) people who depend on out of kernel / binary modules > > treating them as one is not correct or fair to this thread. Right. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 18:51 ` Bill Davidsen 2005-12-07 15:48 ` Arjan van de Ven @ 2005-12-07 18:14 ` Rob Landley 2005-12-10 8:35 ` ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] Pavel Machek 2005-12-10 13:41 ` RFC: Starting a stable kernel series off the 2.6 kernel Bill Davidsen 1 sibling, 2 replies; 639+ messages in thread From: Rob Landley @ 2005-12-07 18:14 UTC (permalink / raw) To: Bill Davidsen Cc: Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Tuesday 06 December 2005 12:51, Bill Davidsen wrote: > Just so we're all on the same page, I think there are two sets of > unhappy people here... one is the group who want new stuff fast and > stable. For the most part that's not me, although I was in the "if > you're going to add ipw2200 support, why not something that works?" > group. But new stuff is going in faster than most people can assimilate > it if they have a real job, so I don't see too much problem there. My laptop has an ipw2200 but I can't get it to work in any kernel I built because the kernels I build aren't modular. I hope to be able to work around this someday with a clever enough initramfs (if necessary, moving the initramfs initialization earlier in the boot sequence), but it hasn't made it far enough up my todo list yet. So whether or not the driver actually works if I could get it initialized is, for me, a moot point. > The other group is the people who use and depend on some feature, be it > cryptoloop, 8k stacks, ndiswrapper, ipchains, whatever... which is Ndiswrapper isn't a kernel feature. Don't confuse the shark with the remoras. The only complaint about 8k stacks I've heard is the ndiswrapper people, and 8k isn't actually sufficient for them anyway (_their_ ad-hoc spec guarantees 12k), so they should have been swapping to their own stack all along. They should probably even statically allocate the sucker at boot time and serialize all drivers using it with a semaphore, because I _really_ doubt it's ever been preempt-safe. Isn't ipchains obsolete since 2.2? it was already deprecated back in 2.4. There has been quite adequate warning on that one, thanks very much. I'm under the impression the problem with cryptoloop is bad cryptography: http://lwn.net/Articles/67216/ Anybody actually using cryptography with an expoitable weakness needs all the wake-up calls they can get. This is _not_ a case where you want to support old broken crap, that defeats the whole purpose of using cryptography in the first place. Especially the cryptoloop removal was an intentional decision that the kernel developers made. People raised their objections at the time, and these were taken into consideration when making the decision: http://kerneltrap.org/node/2433 Re-raising the same objections over and over again when they've already been aired, considered, and rejections is called "whining". > scheduled for extinction. That's a departure from the way 2.{0,2,4} were > done, where adds happened regularly, but features were only deleted in > development trees. If features were really were deleted in development trees, devfs and ipchains never would have made it through 2.4. So you're talking about an idealized version fo the past that doesn't match what people actually did back then. > Deleting features leaves anyone who can't keep their > own tree without security fixes. Security fixes are a separate issue. I asked for one more security fix to flush the pending fixes queue a while ago: http://seclists.org/lists/linux-kernel/2005/Nov/4187.html On Saturday, stable series co-maintainer Chris Wright decided it might be worth a try: http://seclists.org/lists/linux-kernel/2005/Dec/0740.html > About the only thing I think is helpful in this case is perhaps > one extra -stable cycle on the last branch when newest branch is released > (basically flush the queue). That much I'm willing to do in -stable. To which I replied (and again I quote), "Yay, rah, cool!". This gives you a trail of breadcrumbs to follow. There are a series of incremental patches (which may have to be fixed up to apply) that address the known security-specific issues. Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] 2005-12-07 18:14 ` Rob Landley @ 2005-12-10 8:35 ` Pavel Machek 2005-12-11 5:30 ` Rob Landley 2005-12-10 13:41 ` RFC: Starting a stable kernel series off the 2.6 kernel Bill Davidsen 1 sibling, 1 reply; 639+ messages in thread From: Pavel Machek @ 2005-12-10 8:35 UTC (permalink / raw) To: Rob Landley Cc: Bill Davidsen, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Wed 07-12-05 12:14:25, Rob Landley wrote: > On Tuesday 06 December 2005 12:51, Bill Davidsen wrote: > > Just so we're all on the same page, I think there are two sets of > > unhappy people here... one is the group who want new stuff fast and > > stable. For the most part that's not me, although I was in the "if > > you're going to add ipw2200 support, why not something that works?" > > group. But new stuff is going in faster than most people can assimilate > > it if they have a real job, so I don't see too much problem there. > > My laptop has an ipw2200 but I can't get it to work in any kernel I built > because the kernels I build aren't modular. I hope to be able to work around > this someday with a clever enough initramfs (if necessary, moving the > initramfs initialization earlier in the boot sequence), but it hasn't made it > far enough up my todo list yet. Well, building modular kernel for a test is not *that* much work. Anyway, if you are going to fix it, fix it properly (by delayed firmware loading) -- initrd hacks are good for you but unusable for anyone else. Pavel -- Thanks, Sharp! ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] 2005-12-10 8:35 ` ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] Pavel Machek @ 2005-12-11 5:30 ` Rob Landley 2005-12-11 8:37 ` Pavel Machek ` (2 more replies) 0 siblings, 3 replies; 639+ messages in thread From: Rob Landley @ 2005-12-11 5:30 UTC (permalink / raw) To: Pavel Machek Cc: Bill Davidsen, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Saturday 10 December 2005 02:35, Pavel Machek wrote: > On Wed 07-12-05 12:14:25, Rob Landley wrote: > > On Tuesday 06 December 2005 12:51, Bill Davidsen wrote: > > > Just so we're all on the same page, I think there are two sets of > > > unhappy people here... one is the group who want new stuff fast and > > > stable. For the most part that's not me, although I was in the "if > > > you're going to add ipw2200 support, why not something that works?" > > > group. But new stuff is going in faster than most people can assimilate > > > it if they have a real job, so I don't see too much problem there. > > > > My laptop has an ipw2200 but I can't get it to work in any kernel I built > > because the kernels I build aren't modular. I hope to be able to work > > around this someday with a clever enough initramfs (if necessary, moving > > the initramfs initialization earlier in the boot sequence), but it hasn't > > made it far enough up my todo list yet. > > Well, building modular kernel for a test is not *that* much work. > Anyway, if you are going to fix it, fix it properly (by > delayed firmware loading) -- initrd hacks are good for you > but unusable for anyone else. I don't see why that's any less usable than using udev from initramfs to find your root partition. There is an interesting licensing issue, creating a linux kernel image that contains an initramfs that contains binary only firmware. I can happily generate one here and not care, but does distributing such a kernel violate the GPL? It's probable that the "early userspace" mechanism is a clearly defined API. We documented the heck out of the format, it cpio was already a standard anyway. The binaries that are in there call the normal userspace API (syscall/ioctl/procfs etc) that is an established strong barrier to being a derived work. So it's possible that putting those binaries in initramfs counts as "mere aggregation". (If you had the kernel and the firmware in an ISO image, that's the same as being different files on the same CD, and definitely mere aggregation. As separate files in the same tarball, it's closer but functionally equivalent and probably still just aggregation. Probably. Different elf sections in the same binary is one more step, but depending on the intent the analogy could still hold...) I can see distributors shying the heck away from it anyway, and wanting to keep things in _seperate_files_. And I can entirely understand that. This means if initramfs is going to contain firmware, the option of having it be a separate file like initrd for legal reasons is a darn good idea. Query: if you tell lilo or grub that it has an initrd but feed it a gzipped cpio image, will the kernel figure everything out and initialize initramfs from that appropriately? > Pavel Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] 2005-12-11 5:30 ` Rob Landley @ 2005-12-11 8:37 ` Pavel Machek 2005-12-11 9:12 ` Rob Landley 2005-12-11 16:26 ` Horst von Brand 2005-12-12 17:34 ` Ben Slusky 2 siblings, 1 reply; 639+ messages in thread From: Pavel Machek @ 2005-12-11 8:37 UTC (permalink / raw) To: Rob Landley Cc: Bill Davidsen, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On So 10-12-05 23:30:30, Rob Landley wrote: > On Saturday 10 December 2005 02:35, Pavel Machek wrote: > > On Wed 07-12-05 12:14:25, Rob Landley wrote: > > > On Tuesday 06 December 2005 12:51, Bill Davidsen wrote: > > > > Just so we're all on the same page, I think there are two sets of > > > > unhappy people here... one is the group who want new stuff fast and > > > > stable. For the most part that's not me, although I was in the "if > > > > you're going to add ipw2200 support, why not something that works?" > > > > group. But new stuff is going in faster than most people can assimilate > > > > it if they have a real job, so I don't see too much problem there. > > > > > > My laptop has an ipw2200 but I can't get it to work in any kernel I built > > > because the kernels I build aren't modular. I hope to be able to work > > > around this someday with a clever enough initramfs (if necessary, moving > > > the initramfs initialization earlier in the boot sequence), but it hasn't > > > made it far enough up my todo list yet. > > > > Well, building modular kernel for a test is not *that* much work. > > Anyway, if you are going to fix it, fix it properly (by > > delayed firmware loading) -- initrd hacks are good for you > > but unusable for anyone else. > > I don't see why that's any less usable than using udev from initramfs to find > your root partition. Why use udev from initramfs? Just teach ipw2200 to load firmware late. Don't load firmware when ipw2200 is initialized, load it only when someone attempts to talk to your ipw2200. At that time, you should have userland already. Pavel -- Thanks, Sharp! ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] 2005-12-11 8:37 ` Pavel Machek @ 2005-12-11 9:12 ` Rob Landley 2005-12-12 11:49 ` Pavel Machek 0 siblings, 1 reply; 639+ messages in thread From: Rob Landley @ 2005-12-11 9:12 UTC (permalink / raw) To: Pavel Machek Cc: Bill Davidsen, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Sunday 11 December 2005 02:37, Pavel Machek wrote: > On So 10-12-05 23:30:30, Rob Landley wrote: > > On Saturday 10 December 2005 02:35, Pavel Machek wrote: > > > On Wed 07-12-05 12:14:25, Rob Landley wrote: > > > > On Tuesday 06 December 2005 12:51, Bill Davidsen wrote: > > > > > Just so we're all on the same page, I think there are two sets of > > > > > unhappy people here... one is the group who want new stuff fast and > > > > > stable. For the most part that's not me, although I was in the "if > > > > > you're going to add ipw2200 support, why not something that works?" > > > > > group. But new stuff is going in faster than most people can > > > > > assimilate it if they have a real job, so I don't see too much > > > > > problem there. > > > > > > > > My laptop has an ipw2200 but I can't get it to work in any kernel I > > > > built because the kernels I build aren't modular. I hope to be able > > > > to work around this someday with a clever enough initramfs (if > > > > necessary, moving the initramfs initialization earlier in the boot > > > > sequence), but it hasn't made it far enough up my todo list yet. > > > > > > Well, building modular kernel for a test is not *that* much work. > > > Anyway, if you are going to fix it, fix it properly (by > > > delayed firmware loading) -- initrd hacks are good for you > > > but unusable for anyone else. > > > > I don't see why that's any less usable than using udev from initramfs to > > find your root partition. > > Why use udev from initramfs? I don't, but I do use a script that mknods the real root's node based on running "find" against /sys to locacate the appropriate device name and then finding the major/minor numbers there. This has nothing whatsoever to do with ipw2200. It just means I'm not using the in-kernel root-finder code. > Just teach ipw2200 to load firmware late. That's now how I'd fix this. If you want to fix it this way, be my guest. > Don't load firmware when ipw2200 is initialized, load it only > when someone attempts to talk to your ipw2200. At that time, you > should have userland already. Or I could move initramfs extraction earlier in the boot sequence and never have to modify any _other_ drivers that want firmware in order to be able to make them work too, rather than playing whack-a-mole teaching drivers I don't care about how to hold off on wanting firmware. > Pavel Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] 2005-12-11 9:12 ` Rob Landley @ 2005-12-12 11:49 ` Pavel Machek 2005-12-14 12:26 ` Rob Landley 0 siblings, 1 reply; 639+ messages in thread From: Pavel Machek @ 2005-12-12 11:49 UTC (permalink / raw) To: Rob Landley Cc: Bill Davidsen, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree Hi! > > Why use udev from initramfs? > > I don't, but I do use a script that mknods the real root's node based on > running "find" against /sys to locacate the appropriate device name and then > finding the major/minor numbers there. > > This has nothing whatsoever to do with ipw2200. It just means I'm not using > the in-kernel root-finder code. > > > Just teach ipw2200 to load firmware late. > > That's now how I'd fix this. If you want to fix it this way, be my guest. > > > Don't load firmware when ipw2200 is initialized, load it only > > when someone attempts to talk to your ipw2200. At that time, you > > should have userland already. > > Or I could move initramfs extraction earlier in the boot sequence and never > have to modify any _other_ drivers that want firmware in order to be able to > make them work too, rather than playing whack-a-mole teaching drivers I don't > care about how to hold off on wanting firmware. Except that whack-a-mole is a right thing to do here, and that initramfs movement is unlikely to make it into mainline. Pavel -- Thanks, Sharp! ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] 2005-12-12 11:49 ` Pavel Machek @ 2005-12-14 12:26 ` Rob Landley 2004-12-14 16:01 ` Pavel Machek 0 siblings, 1 reply; 639+ messages in thread From: Rob Landley @ 2005-12-14 12:26 UTC (permalink / raw) To: Pavel Machek Cc: Bill Davidsen, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Monday 12 December 2005 05:49, Pavel Machek wrote: > > Or I could move initramfs extraction earlier in the boot sequence and > > never have to modify any _other_ drivers that want firmware in order to > > be able to make them work too, rather than playing whack-a-mole teaching > > drivers I don't care about how to hold off on wanting firmware. > > Except that whack-a-mole is a right thing to do here, and that > initramfs movement is unlikely to make it into mainline. > Pavel Let me guess: for licensing reasons? The option to keep initramfs in a separate file (like initrd) should, in theory, make that a moot point... Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] 2005-12-14 12:26 ` Rob Landley @ 2004-12-14 16:01 ` Pavel Machek 0 siblings, 0 replies; 639+ messages in thread From: Pavel Machek @ 2004-12-14 16:01 UTC (permalink / raw) To: Rob Landley Cc: Bill Davidsen, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Wed 14-12-05 06:26:08, Rob Landley wrote: > On Monday 12 December 2005 05:49, Pavel Machek wrote: > > > Or I could move initramfs extraction earlier in the boot sequence and > > > never have to modify any _other_ drivers that want firmware in order to > > > be able to make them work too, rather than playing whack-a-mole teaching > > > drivers I don't care about how to hold off on wanting firmware. > > > > Except that whack-a-mole is a right thing to do here, and that > > initramfs movement is unlikely to make it into mainline. > > Pavel > > Let me guess: for licensing reasons? Wrong. "Fix the driver" is easier to get into the kernel than "change the boot sequence". -- Thanks, Sharp! ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] 2005-12-11 5:30 ` Rob Landley 2005-12-11 8:37 ` Pavel Machek @ 2005-12-11 16:26 ` Horst von Brand 2005-12-12 17:34 ` Ben Slusky 2 siblings, 0 replies; 639+ messages in thread From: Horst von Brand @ 2005-12-11 16:26 UTC (permalink / raw) To: Rob Landley Cc: Pavel Machek, Bill Davidsen, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree Rob Landley <rob@landley.net> wrote: [...] > There is an interesting licensing issue, creating a linux kernel image that > contains an initramfs that contains binary only firmware. I can happily > generate one here and not care, but does distributing such a kernel violate > the GPL? Some distributions (e.g. INSERT) ship a CD image with ipw2x00 firmware. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] 2005-12-11 5:30 ` Rob Landley 2005-12-11 8:37 ` Pavel Machek 2005-12-11 16:26 ` Horst von Brand @ 2005-12-12 17:34 ` Ben Slusky 2005-12-12 20:02 ` Rob Landley 2 siblings, 1 reply; 639+ messages in thread From: Ben Slusky @ 2005-12-12 17:34 UTC (permalink / raw) To: Rob Landley Cc: Pavel Machek, Bill Davidsen, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Sat, 10 Dec 2005 23:30:30 -0600, Rob Landley wrote: > Query: if you tell lilo or grub that it has an initrd but feed it a gzipped > cpio image, will the kernel figure everything out and initialize initramfs > from that appropriately? Yes, I've been booting my laptop this way (using GRUB) since 2.6.7 or so. -- Ben Slusky | It was only after their population sluskyb@paranoiacs.org | of 50 mysteriously shrank to eight sluskyb@stwing.org | that the other seven dwarfs began PGP keyID ADA44B3B | to suspect Hungry. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] 2005-12-12 17:34 ` Ben Slusky @ 2005-12-12 20:02 ` Rob Landley [not found] ` <439DE10E.4080901@tmr.com> 0 siblings, 1 reply; 639+ messages in thread From: Rob Landley @ 2005-12-12 20:02 UTC (permalink / raw) To: Ben Slusky Cc: Pavel Machek, Bill Davidsen, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Monday 12 December 2005 11:34, Ben Slusky wrote: > On Sat, 10 Dec 2005 23:30:30 -0600, Rob Landley wrote: > > Query: if you tell lilo or grub that it has an initrd but feed it a > > gzipped cpio image, will the kernel figure everything out and initialize > > initramfs from that appropriately? > > Yes, I've been booting my laptop this way (using GRUB) since 2.6.7 or so. Sigh, gotta update the docs again... :) Do you need to compile initrd support in for this to work, or just tell grub/lilo to do its' thing? (I can answer this one myself when I get around to setting up another test environment...) Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
[parent not found: <439DE10E.4080901@tmr.com>]
* Re: ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] [not found] ` <439DE10E.4080901@tmr.com> @ 2005-12-12 21:52 ` Ben Slusky 2005-12-15 2:38 ` ipw2200 Miles Bader 0 siblings, 1 reply; 639+ messages in thread From: Ben Slusky @ 2005-12-12 21:52 UTC (permalink / raw) To: Bill Davidsen Cc: Rob Landley, Pavel Machek, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Mon, 12 Dec 2005 15:43:58 -0500, Bill Davidsen wrote: > Rob Landley wrote: > I haven't tried that, since I use initrd files and have the kernel > support, but I confess I don't see the cpio as being easier to create. > You presumably still want to include the modules from the fresh built > kernel, so creating a new cpio file would seem needed for most people. cpio files are somewhat easier to create in that they can created by an unprivileged user. Most of the steps in making an initrd can only be done by root. -- Ben Slusky | You must be smarter than sluskyb@paranoiacs.org | <= this stick to ride the sluskyb@stwing.org | internet. PGP keyID ADA44B3B ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: ipw2200 2005-12-12 21:52 ` Ben Slusky @ 2005-12-15 2:38 ` Miles Bader 0 siblings, 0 replies; 639+ messages in thread From: Miles Bader @ 2005-12-15 2:38 UTC (permalink / raw) To: Bill Davidsen Cc: Rob Landley, Pavel Machek, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree Ben Slusky <sluskyb@paranoiacs.org> writes: >> I confess I don't see the cpio as being easier to create. You >> presumably still want to include the modules from the fresh built >> kernel, so creating a new cpio file would seem needed for most >> people. > > cpio files are somewhat easier to create in that they can created by > an unprivileged user. Most of the steps in making an initrd can only > be done by root. Initrds are also annoying because you have to guess/calculate a "disk" size big enough to hold all the contents and inevitably waste some space providing a margin for error. Initrd seems at best a kind of kludge anyway; initramfs is just all-around a cleaner concept. -Miles -- `To alcohol! The cause of, and solution to, all of life's problems' --Homer J. Simpson ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-07 18:14 ` Rob Landley 2005-12-10 8:35 ` ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] Pavel Machek @ 2005-12-10 13:41 ` Bill Davidsen 2005-12-10 17:05 ` Douglas McNaught 2005-12-11 5:33 ` Rob Landley 1 sibling, 2 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-10 13:41 UTC (permalink / raw) To: Rob Landley Cc: Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree Rob Landley wrote: > >I'm under the impression the problem with cryptoloop is bad cryptography: >http://lwn.net/Articles/67216/ > >Anybody actually using cryptography with an expoitable weakness needs all the >wake-up calls they can get. This is _not_ a case where you want to support >old broken crap, that defeats the whole purpose of using cryptography in the >first place. > >Especially the cryptoloop removal was an intentional decision that the kernel >developers made. People raised their objections at the time, and these were >taken into consideration when making the decision: >http://kerneltrap.org/node/2433 > >Re-raising the same objections over and over again when they've already been >aired, considered, and rejections is called "whining". > Repeating the same information over and over until it sinks in is called "rote learning." The question is not if cryptoloop is perfect, every crypto seems to fail eventually, recently md5 was cracked, etc. But people have used cryptoloop now, and removing it from the kernel will lock them out of their own data, or prevent them from moving forward. There's no replacement for cryptoloop, so I can't just reconfigure X and still read my 147 DVDs full of business data, or access the current data on 34 laptops around the country. In most cases CL is not expected to protect against goverment agencies but rather stolen laptops in airports (yes the pros have added MacOS and Linux to their business model) or the occasional lost DVD in the mail. Removing CL is not a hell of a lot better morally than these viruses which encrypt your data and then hold it for ransom, with the price being doing without security fixes in future kernels. Given that CL has minimal (essentially no) maintenence cost, I wish the ivory tower developers could understand that real people have invested real money in it, and real data in the technology. Since there is no alternative solution offered, CL is far better than no crypto at all, and I wish there were a few more developers who had experience working in the real word. -- bill davidsen <davidsen@tmr.com> CTO TMR Associates, Inc Doing interesting things with small computers since 1979 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-10 13:41 ` RFC: Starting a stable kernel series off the 2.6 kernel Bill Davidsen @ 2005-12-10 17:05 ` Douglas McNaught 2005-12-11 5:52 ` Rob Landley 2005-12-12 3:25 ` Bill Davidsen 2005-12-11 5:33 ` Rob Landley 1 sibling, 2 replies; 639+ messages in thread From: Douglas McNaught @ 2005-12-10 17:05 UTC (permalink / raw) To: Bill Davidsen Cc: Rob Landley, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree Bill Davidsen <davidsen@tmr.com> writes: > Rob Landley wrote: > >> Re-raising the same objections over and over again when they've >> already been aired, considered, and rejections is called "whining". >> > Repeating the same information over and over until it sinks in is > called "rote learning." The question is not if cryptoloop is perfect, > every crypto seems to fail eventually, recently md5 was cracked, > etc. But people have used cryptoloop now, and removing it from the > kernel will lock them out of their own data, or prevent them from > moving forward. There's no replacement for cryptoloop, so I can't just > reconfigure X and still read my 147 DVDs full of business data, or > access the current data on 34 laptops around the country. Bill, I still don't think your complaints are justified. You're only "locked out of your own data" if you knowingly upgrade to a kernel that doesn't support cryptoloop. Nobody's forcing you to do that. The kernel developers owe *nothing* to J. Random User. They are either doing what they do for their own reasons (the "fun" of it), or being paid by an organization with specific objectives (even if, in Linus' case, the objective is just "make the best kernel possible, based on your judgement and that of people you trust"). That said, of course none of them want to break things unnecessarily. But they make technical decisions, with the goal of having the best kernel, that do sometimes have painful consequences. You're free, of course, to disagree with those decisions and maintain your own kernel. They don't owe you security fixes either. Sorry, but that's the way it is. We're all lucky that they take security very seriously and respond quickly to problems. > In most cases CL is not expected to protect against goverment agencies > but rather stolen laptops in airports (yes the pros have added MacOS > and Linux to their business model) or the occasional lost DVD in the > mail. Removing CL is not a hell of a lot better morally than these > viruses which encrypt your data and then hold it for ransom, with the > price being doing without security fixes in future kernels. That last sentence is crap. You're free to backport security fixes to cryptoloop-supporting kernels forever, or pay someone to do so. Or maintain a cryptoloop patch against current kernels, or pay someone to do so. Or write a converter for cryptoloop data to whatever's currently in the kernel, or pay someone to do so. > Given that CL has minimal (essentially no) maintenence cost, I wish > the ivory tower developers could understand that real people have > invested real money in it, and real data in the technology. Since > there is no alternative solution offered, CL is far better than no > crypto at all, and I wish there were a few more developers who had > experience working in the real word. If you include a crypto solution in the mainstream kernel, you're in some sense endorsing its security. If that solution has known weaknesses, I can understand wanting to either fix it or rip it out. Crypto is hard enough to get right as it is. Your "ivory tower" statement is really condescending. Linux is way past the stage where college students were the main contributors (if it ever was so after Linus graduated). and a great majority of developers now are paid to work on the kernel. There are probably very few of them that don't have at least a little sysadmin experience. If you've invested money and put important data in a system, and you haven't contracted with anyone to support that system, supply security fixes, and make sure it does what you want it to do, who's the fool? Basically, you're complaining about something you get *for free* that represents millions of hours of work, because it doesn't work quite the way you want it to, when you have perfect freedom to make it meet your needs by putting in your own time, effort and/or money. -Doug ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-10 17:05 ` Douglas McNaught @ 2005-12-11 5:52 ` Rob Landley 2005-12-12 3:25 ` Bill Davidsen 1 sibling, 0 replies; 639+ messages in thread From: Rob Landley @ 2005-12-11 5:52 UTC (permalink / raw) To: Douglas McNaught Cc: Bill Davidsen, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Saturday 10 December 2005 11:05, Douglas McNaught wrote: > The kernel developers owe *nothing* to J. Random User. They are > either doing what they do for their own reasons (the "fun" of it), or > being paid by an organization with specific objectives (even if, in > Linus' case, the objective is just "make the best kernel possible, > based on your judgement and that of people you trust"). That's not a good argument. I don't think Bill has a good argument either, but this isn't one. The kernel developers have very good reasons for what they're doing, and ultimately they believe that what they're doing is in the best long-term interests of the users. Bill has objections based on the short-term interests of users. The kernel developers are saying that looking after the short-term interests of the users is the distros' problems, while they focus on the long-term and the big picture. Both are doing what they believe to be in the best interests of the users. Bill believes (and keeps uselessly repeating) that the kernel developers should pay more attention to short-term interests. The whole "stable series" vs "continuous development" is about short-term interests vs long-term interests. Overall, development of new technology (and adoption of new technology) goes faster when it's continuous than when you have large discontinuous jumps. You find problems faster, and you find them one at a time when it's easy to isolate what's wrong rather than receiving three years of development in one gulp and experiencing fifteen different failures all at once. Problems are more frequent, but they're smaller and simpler and easier to diagnose and easier to fix. User can _cope_ with a higher rate of change when it comes in smaller chunks. The learning curve isn't a cliff. > They don't owe you security fixes either. Actually, they seem to think they do. They're quite dilligent about providing security fixes. But the security fixes they give are part of the ongoing development process. Separating them out and backporting them to previous versions is your problem, and if you don't feel up to it they point you to distributions, which will do that for you. So there are two different ways you can get the fixes. (Upgrade, or use a distro maintained kernel.) Some people think they're owed a third way, and want somebody else to provide it for them. > Your "ivory tower" statement is really condescending. *shrug* This whole thread is basically people bitching about what other people should do, instead of banding together and giving it a try themselves. It started condescending. In the absence of constructive suggestions or anyone actually doing real work rather than merely bitching about the state of the world, I think most of the developers have written off the whiners with a silent "sucks to be you" and moved on... (I do hope we get the "buffer flush" dot-releases though. That would be nice.) Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-10 17:05 ` Douglas McNaught 2005-12-11 5:52 ` Rob Landley @ 2005-12-12 3:25 ` Bill Davidsen 1 sibling, 0 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-12 3:25 UTC (permalink / raw) To: Douglas McNaught Cc: Rob Landley, Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree Douglas McNaught wrote: >Bill Davidsen <davidsen@tmr.com> writes: > > > >>Rob Landley wrote: >> >> >> >>>Re-raising the same objections over and over again when they've >>>already been aired, considered, and rejections is called "whining". >>> >>> >>> >>Repeating the same information over and over until it sinks in is >>called "rote learning." The question is not if cryptoloop is perfect, >>every crypto seems to fail eventually, recently md5 was cracked, >>etc. But people have used cryptoloop now, and removing it from the >>kernel will lock them out of their own data, or prevent them from >>moving forward. There's no replacement for cryptoloop, so I can't just >>reconfigure X and still read my 147 DVDs full of business data, or >>access the current data on 34 laptops around the country. >> >> > >Bill, I still don't think your complaints are justified. > > I never expected anyone to admit they were wrong, so that doesn't surprise me... >You're only "locked out of your own data" if you knowingly upgrade to >a kernel that doesn't support cryptoloop. Nobody's forcing you to do >that. > > Are you endorsing ignoring security fixes? Of course you're forced to if you are trying to be secure. If there were a replacement for cryptoloop that wouldn't be a problem. But saying that CL must go because it isn't perfect is like saying that you shouldn't lock your window because someone could still break it and get in. >The kernel developers owe *nothing* to J. Random User. They are >either doing what they do for their own reasons (the "fun" of it), or >being paid by an organization with specific objectives (even if, in >Linus' case, the objective is just "make the best kernel possible, >based on your judgement and that of people you trust"). > > A little later you say that most of the developers are paid for working on Linux. Just who is the ultimate source of that funding if not the random user? Almost all of it comes from people who lack the ability to maintain the kernel, one way or the other. If kernel features are left to the vendors you encourage fragmentation, and all you have to do is look at BSD to see what a success that is. What Linux has going over Windows is choice... the ability to configure WITHOUT having to depend on the judgement of someone else. And when that judgement is to remove a feature which has no replacement, in which uses have made an investment, then the choice is gone. >That said, of course none of them want to break things unnecessarily. >But they make technical decisions, with the goal of having the best >kernel, that do sometimes have painful consequences. You're free, of >course, to disagree with those decisions and maintain your own kernel. > > Why waste electrons on statments like that. Yes, I could do that, but the average users can't, and after maintaining GECOS, and MULTICS, and supporting BSD installations and writing a realtime control o/s, I certainly don't have the slightest interest in spending my time doing that. Effectively anything not in a kernel.org kernel is going to die. >They don't owe you security fixes either. Sorry, but that's the way >it is. We're all lucky that they take security very seriously and >respond quickly to problems. > > > >>In most cases CL is not expected to protect against goverment agencies >>but rather stolen laptops in airports (yes the pros have added MacOS >>and Linux to their business model) or the occasional lost DVD in the >>mail. Removing CL is not a hell of a lot better morally than these >>viruses which encrypt your data and then hold it for ransom, with the >>price being doing without security fixes in future kernels. >> >> > >That last sentence is crap. > >You're free to backport security fixes to cryptoloop-supporting >kernels forever, or pay someone to do so. Or maintain a cryptoloop >patch against current kernels, or pay someone to do so. Or write a >converter for cryptoloop data to whatever's currently in the kernel, >or pay someone to do so. > > Given that CL has minimal (essentially no) maintenence cost, I wish >>the ivory tower developers could understand that real people have >>invested real money in it, and real data in the technology. Since >>there is no alternative solution offered, CL is far better than no >>crypto at all, and I wish there were a few more developers who had >>experience working in the real word. >> >> > >If you include a crypto solution in the mainstream kernel, you're in >some sense endorsing its security. If that solution has known >weaknesses, I can understand wanting to either fix it or rip it out. >Crypto is hard enough to get right as it is. > >Your "ivory tower" statement is really condescending. Linux is way >past the stage where college students were the main contributors (if >it ever was so after Linus graduated). and a great majority of >developers now are paid to work on the kernel. There are probably >very few of them that don't have at least a little sysadmin >experience. > > I wonder... running servers is relatively easy, supporting end user systems (admin, not help desk) is hard. >If you've invested money and put important data in a system, and you >haven't contracted with anyone to support that system, supply security >fixes, and make sure it does what you want it to do, who's the fool? > > The advantage of using dynamic systems rather than locking in with something like RHEL was attractive. Trusting a third party was not. >Basically, you're complaining about something you get *for free* that >represents millions of hours of work, because it doesn't work quite >the way you want it to, when you have perfect freedom to make it meet >your needs by putting in your own time, effort and/or money. > Most users have no such ability, but people jumped abord Linux when 2.6.0 came out, and it WAS called a "new stable release." Redefining what stable means after people have used the software is not something I would feel comfortable doing. -- bill davidsen <davidsen@tmr.com> CTO TMR Associates, Inc Doing interesting things with small computers since 1979 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-10 13:41 ` RFC: Starting a stable kernel series off the 2.6 kernel Bill Davidsen 2005-12-10 17:05 ` Douglas McNaught @ 2005-12-11 5:33 ` Rob Landley 1 sibling, 0 replies; 639+ messages in thread From: Rob Landley @ 2005-12-11 5:33 UTC (permalink / raw) To: Bill Davidsen Cc: Mark Lord, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Saturday 10 December 2005 07:41, Bill Davidsen wrote: > Given that CL has minimal (essentially no) maintenence cost, then by your own admission maintaining it as an external patch for people who need it for legacy reasons is trivial, and removing it to discourage new users from picking it up remains a good idea. Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 16:28 ` Lee Revell 2005-12-05 16:44 ` Matthias Andree 2005-12-05 17:58 ` Rob Landley @ 2005-12-05 21:22 ` Bill Davidsen 2005-12-06 14:59 ` Bill Davidsen 3 siblings, 0 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-05 21:22 UTC (permalink / raw) To: Lee Revell Cc: Rob Landley, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree Lee Revell wrote: > On Mon, 2005-12-05 at 11:17 -0500, Mark Lord wrote: > >>>>>Ahh OK .. I don't use it, so wouldn't have been affected. That's one >>>>>userspace interface broken during the series, does anyone have any more? >> >>Ah.. another one, that I was just reminded of again >>by the umpteenth person posting that their wireless >>no longer is WPA capable after upgrading from 2.6.12. >> >>Of course, the known solution for that issue is to >>upgrade to the recently "fixed" latest wpa_supplicant >>daemon in userspace, since the old one no longer works. >> >>Things like this are all too regular an occurance. > > > The distro should have solved this problem by making sure that the > kernel upgrade depends on a new wpa_supplicant package. Don't they > bother to test this stuff before they ship it?!? Could you provide a little detail on the technology by which a distro checks for functionality against a kernel which wasn't necessarily released when the distro shipped. My udev doesn't generate /dev/timewarp. Going to a new kernel in the same series shouldn't have to be treated as if it were a change to a whole new operating system, and shouldn't require completely replacing existing utilities with new ones which aren't backware compatible to allow fallback to the original kernel. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 16:28 ` Lee Revell ` (2 preceding siblings ...) 2005-12-05 21:22 ` Bill Davidsen @ 2005-12-06 14:59 ` Bill Davidsen 3 siblings, 0 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-06 14:59 UTC (permalink / raw) To: Lee Revell Cc: Rob Landley, Adrian Bunk, David Ranson, Steven Rostedt, linux-kernel, Matthias Andree Lee Revell wrote: > On Mon, 2005-12-05 at 11:17 -0500, Mark Lord wrote: > >>>>>Ahh OK .. I don't use it, so wouldn't have been affected. That's one >>>>>userspace interface broken during the series, does anyone have any more? >> >>Ah.. another one, that I was just reminded of again >>by the umpteenth person posting that their wireless >>no longer is WPA capable after upgrading from 2.6.12. >> >>Of course, the known solution for that issue is to >>upgrade to the recently "fixed" latest wpa_supplicant >>daemon in userspace, since the old one no longer works. >> >>Things like this are all too regular an occurance. > > > The distro should have solved this problem by making sure that the > kernel upgrade depends on a new wpa_supplicant package. Don't they > bother to test this stuff before they ship it?!? Could you provide a little detail on the technology by which a distro checks for functionality against a kernel which wasn't necessarily released when the distro shipped. My udev doesn't generate /dev/timewarp. Going to a new kernel in the same series shouldn't have to be treated as if it were a change to a whole new operating system, and shouldn't require completely replacing existing utilities with new ones which aren't backware compatible to allow fallback to the original kernel. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 3:31 ` Rob Landley 2005-12-05 16:17 ` Mark Lord @ 2005-12-05 18:44 ` Adrian Bunk 1 sibling, 0 replies; 639+ messages in thread From: Adrian Bunk @ 2005-12-05 18:44 UTC (permalink / raw) To: Rob Landley; +Cc: David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Sun, Dec 04, 2005 at 09:31:12PM -0600, Rob Landley wrote: > On Saturday 03 December 2005 11:53, Adrian Bunk wrote: > > On Sat, Dec 03, 2005 at 05:17:41PM +0000, David Ranson wrote: > > > Steven Rostedt wrote: > > > >udev ;) > > > > > > > >http://seclists.org/lists/linux-kernel/2005/Dec/0180.html > > > > > > Ahh OK .. I don't use it, so wouldn't have been affected. That's one > > > userspace interface broken during the series, does anyone have any more? > > > > - support for ipfwadm and ipchains was removed during 2.6 > > - devfs support was removed during 2.6 > > - removal of kernel support for pcmcia-cs is pending > > - ip{,6}_queue removal is pending > > - removal of the RAW driver is pending > > So what you're upset about is the feature removal scheduling mechanism, which > usually gives a full year's warning, and the removal patch can be reversed > into a feature addition patch you can maintain outside the tree if you really > care? I'm not upset about is the feature removal scheduling mechanism. Please check who has most entries in Documentation/feature-removal-schedule.txt ... The removal of features within the 2.6 series is an essential part of the current development model. The problem is the lack of a long-living relatively stable series within the development model. > Rob cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 17:17 ` David Ranson 2005-12-03 17:53 ` Adrian Bunk @ 2005-12-03 18:21 ` Mark Lord 2005-12-03 19:22 ` Linus Torvalds 2005-12-03 22:21 ` Matthias Andree 2 siblings, 1 reply; 639+ messages in thread From: Mark Lord @ 2005-12-03 18:21 UTC (permalink / raw) To: David Ranson; +Cc: Steven Rostedt, linux-kernel, Matthias Andree David Ranson wrote: > > Ahh OK .. I don't use it, so wouldn't have been affected. That's one > userspace interface broken during the series, does anyone have any more? vbetool. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 18:21 ` Mark Lord @ 2005-12-03 19:22 ` Linus Torvalds 2005-12-04 3:32 ` Mark Lord 0 siblings, 1 reply; 639+ messages in thread From: Linus Torvalds @ 2005-12-03 19:22 UTC (permalink / raw) To: Mark Lord; +Cc: David Ranson, Steven Rostedt, linux-kernel, Matthias Andree On Sat, 3 Dec 2005, Mark Lord wrote: > David Ranson wrote: > > > > Ahh OK .. I don't use it, so wouldn't have been affected. That's one > > userspace interface broken during the series, does anyone have any more? > > vbetool. I don't think vbetool has been "broken", it should work fine again. It was just temporarily (and unintentionally) broken for a while. But if it's still broken in 2.6.15-rc4, please do holler (with as many details as you can) Linus ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 19:22 ` Linus Torvalds @ 2005-12-04 3:32 ` Mark Lord 0 siblings, 0 replies; 639+ messages in thread From: Mark Lord @ 2005-12-04 3:32 UTC (permalink / raw) To: Linus Torvalds Cc: David Ranson, Steven Rostedt, linux-kernel, Matthias Andree Linus Torvalds wrote: > > I don't think vbetool has been "broken", it should work fine again. It was > just temporarily (and unintentionally) broken for a while. > > But if it's still broken in 2.6.15-rc4, please do holler Yup, working fine again now with rc4. Thanks Linus! ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 17:17 ` David Ranson 2005-12-03 17:53 ` Adrian Bunk 2005-12-03 18:21 ` Mark Lord @ 2005-12-03 22:21 ` Matthias Andree 2005-12-03 22:29 ` Greg KH 2 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2005-12-03 22:21 UTC (permalink / raw) To: linux-kernel On Sat, 03 Dec 2005, David Ranson wrote: > Steven Rostedt wrote: > > >udev ;) > > > >http://seclists.org/lists/linux-kernel/2005/Dec/0180.html > > > > > Ahh OK .. I don't use it, so wouldn't have been affected. That's one > userspace interface broken during the series, does anyone have any more? Not only that, udev is default for instance in recent SUSE Linux releases, so whether to use or not to use it is a major effort. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:21 ` Matthias Andree @ 2005-12-03 22:29 ` Greg KH 2005-12-03 22:41 ` Matthias Andree 2005-12-03 22:48 ` Steven Rostedt 0 siblings, 2 replies; 639+ messages in thread From: Greg KH @ 2005-12-03 22:29 UTC (permalink / raw) To: linux-kernel On Sat, Dec 03, 2005 at 11:21:38PM +0100, Matthias Andree wrote: > On Sat, 03 Dec 2005, David Ranson wrote: > > > Steven Rostedt wrote: > > > > >udev ;) > > > > > >http://seclists.org/lists/linux-kernel/2005/Dec/0180.html > > > > > > > > Ahh OK .. I don't use it, so wouldn't have been affected. That's one > > userspace interface broken during the series, does anyone have any more? > > Not only that, udev is default for instance in recent SUSE Linux > releases, so whether to use or not to use it is a major effort. And if you use SUSE releases, use OpenSuSE to keep up to date with all of the needed kernel programs for the latest kernels. Same with Fedora, Debian, or Gentoo. They all keep up to date quite well. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:29 ` Greg KH @ 2005-12-03 22:41 ` Matthias Andree 2005-12-03 22:48 ` Steven Rostedt 1 sibling, 0 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-03 22:41 UTC (permalink / raw) To: linux-kernel On Sat, 03 Dec 2005, Greg KH wrote: > And if you use SUSE releases, use OpenSuSE to keep up to date with all > of the needed kernel programs for the latest kernels. Same with Fedora, > Debian, or Gentoo. They all keep up to date quite well. Well, particular problem I've had: netfilter-enabled machines were incapable to download large files, for instance newer ICC8 releases, (major annoyance, their IIS crap doesn't support "Bytes" ranges, so you start all over if one packet went down the wrong throat). I was told "try newer kernels first", there had been fixes with out-of-window ACKs and whatnot. I do not intend to upgrade all of my distro to the bleeding OpenSUSE release just to find out if the new kernel fixes it and to see new regressions. I have more interesting things to do with my time than chase the userspace change of the day. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 22:29 ` Greg KH 2005-12-03 22:41 ` Matthias Andree @ 2005-12-03 22:48 ` Steven Rostedt 1 sibling, 0 replies; 639+ messages in thread From: Steven Rostedt @ 2005-12-03 22:48 UTC (permalink / raw) To: Greg KH; +Cc: linux-kernel On Sat, 2005-12-03 at 14:29 -0800, Greg KH wrote: > On Sat, Dec 03, 2005 at 11:21:38PM +0100, Matthias Andree wrote: > > On Sat, 03 Dec 2005, David Ranson wrote: > > > > > Steven Rostedt wrote: > > > > > > >udev ;) > > > > > > > >http://seclists.org/lists/linux-kernel/2005/Dec/0180.html > > > > > > > > > > > Ahh OK .. I don't use it, so wouldn't have been affected. That's one > > > userspace interface broken during the series, does anyone have any more? > > > > Not only that, udev is default for instance in recent SUSE Linux > > releases, so whether to use or not to use it is a major effort. > > And if you use SUSE releases, use OpenSuSE to keep up to date with all > of the needed kernel programs for the latest kernels. Same with Fedora, > Debian, or Gentoo. They all keep up to date quite well. And if you use Debian, make sure you remember to do an update after changing a kernel and finding out that something in userland doesn't work. As Greg mentioned to me in the above mentioned thread. Debian/unstable had the proper udev. I just haven't done an update recently, but I download kernels practically every day. -- Steve ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 16:27 ` Matthias Andree 2005-12-03 16:40 ` Otavio Salvador 2005-12-03 16:58 ` David Ranson @ 2005-12-03 17:22 ` Arjan van de Ven 2005-12-03 17:35 ` M. 2005-12-03 23:05 ` Matthias Andree 2 siblings, 2 replies; 639+ messages in thread From: Arjan van de Ven @ 2005-12-03 17:22 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel > Exactly that, and kernel interfaces going away just to annoy binary > module providers also hurts third-party OSS modules, such as > Fujitsu-Siemens's ServerView agents. in kernel API never was and never will be stable, that's entirely irrelevant and independent of the proposal at hand. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 17:22 ` Arjan van de Ven @ 2005-12-03 17:35 ` M. 2005-12-03 23:05 ` Matthias Andree 1 sibling, 0 replies; 639+ messages in thread From: M. @ 2005-12-03 17:35 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Matthias Andree, linux-kernel Hi everyone. Here to expose my point of view, hope u'll enjoy :) Starting stable series off the 2.6 simply wouldn't work: there are no paid QA guys ready to test, fix, stabilize those series, kernel developers wants to hang on new exciting stuff. This tendence leads to faster innovations in kernel core and features set but leaving the "forking" effort to distributions leads to fragmentation too: almost every distro has a different base kernel on which doing testing and fixing and this, in my opinion, is not positive for the kernel.org kernels. Another problem of the current development model is that fast changes are difficult to track for small external projects (those whitout big $$ behind), the small projects which made linux so great in the past. Maybe a way to reduce those problems is to release the kernel like GNOME, KDE, fedora & co. do for their stuff: one stable release every 6 months but build on top of the current way of doing things. Example: 2.6.14 released on 27 October, then: 2.6.14.1-gitN until 2.6.14.1-rcN -> 2.6.14.1 2.6.14.2-gitN until 2.6.14.2-rcN -> 2.6.14.2 ... (maybe last 2.6.14-N, which could be called 2.6.15-gitN -> 2.6.15-rcN, only bugfixes and small changes, main development in -mm or in a new -something tree during this last phase) 2.6.15 release on March those middle releases would be handled with the current development model except for the last one. So, the largest part of developers would continue to think and to work using the current development model and some guys would be able to plan a list of features and functionalities every 6 months and, based on this list, handle the 6-months release giving guide lines (like Linus and friends already do but, i repeat, focusing on a 6 months time window) Doing things this way would lead to distributions aligning to the same kernel and open up a possible scenario of distros collaborating to mantain the latest stable release. This should make small projects and users who want to run bleeding edge stuff lifes easier too. of couse the time window could be larger or smaller but doing things 6months-based should align kernel development to some other big projects too. cheers, Michele ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 17:22 ` Arjan van de Ven 2005-12-03 17:35 ` M. @ 2005-12-03 23:05 ` Matthias Andree 2005-12-03 23:37 ` Greg KH 2005-12-04 0:52 ` Jeff V. Merkey 1 sibling, 2 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-03 23:05 UTC (permalink / raw) To: linux-kernel On Sat, 03 Dec 2005, Arjan van de Ven wrote: > > > Exactly that, and kernel interfaces going away just to annoy binary > > module providers also hurts third-party OSS modules, such as > > Fujitsu-Siemens's ServerView agents. > > in kernel API never was and never will be stable, that's entirely > irrelevant and independent of the proposal at hand. It's still an annoying side effect - is there a list of kernel functions removed, with version removed, and with replacement? I know of none, but then again I don't hack the kernel very often. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 23:05 ` Matthias Andree @ 2005-12-03 23:37 ` Greg KH 2005-12-04 0:52 ` Jeff V. Merkey 1 sibling, 0 replies; 639+ messages in thread From: Greg KH @ 2005-12-03 23:37 UTC (permalink / raw) To: linux-kernel On Sun, Dec 04, 2005 at 12:05:20AM +0100, Matthias Andree wrote: > On Sat, 03 Dec 2005, Arjan van de Ven wrote: > > > > > > Exactly that, and kernel interfaces going away just to annoy binary > > > module providers also hurts third-party OSS modules, such as > > > Fujitsu-Siemens's ServerView agents. > > > > in kernel API never was and never will be stable, that's entirely > > irrelevant and independent of the proposal at hand. > > It's still an annoying side effect - is there a list of kernel functions > removed, with version removed, and with replacement? I know of none, but > then again I don't hack the kernel very often. Both lwn.net and the kernelnewbies wiki are trying to track this. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 23:05 ` Matthias Andree 2005-12-03 23:37 ` Greg KH @ 2005-12-04 0:52 ` Jeff V. Merkey 2005-12-04 12:12 ` Matthias Andree ` (2 more replies) 1 sibling, 3 replies; 639+ messages in thread From: Jeff V. Merkey @ 2005-12-04 0:52 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel Matthias Andree wrote: >On Sat, 03 Dec 2005, Arjan van de Ven wrote: > > > >>>Exactly that, and kernel interfaces going away just to annoy binary >>>module providers also hurts third-party OSS modules, such as >>>Fujitsu-Siemens's ServerView agents. >>> >>> >>in kernel API never was and never will be stable, that's entirely >>irrelevant and independent of the proposal at hand. >> >> > >It's still an annoying side effect - is there a list of kernel functions >removed, with version removed, and with replacement? I know of none, but >then again I don't hack the kernel very often. > > > These folks have nothing new to innovate here. The memory manager and VM gets revamped every other release. Exports get broken, binary only module compatibility busted every rev of the kernel. I spend weeks on each kernel fixing the breakage. These people don't get it, don't care, and to be honest, you are wasting your time here trying to convince them. It's never stable because they don't want it to be. This is how they maintain control of this code. I have apps written for Windows in 1990 and 1998 that still run on Windows XP today. Linux has no such concept of backwards compatiblity. Every company who has embraced it outside of hardware based solutions is dying or has died. IBM is secretly forking it as we speak and using it to get out of paying for Unix licenses. As annoying as it is, accept it and live with it. These people have no sense of loyalty to you or your customers. They don't even care about each other. Linux is not a "family" in any sense. I wanted very much to believe this and I was loyal to these folks for 10 years, invested millions of dollars in development of my and others money in development to support it, crippled Novell and pushed them to embrace Linux and my reward was smearing and expulsion from the community. They have no direction and the whole thing is stagnant now. All the development is incremental bug fixes and anti-competitive mods to break each others distros. You are standing on a battlefield. Quietly fork each release, make your mods, post patches somewhere for the poor civilians who are "collateral damage" in the war with constantly busted software, and you might help some folks. Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 0:52 ` Jeff V. Merkey @ 2005-12-04 12:12 ` Matthias Andree 2005-12-04 12:32 ` Arjan van de Ven 2005-12-04 19:57 ` Horst von Brand 2005-12-04 21:35 ` Bernd Petrovitsch 2 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2005-12-04 12:12 UTC (permalink / raw) To: Jeff V. Merkey; +Cc: linux-kernel On Sat, 03 Dec 2005, Jeff V. Merkey wrote: > These folks have nothing new to innovate here. The memory manager and VM > gets revamped every other release. Exports get broken, binary only > module compatibility busted every rev of the kernel. I spend weeks on Who cares for binary modules? It hurts however if external OSS modules are broken. > of this code. I have apps written for Windows in 1990 and 1998 that > still run on Windows XP today. Sure, you're loading Windows 3.1 drivers into XP... You can tell us more of that crap later, but not here. Properly written 1995 software usually still works on Linux as long as it doesn't need to care about kernel or devices. [rest of Merkey rantings removed] -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 12:12 ` Matthias Andree @ 2005-12-04 12:32 ` Arjan van de Ven 2005-12-04 13:28 ` Matthias Andree 0 siblings, 1 reply; 639+ messages in thread From: Arjan van de Ven @ 2005-12-04 12:32 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel On Sun, 2005-12-04 at 13:12 +0100, Matthias Andree wrote: > On Sat, 03 Dec 2005, Jeff V. Merkey wrote: > > > These folks have nothing new to innovate here. The memory manager and VM > > gets revamped every other release. Exports get broken, binary only > > module compatibility busted every rev of the kernel. I spend weeks on > > Who cares for binary modules? > > It hurts however if external OSS modules are broken. then those modules should be submitted realistically. That's just best for everyone involved. Which modules in particular do you mean btw? It's rare even in the 2.6 tree to mass-break well written drivers. Just because it's a lot of work to fix all in kernel drivers up. But a fully stable API is also not good. My guess is that the drivers that break most are the ones using the not-right APIs (eg internals and such). ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 12:32 ` Arjan van de Ven @ 2005-12-04 13:28 ` Matthias Andree 2005-12-04 13:35 ` Arjan van de Ven 0 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2005-12-04 13:28 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Matthias Andree, linux-kernel On Sun, 04 Dec 2005, Arjan van de Ven wrote: > On Sun, 2005-12-04 at 13:12 +0100, Matthias Andree wrote: > > On Sat, 03 Dec 2005, Jeff V. Merkey wrote: > > > > > These folks have nothing new to innovate here. The memory manager and VM > > > gets revamped every other release. Exports get broken, binary only > > > module compatibility busted every rev of the kernel. I spend weeks on > > > > Who cares for binary modules? > > > > It hurts however if external OSS modules are broken. > > then those modules should be submitted realistically. That's just best > for everyone involved. Which modules in particular do you mean btw? I meant the ipmi, smbus and copa modules by Fujitsu-Siemens. They are provided in source form, but I just found out (reading the headers and not just the lines that broke the compile) they are not open source. Perhaps one should prod them to slap a modified-BSD or perhaps GPL label onto their modules. It seems you'd then maintain them after their submission? :-) > It's rare even in the 2.6 tree to mass-break well written drivers. Just > because it's a lot of work to fix all in kernel drivers up. But a fully > stable API is also not good. My guess is that the drivers that break > most are the ones using the not-right APIs (eg internals and such). These use inter_module_get() (ok, inter_module_get_request isn't difficult) and some #include headers that have moved around between linux and asm directories. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 13:28 ` Matthias Andree @ 2005-12-04 13:35 ` Arjan van de Ven 2005-12-04 14:25 ` Matthias Andree 0 siblings, 1 reply; 639+ messages in thread From: Arjan van de Ven @ 2005-12-04 13:35 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel On Sun, 2005-12-04 at 14:28 +0100, Matthias Andree wrote: > I meant the ipmi, smbus and copa modules by Fujitsu-Siemens. > > They are provided in source form, but I just found out (reading the > headers and not just the lines that broke the compile) they are not open > source. Perhaps one should prod them to slap a modified-BSD or perhaps > GPL label onto their modules. is there an URL to these? > > It seems you'd then maintain them after their submission? :-) usually such modules are extremely low maintenance once merged.... There are many many drivers without a maintainer, and they still get fixed. > > It's rare even in the 2.6 tree to mass-break well written drivers. Just > > because it's a lot of work to fix all in kernel drivers up. But a fully > > stable API is also not good. My guess is that the drivers that break > > most are the ones using the not-right APIs (eg internals and such). > > These use inter_module_get() which is still there even in the latest 2.6.15-rc. It should be going out but hasn't yet. And that is the case for at least a year (eg they are __deprecated but still there). > and some #include headers that have moved around between > linux and asm directories. Most of those were already in the final place with a temporary compat header in the old one I guess. But if this is all.... that really isn't a big deal. I suspect some of these headers aren't even used by the driver (sometimes people just include the world for no reason).. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 13:35 ` Arjan van de Ven @ 2005-12-04 14:25 ` Matthias Andree 2005-12-04 14:50 ` Arjan van de Ven ` (2 more replies) 0 siblings, 3 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-04 14:25 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Linux-Kernel mailing list On Sun, 04 Dec 2005, Arjan van de Ven wrote: > On Sun, 2005-12-04 at 14:28 +0100, Matthias Andree wrote: > > > I meant the ipmi, smbus and copa modules by Fujitsu-Siemens. > > > > They are provided in source form, but I just found out (reading the > > headers and not just the lines that broke the compile) they are not open > > source. Perhaps one should prod them to slap a modified-BSD or perhaps > > GPL label onto their modules. > > is there an URL to these? http://download.fujitsu-siemens.com/prim_supportcd/Programs/General/ServView/Linux/agents/srvmagt-mods_src.suse.rpm > > It seems you'd then maintain them after their submission? :-) > > usually such modules are extremely low maintenance once merged.... There > are many many drivers without a maintainer, and they still get fixed. As I say, these aren't licensed for inclusion into the kernel, they bear a (C) Copyright notice and "All rights reserved." > > These use inter_module_get() > > which is still there even in the latest 2.6.15-rc. It should be going > out but hasn't yet. And that is the case for at least a year (eg they > are __deprecated but still there). No, they aren't - at least not anywhere declared below include/ and thus uncompilable with GCC4. > Most of those were already in the final place with a temporary compat > header in the old one I guess. But if this is all.... that really isn't > a big deal. I suspect some of these headers aren't even used by the > driver (sometimes people just include the world for no reason).. The reason would be convenience, or maintainer efficiency as the Camel book ("Programming Perl") words it. If not including the right header file (such as ioctl32.h on x86_64) breaks the compile, I presume they are needed. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 14:25 ` Matthias Andree @ 2005-12-04 14:50 ` Arjan van de Ven 2005-12-04 15:08 ` Matthias Andree 2005-12-04 15:20 ` Arjan van de Ven 2005-12-04 15:25 ` Richard Knutsson 2 siblings, 1 reply; 639+ messages in thread From: Arjan van de Ven @ 2005-12-04 14:50 UTC (permalink / raw) To: Matthias Andree; +Cc: Linux-Kernel mailing list On Sun, 2005-12-04 at 15:25 +0100, Matthias Andree wrote: > On Sun, 04 Dec 2005, Arjan van de Ven wrote: > > > On Sun, 2005-12-04 at 14:28 +0100, Matthias Andree wrote: > > > > > I meant the ipmi, smbus and copa modules by Fujitsu-Siemens. > > > > > > They are provided in source form, but I just found out (reading the > > > headers and not just the lines that broke the compile) they are not open > > > source. Perhaps one should prod them to slap a modified-BSD or perhaps > > > GPL label onto their modules. > > > > is there an URL to these? > > http://download.fujitsu-siemens.com/prim_supportcd/Programs/General/ServView/Linux/agents/srvmagt-mods_src.suse.rpm > > > > It seems you'd then maintain them after their submission? :-) > > > > usually such modules are extremely low maintenance once merged.... There > > are many many drivers without a maintainer, and they still get fixed. > > As I say, these aren't licensed for inclusion into the kernel, they bear > a (C) Copyright notice and "All rights reserved." and MODULE_LICENSE("GPL"); so it *IS* gpl licensed! the code is a bit horrible though and no surprise it breaks ;) you can always make drivers broken enough to break at the slightest change ;) (it also seems to contain an entire ipmi layer, linux already has one so I wonder why they're not just using that as basis) ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 14:50 ` Arjan van de Ven @ 2005-12-04 15:08 ` Matthias Andree 2005-12-04 15:11 ` Arjan van de Ven 2005-12-04 15:36 ` Andreas Schwab 0 siblings, 2 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-04 15:08 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Matthias Andree, Linux-Kernel mailing list On Sun, 04 Dec 2005, Arjan van de Ven wrote: > > As I say, these aren't licensed for inclusion into the kernel, they bear > > a (C) Copyright notice and "All rights reserved." > > and > MODULE_LICENSE("GPL"); > > so it *IS* gpl licensed! > > the code is a bit horrible though and no surprise it breaks ;) Yes. "extern type foo; static type foo;" is way stupid, but 10% of the blame can be shifted on the GCC guys for being much too tolerant. > you can always make drivers broken enough to break at the slightest > change ;) > > (it also seems to contain an entire ipmi layer, linux already has one so > I wonder why they're not just using that as basis) Perhaps the dates give a clue. Since when has Linux had IPMI in the baseline code? -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 15:08 ` Matthias Andree @ 2005-12-04 15:11 ` Arjan van de Ven 2005-12-04 15:36 ` Andreas Schwab 1 sibling, 0 replies; 639+ messages in thread From: Arjan van de Ven @ 2005-12-04 15:11 UTC (permalink / raw) To: Matthias Andree; +Cc: Linux-Kernel mailing list > Perhaps the dates give a clue. Since when has Linux had IPMI in the > baseline code? 2.4.21 already had it, so it's been quite a while ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 15:08 ` Matthias Andree 2005-12-04 15:11 ` Arjan van de Ven @ 2005-12-04 15:36 ` Andreas Schwab 2005-12-04 16:17 ` Matthias Andree 1 sibling, 1 reply; 639+ messages in thread From: Andreas Schwab @ 2005-12-04 15:36 UTC (permalink / raw) To: Linux-Kernel mailing list Matthias Andree <matthias.andree@gmx.de> writes: > Yes. "extern type foo; static type foo;" is way stupid, but 10% of the > blame can be shifted on the GCC guys for being much too tolerant. You should rather blame the C standard. Andreas. -- Andreas Schwab, SuSE Labs, schwab@suse.de SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany PGP key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 15:36 ` Andreas Schwab @ 2005-12-04 16:17 ` Matthias Andree 2005-12-05 3:09 ` Joel Becker 0 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2005-12-04 16:17 UTC (permalink / raw) To: Linux-Kernel mailing list On Sun, 04 Dec 2005, Andreas Schwab wrote: > Matthias Andree <matthias.andree@gmx.de> writes: > > > Yes. "extern type foo; static type foo;" is way stupid, but 10% of the > > blame can be shifted on the GCC guys for being much too tolerant. > > You should rather blame the C standard. There are things that old Sun Workshop versions bitch about that GCC deals with without complaining, and I'm not talking about C99/C++-style comments. C standard issue? I believe not. Anyways, this is getting off-topic and ultimately the author of broken code is responsible, of course. But it's still nice if the tools help produce good code. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 16:17 ` Matthias Andree @ 2005-12-05 3:09 ` Joel Becker 2005-12-06 20:13 ` Alan Cox 0 siblings, 1 reply; 639+ messages in thread From: Joel Becker @ 2005-12-05 3:09 UTC (permalink / raw) To: Linux-Kernel mailing list On Sun, Dec 04, 2005 at 05:17:09PM +0100, Matthias Andree wrote: > There are things that old Sun Workshop versions bitch about that GCC > deals with without complaining, and I'm not talking about C99/C++-style > comments. C standard issue? I believe not. I have seen many a code like so: char buf[4]; memcpy(buf, source, 5); accepted by the Sun compilers and run just fine. When the application was ported to Linux/GCC, the developers complained their program segfaulted, and "it must be something broken on Linux!" Just because Sun's compiler does something doesn't mean it's right :-) Joel -- Life's Little Instruction Book #510 "Count your blessings." Joel Becker Principal Software Developer Oracle E-mail: joel.becker@oracle.com Phone: (650) 506-8127 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 3:09 ` Joel Becker @ 2005-12-06 20:13 ` Alan Cox 0 siblings, 0 replies; 639+ messages in thread From: Alan Cox @ 2005-12-06 20:13 UTC (permalink / raw) To: Joel Becker; +Cc: Linux-Kernel mailing list On Sul, 2005-12-04 at 19:09 -0800, Joel Becker wrote: > On Sun, Dec 04, 2005 at 05:17:09PM +0100, Matthias Andree wrote: > > There are things that old Sun Workshop versions bitch about that GCC > > deals with without complaining, and I'm not talking about C99/C++-style > > comments. C standard issue? I believe not. > > I have seen many a code like so: > > char buf[4]; > memcpy(buf, source, 5); > > accepted by the Sun compilers and run just fine. When the application > was ported to Linux/GCC, the developers complained their program > segfaulted, and "it must be something broken on Linux!" > Just because Sun's compiler does something doesn't mean it's It isnt the compiler quite often. The usual case is char buf[4]; strcpy(buf, "bits"); And those cases usually work because its a big endian box and the \00 ends up overwriting the \00 in the return address. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 14:25 ` Matthias Andree 2005-12-04 14:50 ` Arjan van de Ven @ 2005-12-04 15:20 ` Arjan van de Ven 2005-12-04 16:23 ` Matthias Andree 2005-12-04 15:25 ` Richard Knutsson 2 siblings, 1 reply; 639+ messages in thread From: Arjan van de Ven @ 2005-12-04 15:20 UTC (permalink / raw) To: Matthias Andree; +Cc: Linux-Kernel mailing list > (C) Copyright notice and "All rights reserved." > > > > These use inter_module_get() > > > > which is still there even in the latest 2.6.15-rc. It should be going > > out but hasn't yet. And that is the case for at least a year (eg they > > are __deprecated but still there). > > No, they aren't - at least not anywhere declared below include/ and thus > uncompilable with GCC4. # pwd /mnt/raid/linux/linux-2.6.15-rc4/include/linux [root@jackhammer linux]# grep inter_mod * module.h:extern void __deprecated inter_module_register(const char *, module.h:extern void __deprecated inter_module_unregister(const char *); module.h:extern const void * __deprecated inter_module_get_request(const char *, module.h:extern void __deprecated inter_module_put(const char *); sounds you need to invoke the warranty on your grep binary ;) ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 15:20 ` Arjan van de Ven @ 2005-12-04 16:23 ` Matthias Andree 0 siblings, 0 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-04 16:23 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Matthias Andree, Linux-Kernel mailing list On Sun, 04 Dec 2005, Arjan van de Ven wrote: > > (C) Copyright notice and "All rights reserved." > > > > > > These use inter_module_get() > > > > > > which is still there even in the latest 2.6.15-rc. It should be going > > > out but hasn't yet. And that is the case for at least a year (eg they > > > are __deprecated but still there). > > > > No, they aren't - at least not anywhere declared below include/ and thus > > uncompilable with GCC4. > > # pwd > /mnt/raid/linux/linux-2.6.15-rc4/include/linux > [root@jackhammer linux]# grep inter_mod * > module.h:extern void __deprecated inter_module_register(const char *, > module.h:extern void __deprecated inter_module_unregister(const char *); > module.h:extern const void * __deprecated inter_module_get_request(const > char *, > module.h:extern void __deprecated inter_module_put(const char *); Same story with -rc5. As you can see, there is no declaration for inter_module_get(), just the _request() variant. So what now? :-P -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 14:25 ` Matthias Andree 2005-12-04 14:50 ` Arjan van de Ven 2005-12-04 15:20 ` Arjan van de Ven @ 2005-12-04 15:25 ` Richard Knutsson 2005-12-04 15:23 ` Arjan van de Ven ` (2 more replies) 2 siblings, 3 replies; 639+ messages in thread From: Richard Knutsson @ 2005-12-04 15:25 UTC (permalink / raw) To: Matthias Andree; +Cc: Arjan van de Ven, Linux-Kernel mailing list Matthias Andree wrote: >As I say, these aren't licensed for inclusion into the kernel, they bear >a (C) Copyright notice and "All rights reserved." > > In the 2.6.15-rc5 kernel we find: data@amazon linux-2.6.15-rc5]$ find . -name *.[chS] | xargs grep -n "All rights reserved" | wc -l 932 [data@amazon linux-2.6.15-rc5]$ find . -name *.[chS] | xargs grep -n "Copyright" | wc -l 15083 But I do wonder how copyright and GPL can co-exist. Do the copyright holder own the changes anybody else does to the code? Anyone care to explain? Thanks Richard Knutsson ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 15:25 ` Richard Knutsson @ 2005-12-04 15:23 ` Arjan van de Ven 2005-12-05 23:51 ` Rob Landley 2005-12-06 20:40 ` Matan Peled 2 siblings, 0 replies; 639+ messages in thread From: Arjan van de Ven @ 2005-12-04 15:23 UTC (permalink / raw) To: Richard Knutsson; +Cc: Matthias Andree, Linux-Kernel mailing list > But I do wonder how copyright and GPL can co-exist. Do the copyright > holder own the changes anybody else does to the code? > Anyone care to explain? The GPL *is* copyright. You and I as copyright holders reserve all rights, and then grant selected rights; the rights and the conditions they are granted under are described in the COPYING file. It's a misunderstanding to think that GPL means "no copyright" or "can do anything I want"; in a way the GPL is quite a restrictive license. (while it allows you to distribute, copy and make derived works, it only does allow that under the condition that the result is made available under the GPL as well in full source form, and there's some additional conditions as well) so GPL can copyright are not in conflict, the GPL can exist BECAUSE of copyright actually. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 15:25 ` Richard Knutsson 2005-12-04 15:23 ` Arjan van de Ven @ 2005-12-05 23:51 ` Rob Landley 2005-12-06 20:40 ` Matan Peled 2 siblings, 0 replies; 639+ messages in thread From: Rob Landley @ 2005-12-05 23:51 UTC (permalink / raw) To: Richard Knutsson Cc: Matthias Andree, Arjan van de Ven, Linux-Kernel mailing list On Sunday 04 December 2005 09:25, Richard Knutsson wrote: > But I do wonder how copyright and GPL can co-exist. Do the copyright > holder own the changes anybody else does to the code? > Anyone care to explain? The GPL is a copyright license. A license is a permission statement, ala "you can pitch a tent on my lawn as long as you don't leave trash all over it". Doesn't change ownership of the lawn, just says what you can do with the lawn somebody else owns, and it can have strings attached. > Thanks > Richard Knutsson Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 15:25 ` Richard Knutsson 2005-12-04 15:23 ` Arjan van de Ven 2005-12-05 23:51 ` Rob Landley @ 2005-12-06 20:40 ` Matan Peled 2 siblings, 0 replies; 639+ messages in thread From: Matan Peled @ 2005-12-06 20:40 UTC (permalink / raw) To: Richard Knutsson Cc: Matthias Andree, Arjan van de Ven, Linux-Kernel mailing list Richard Knutsson wrote: > But I do wonder how copyright and GPL can co-exist. Do the copyright > holder own the changes anybody else does to the code? > Anyone care to explain? IANAL, but GPL is a copyright license. Copyright is the right to make copies of somethings, to distribute it to be precise. So if I write foo.c and release it under the GPL, and JR Hacker takes it and writes foo++.c but doesn't give his super duper sekrit version to anyone, then he isn't bound by copyright laws (he isn't making copies) and therefore the GPL doesn't hold for him. The moment he wants to give a copy to his best friend, the GPL does kick in, though, and he has to abide by the GPL and distribute the whole piece (as it is a "derivative work") with the source code included (or an offer, or whatever. Read the GPL sometime, its not legalese at all). For more information, ask your friendly (*cough*) neighborhood lawyer. -- [Name ] :: [Matan I. Peled ] [Location ] :: [Israel ] [Public Key] :: [0xD6F42CA5 ] [Keyserver ] :: [keyserver.kjsl.com] encrypted/signed plain text preferred ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 0:52 ` Jeff V. Merkey 2005-12-04 12:12 ` Matthias Andree @ 2005-12-04 19:57 ` Horst von Brand 2005-12-04 21:35 ` Bernd Petrovitsch 2 siblings, 0 replies; 639+ messages in thread From: Horst von Brand @ 2005-12-04 19:57 UTC (permalink / raw) To: Jeff V. Merkey; +Cc: Matthias Andree, linux-kernel Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote: > Matthias Andree wrote: > >On Sat, 03 Dec 2005, Arjan van de Ven wrote: [...] > These folks have nothing new to innovate here. The memory manager and > VM gets revamped every other release. That is a sign of movement. Sure, one can argue if it is random movement of definite progress, but change /is/ a part of innovation. > Exports get broken, binary only > module compatibility busted every rev of the kernel. In-kernel API was /never/ guaranteed, so you have nothing to complain here. > I spend weeks on > each kernel fixing the breakage. These people don't get it, They do get it, and have their reasons to act this way. Reasons you don't get, it seems. > don't care, They do care. But not about the same thing you care about. > and to be honest, you are wasting your time here trying to convince > them. It is /their/ code, with which they are free to do as they wish. If you contribute enough, you get a say in it too; until that time, just be grateful for what you get given freely. > It's never stable because they don't want it to be. This is how > they maintain control of this code. Licensing under GPL while "maintaining control" is a contradiction in terms. If you don't like where the development is going, you are free to fork it. Just the developers won't follow you, as the consensus between them is against your wishes. > I have apps written for Windows in > 1990 and 1998 that still run on Windows XP today. I have programs written in the early 80s that still can be compiled and run on Linux. And AFAIK the very first a.out binaries for Linux still work (I'd assume that setting up the shared libraries for them is a royal pain). I have programs by /Microsoft/ which don't run after Win98, even though they are supposed to run on later versions. Besides, this backward compatibility is exactly one of the major reasons for Windows breakage, they just can't get a clean cut from the past. > Linux has no such > concept of backwards compatiblity. For user programs? Sure is. > Every company who has embraced it > outside of hardware based solutions is dying or has died. So? > IBM is secretly > forking it as we speak and using it to get out of paying for Unix > licenses. Could you please give some kind of evidence for this misbehaviour by IBM? > As annoying as it is, accept it and live with it. These people have no > sense of loyalty to you or your customers. They don't even care about > each other. Right. Their only loyalty is to a better kernel (system). Nothing else. Nobody /ever/ promised anything else, in fact, they have /always/ stated that that is their only objective. You definitely haven't gotten shafted. [...] > You are standing on a battlefield. Quietly fork each release, make your > mods, post patches somewhere for the poor civilians who are "collateral > damage" in the war with constantly busted software, and you might help > some folks. This is what distributions do routinely... and the current development model is precisely to /disminish/ the cost of keeping up to date by third parties. In that sense the kernel community /does/ care for them, just that they don't make any promises. Take it or leave it. There are alternatives, like the various BSD flavors. Or even Solaris, or closed Unix variants. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 0:52 ` Jeff V. Merkey 2005-12-04 12:12 ` Matthias Andree 2005-12-04 19:57 ` Horst von Brand @ 2005-12-04 21:35 ` Bernd Petrovitsch 2005-12-05 0:43 ` Jeff V. Merkey 2 siblings, 1 reply; 639+ messages in thread From: Bernd Petrovitsch @ 2005-12-04 21:35 UTC (permalink / raw) To: Jeff V. Merkey; +Cc: Matthias Andree, linux-kernel On Sat, 2005-12-03 at 17:52 -0700, Jeff V. Merkey wrote: [...] > of this code. I have apps written for Windows in 1990 and 1998 that ^^^^ > still run on Windows XP today. Linux has no such concept of But this not even holds for nearly all apps. > backwards compatiblity. Every company who has embraced it outside of The same holds (probably) for Linux apps (given that your kernel can start a.out). And AFAIBT by Win* driver developers even in the Win* world you have to change your driver because of a new Win* version now and then. Bernd -- Firmix Software GmbH http://www.firmix.at/ mobil: +43 664 4416156 fax: +43 1 7890849-55 Embedded Linux Development and Services ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 21:35 ` Bernd Petrovitsch @ 2005-12-05 0:43 ` Jeff V. Merkey 2005-12-05 9:06 ` Bernd Petrovitsch 0 siblings, 1 reply; 639+ messages in thread From: Jeff V. Merkey @ 2005-12-05 0:43 UTC (permalink / raw) To: Bernd Petrovitsch; +Cc: Matthias Andree, linux-kernel Bernd Petrovitsch wrote: >On Sat, 2005-12-03 at 17:52 -0700, Jeff V. Merkey wrote: >[...] > > >>of this code. I have apps written for Windows in 1990 and 1998 that >> >> > ^^^^ > > >>still run on Windows XP today. Linux has no such concept of >> >> > >But this not even holds for nearly all apps. > > > >>backwards compatiblity. Every company who has embraced it outside of >> >> > >The same holds (probably) for Linux apps (given that your kernel can >start a.out). And AFAIBT by Win* driver developers even in the Win* >world you have to change your driver because of a new Win* version now >and then. > > Bernd > > No. BIND was has been busted between 2.4 and 2.6. Not to mention the whole libc -> glib switchover. It's hilarious that BSD had to create a Linux app compat lib, and the RedHat shipped compat libs for 3 releases as well. Not even close. Windows has won. M$ has won. Linux lost the desktop wars and will soon loose the server wars as well. The reason - infighting and lack of backwards compatibility. Binary only module breakage kernel to kernel will continue. Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 0:43 ` Jeff V. Merkey @ 2005-12-05 9:06 ` Bernd Petrovitsch 2005-12-06 0:41 ` Horst von Brand 0 siblings, 1 reply; 639+ messages in thread From: Bernd Petrovitsch @ 2005-12-05 9:06 UTC (permalink / raw) To: Jeff V. Merkey; +Cc: Matthias Andree, linux-kernel [ Minimized quoted part ] On Sun, 2005-12-04 at 17:43 -0700, Jeff V. Merkey wrote: > Bernd Petrovitsch wrote: > >On Sat, 2005-12-03 at 17:52 -0700, Jeff V. Merkey wrote: [...] > >>of this code. I have apps written for Windows in 1990 and 1998 that > > ^^^^ > >>still run on Windows XP today. Linux has no such concept of > > > >But this not even holds for nearly all apps. > > > >>backwards compatiblity. Every company who has embraced it outside of > > > >The same holds (probably) for Linux apps (given that your kernel can > >start a.out). And AFAIBT by Win* driver developers even in the Win* > >world you have to change your driver because of a new Win* version now > >and then. [...] > No. BIND was has been busted between 2.4 and 2.6. Not to mention the Hmmm, URL? Details? I can't remember anything about such issues. > whole libc -> glib switchover. glib has AFAIK next to nothing to do with a libc AFAICT (i.e. it is using standard libc functions but that's all). > It's hilarious that BSD had to create a Linux app compat lib, and the > RedHat shipped compat libs for 3 releases Here you have your backwards compatibility. > as well. Not even close. Windows has won. M$ has won. Linux lost > the desktop wars and will soon loose > the server wars as well. The reason - infighting and lack of backwards Yes, probably - MSFT is spreading the same story since ages. > compatibility. Binary only module > breakage kernel to kernel will continue. As other told there never was a stable kernel module interface. Of course there is probably enough willing manpower out there who will work on that once you pay them. Or you can provide such support on your own. Or do you (or anybody else) has drivers which should be maintained for vanilla-kernel and/or vendor kernels and/or other kernels (to fix the breakage in a cosntructive way), we can provide you with an offer to do that. Bernd -- Firmix Software GmbH http://www.firmix.at/ mobil: +43 664 4416156 fax: +43 1 7890849-55 Embedded Linux Development and Services ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 9:06 ` Bernd Petrovitsch @ 2005-12-06 0:41 ` Horst von Brand 2005-12-06 9:38 ` Bernd Petrovitsch 0 siblings, 1 reply; 639+ messages in thread From: Horst von Brand @ 2005-12-06 0:41 UTC (permalink / raw) To: Bernd Petrovitsch; +Cc: Jeff V. Merkey, Matthias Andree, linux-kernel Bernd Petrovitsch <bernd@firmix.at> wrote: > [ Minimized quoted part ] > On Sun, 2005-12-04 at 17:43 -0700, Jeff V. Merkey wrote: > > Bernd Petrovitsch wrote: > > >On Sat, 2005-12-03 at 17:52 -0700, Jeff V. Merkey wrote: > [...] > > >>of this code. I have apps written for Windows in 1990 and 1998 that > > > ^^^^ > > >>still run on Windows XP today. Linux has no such concept of > > >But this not even holds for nearly all apps. > > >>backwards compatiblity. Every company who has embraced it outside of > > >The same holds (probably) for Linux apps (given that your kernel can > > >start a.out). And AFAIBT by Win* driver developers even in the Win* > > >world you have to change your driver because of a new Win* version now > > >and then. > [...] > > whole libc -> glib switchover. > glib has AFAIK next to nothing to do with a libc AFAICT (i.e. it is > using standard libc functions but that's all). He refers to the a.out to ELF switchover. Yes, it was painful. But not as much as he makes out. The Win98 --> WinNT change was worse, IMHO. > > It's hilarious that BSD had to create a Linux app compat lib, And Solaris forever had a BSD compatibility suite, including libraries and tools. So what? > > and the > > RedHat shipped compat libs for 3 releases So legacy stuff continued working. And that is bad how? > Here you have your backwards compatibility. Right. > > as well. Not even close. Windows has won. M$ has won. Linux lost > > the desktop wars First of all, Linux isn't about "winning a war". And the desktop wars haven't really started... > > and will soon loose the server wars as well. Sorry, but that one is almost over, and Linux has won. > > The > > reason - infighting and lack of backwards > Yes, probably - MSFT is spreading the same story since ages. Gandhi-con 3 ;-) > > compatibility. Binary only > > module breakage kernel to kernel will continue. So what? Binary modules are mostly bad and break the kernel, so... > As other told there never was a stable kernel module interface. Of > course there is probably enough willing manpower out there who will work > on that once you pay them. Or you can provide such support on your own. Right. > Or do you (or anybody else) has drivers which should be maintained for > vanilla-kernel and/or vendor kernels and/or other kernels (to fix the > breakage in a cosntructive way), we can provide you with an offer to do > that. Constructive criticism? Even of the sort that contributes something? What are you thinking about?! -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 0:41 ` Horst von Brand @ 2005-12-06 9:38 ` Bernd Petrovitsch 0 siblings, 0 replies; 639+ messages in thread From: Bernd Petrovitsch @ 2005-12-06 9:38 UTC (permalink / raw) To: Horst von Brand; +Cc: Jeff V. Merkey, Matthias Andree, linux-kernel On Mon, 2005-12-05 at 21:41 -0300, Horst von Brand wrote: > Bernd Petrovitsch <bernd@firmix.at> wrote: [...] [...] > > > whole libc -> glib switchover. Ah, that should have read "libc.5(*) to glibc" switchover"? (*) IIRC. > > glib has AFAIK next to nothing to do with a libc AFAICT (i.e. it is > > using standard libc functions but that's all). > > He refers to the a.out to ELF switchover. Yes, it was painful. But not as Was it? And it was ages ago (i can't even remember since when I disable a.out in the kernel completely and never had a problem with it). > much as he makes out. The Win98 --> WinNT change was worse, IMHO. Of course. Especially if you started to use the permission system and not let the NT installation stay in the default mode where every user may do everything everywhere (and they are hiding the contents of certain directories in the file browser instead of simply letting the administrator change it's contents so that folks really learn it). [....] > > > The > > > reason - infighting and lack of backwards > > > Yes, probably - MSFT is spreading the same story since ages. > > Gandhi-con 3 ;-) ???? Sorry, what do you mean? [...] > > As other told there never was a stable kernel module interface. Of > > course there is probably enough willing manpower out there who will work > > on that once you pay them. Or you can provide such support on your own. > > Right. > > > Or do you (or anybody else) has drivers which should be maintained for > > vanilla-kernel and/or vendor kernels and/or other kernels (to fix the > > breakage in a cosntructive way), we can provide you with an offer to do > > that. > > Constructive criticism? Even of the sort that contributes something? What No, since we interface here with the commercial world , it is a commercial offer (well, sort of - at least a an offer to provide an offer it the details and requirements are defined/clear). > are you thinking about?! $COMPANY wants a maintained "open" driver (probably GPL but that's not the point)? $COMPANY gives us money (a to be defined amount of money for a to be defined time, for to be defined distributions and/or kernel trees, to be defined QA with respect to the hardware driven by the driver, etc.) and we do that for you. Feel free to ignore it .... Bernd -- Firmix Software GmbH http://www.firmix.at/ mobil: +43 664 4416156 fax: +43 1 7890849-55 Embedded Linux Development and Services ^ permalink raw reply [flat|nested] 639+ messages in thread
* Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) 2005-12-03 15:23 ` Adrian Bunk 2005-12-03 15:39 ` Arjan van de Ven 2005-12-03 16:27 ` Matthias Andree @ 2005-12-03 18:43 ` Jeff Garzik 2005-12-03 20:34 ` Greg KH ` (2 more replies) 2005-12-05 3:23 ` RFC: Starting a stable kernel series off the 2.6 kernel Rob Landley 2005-12-10 19:48 ` Ryan Anderson 4 siblings, 3 replies; 639+ messages in thread From: Jeff Garzik @ 2005-12-03 18:43 UTC (permalink / raw) To: Adrian Bunk Cc: Arjan van de Ven, linux-kernel, Andrew Morton, Greg KH, James Bottomley Adrian Bunk wrote: > IOW, we should e.g. ensure that today's udev will still work flawlessly > with kernel 2.6.30 (sic)? > > This could work, but it should be officially announced that e.g. a > userspace running kernel 2.6.15 must work flawlessly with _any_ future > 2.6 kernel. Fix the real problem: publicly shame kernel hackers that change userland ABI/API without LOTS of notice, and hopefully an old-userland compatibility solution implemented. We change kernel APIs all the time. Having made that policy decision, we have the freedom to rapidly improve the kernel, and avoid being stuck with poor designs of the past. Userland isn't the same. IMO sysfs hackers have forgotten this. Anytime you change or remove sysfs attributes these days, you have the potential to break userland, which breaks one of the grand axioms of Linux. Everybody knows "the rules" when it comes to removing system calls, but forgets/ignores them when it comes to ioctls, sysfs attributes, and the like. Thus, I've often felt that heavy sysfs (and procfs) use made it too easy to break userland. Maybe we should change the sysfs API to include some sort of interface versioning, or otherwise make it more obvious to the programmer that they could be breaking userland compat. Offhand, once implemented and out in the field, I would say a userland interface should live at least 1-2 years after the "we are removing this interface" warning is given. Yes, 1-2 years. Maybe even that is too small. We still have old_mmap syscall around :) Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) 2005-12-03 18:43 ` Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) Jeff Garzik @ 2005-12-03 20:34 ` Greg KH 2005-12-04 1:40 ` Dmitry Torokhov 2005-12-03 22:35 ` Matthias Andree 2005-12-04 15:37 ` Michael Frank 2 siblings, 1 reply; 639+ messages in thread From: Greg KH @ 2005-12-03 20:34 UTC (permalink / raw) To: Jeff Garzik Cc: Adrian Bunk, Arjan van de Ven, linux-kernel, Andrew Morton, Greg KH, James Bottomley On Sat, Dec 03, 2005 at 01:43:48PM -0500, Jeff Garzik wrote: > > Userland isn't the same. IMO sysfs hackers have forgotten this. No we have not. > Anytime you change or remove sysfs attributes these days, you have the > potential to break userland, which breaks one of the grand axioms of > Linux. Everybody knows "the rules" when it comes to removing system > calls, but forgets/ignores them when it comes to ioctls, sysfs > attributes, and the like. The _main_ reason of making sysfs contain "one value per file" was to help mitigate the problems that proc has had in the past where file format changes broke userspace programs. Programs that rely on sysfs need to be able to safely handle the fact that some attributes might or might not be there. Now to be fair, yes, udev has had big problems with this in the past, and kernel updates have broken udev. But that was because the bug was MY fault in udev, not in the kernel. I'll take full responsibility for that. And in the future, the driver/class model changes we are going to be doing (see http://lwn.net/Articles/162242/ for more details on this), will be going to great lengths to prevent anything in userspace from breaking. So, to change your wording a bit, users of sysfs have forgotten how to program defensively to handle changes that might happen in the future. I know I'm guilty of this, and am working hard to not make the same mistakes in the future. thanks, greg "still trying to delete devfs after 1 and 1/2 years notice" k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) 2005-12-03 20:34 ` Greg KH @ 2005-12-04 1:40 ` Dmitry Torokhov 2005-12-04 23:29 ` Greg KH 0 siblings, 1 reply; 639+ messages in thread From: Dmitry Torokhov @ 2005-12-04 1:40 UTC (permalink / raw) To: Greg KH Cc: Jeff Garzik, Adrian Bunk, Arjan van de Ven, linux-kernel, Andrew Morton, Greg KH, James Bottomley On Saturday 03 December 2005 15:34, Greg KH wrote: > And in the future, the driver/class model changes we are going to be > doing (see http://lwn.net/Articles/162242/ for more details on this), > will be going to great lengths to prevent anything in userspace from > breaking. It is usually considered a bad netiquette to cross-post in public and subscription-only lists. I wonder if pointing to subscription-only service to get the feeling about planned driver core changes is a good idea. I would expect it be stated here or on hotplug list but don't recall anything interesting in the last couple of weeks. Is there a driver core mailing list I need to subscribe? -- Dmitry ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) 2005-12-04 1:40 ` Dmitry Torokhov @ 2005-12-04 23:29 ` Greg KH 2005-12-05 21:41 ` Dmitry Torokhov 0 siblings, 1 reply; 639+ messages in thread From: Greg KH @ 2005-12-04 23:29 UTC (permalink / raw) To: Dmitry Torokhov Cc: Jeff Garzik, Adrian Bunk, Arjan van de Ven, linux-kernel, Andrew Morton, Greg KH, James Bottomley On Sat, Dec 03, 2005 at 08:40:59PM -0500, Dmitry Torokhov wrote: > On Saturday 03 December 2005 15:34, Greg KH wrote: > > And in the future, the driver/class model changes we are going to be > > doing (see http://lwn.net/Articles/162242/ for more details on this), > > will be going to great lengths to prevent anything in userspace from > > breaking. > > It is usually considered a bad netiquette to cross-post in public and > subscription-only lists. I wonder if pointing to subscription-only > service to get the feeling about planned driver core changes is a good > idea. My apologies. It is merely a detailed description of what I wrote up here: http://www.kroah.com/log/linux/driver_model_changes.html I'll forward you a link to it off-list in a minute (and to anyone else who wants it.) After a week, lwn.net is free, so it will be public. thanks, greg k-h ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) 2005-12-04 23:29 ` Greg KH @ 2005-12-05 21:41 ` Dmitry Torokhov 0 siblings, 0 replies; 639+ messages in thread From: Dmitry Torokhov @ 2005-12-05 21:41 UTC (permalink / raw) To: Greg KH Cc: Jeff Garzik, Adrian Bunk, Arjan van de Ven, linux-kernel, Andrew Morton, Greg KH, James Bottomley On 12/4/05, Greg KH <greg@kroah.com> wrote: > On Sat, Dec 03, 2005 at 08:40:59PM -0500, Dmitry Torokhov wrote: > > On Saturday 03 December 2005 15:34, Greg KH wrote: > > > And in the future, the driver/class model changes we are going to be > > > doing (see http://lwn.net/Articles/162242/ for more details on this), > > > will be going to great lengths to prevent anything in userspace from > > > breaking. > > > > It is usually considered a bad netiquette to cross-post in public and > > subscription-only lists. I wonder if pointing to subscription-only > > service to get the feeling about planned driver core changes is a good > > idea. > > My apologies. It is merely a detailed description of what I wrote up > here: > http://www.kroah.com/log/linux/driver_model_changes.html > Ahh, I see. > I'll forward you a link to it off-list in a minute (and to anyone else > who wants it.) After a week, lwn.net is free, so it will be public. > That is allright, the link above is all I needed. I don't really want to use LWN "ahead of time" - they sell subsciptions - good for them. -- Dmitry ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) 2005-12-03 18:43 ` Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) Jeff Garzik 2005-12-03 20:34 ` Greg KH @ 2005-12-03 22:35 ` Matthias Andree 2005-12-04 15:37 ` Michael Frank 2 siblings, 0 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-03 22:35 UTC (permalink / raw) To: linux-kernel > Userland isn't the same. IMO sysfs hackers have forgotten this. > Anytime you change or remove sysfs attributes these days, you have the > potential to break userland, which breaks one of the grand axioms of > Linux. Everybody knows "the rules" when it comes to removing system > calls, but forgets/ignores them when it comes to ioctls, sysfs > attributes, and the like. procfs needs to be mentioned in a main clause (rather than parenthesized and in a subordinate clause), too. I don't recall if breakage happened at 2.6.0 or some other time, this however is pretty annoying, as much as the need to use unstable and undocumented 0.X version kernel support libraries or daemons utilities. One /proc example: The removal of /proc/scsi/$DRIVER/$CARD broke for instance the 3ware Escalade 6000/7000/8000 series 3DM tools - and the driver IS open source. OK, in this case it doesn't hurt because the 9000 series 3DM2 tools work, but it takes a while to figure *that* out. > Offhand, once implemented and out in the field, I would say a userland > interface should live at least 1-2 years after the "we are removing this > interface" warning is given. > > Yes, 1-2 years. Maybe even that is too small. We still have old_mmap > syscall around :) It should be "2 years or next kernel minor release, whatever comes later". I understand that to developers, keeping old gears and wheels around is an annoyance, but knowing one is in to support this for three years or so makes some people think twice about adding things or changing things in incompatible ways. And making people think twice can only improve quality. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) 2005-12-03 18:43 ` Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) Jeff Garzik 2005-12-03 20:34 ` Greg KH 2005-12-03 22:35 ` Matthias Andree @ 2005-12-04 15:37 ` Michael Frank 2 siblings, 0 replies; 639+ messages in thread From: Michael Frank @ 2005-12-04 15:37 UTC (permalink / raw) To: Jeff Garzik Cc: Adrian Bunk, Arjan van de Ven, linux-kernel, Andrew Morton, Greg KH, James Bottomley On Saturday 03 December 2005 19:43, Jeff Garzik wrote: > Adrian Bunk wrote: > > IOW, we should e.g. ensure that today's udev will still > > work flawlessly with kernel 2.6.30 (sic)? > > > > This could work, but it should be officially announced > > that e.g. a userspace running kernel 2.6.15 must work > > flawlessly with _any_ future 2.6 kernel. > > Fix the real problem: publicly shame kernel hackers that > change userland ABI/API without LOTS of notice, and > hopefully an old-userland compatibility solution > implemented. > > We change kernel APIs all the time. Having made that > policy decision, we have the freedom to rapidly improve > the kernel, and avoid being stuck with poor designs of > the past. > > Userland isn't the same. IMO sysfs hackers have > forgotten this. Anytime you change or remove sysfs > attributes these days, you have the potential to break > userland, which breaks one of the grand axioms of Linux. > Everybody knows "the rules" when it comes to removing > system calls, but forgets/ignores them when it comes to > ioctls, sysfs attributes, and the like. WRT sysfs, sysfs is dynamic by design to accommodate individual HW configuration. Thus isn't this really a fault of user-space implementation? > > Thus, I've often felt that heavy sysfs (and procfs) use > made it too easy to break userland. Maybe we should > change the sysfs API to include some sort of interface > versioning, or otherwise make it more obvious to the > programmer that they could be breaking userland compat. You might need versions for every entry. I'd go for more documentation on proper use. > > Offhand, once implemented and out in the field, I would > say a userland interface should live at least 1-2 years > after the "we are removing this interface" warning is > given. > > Yes, 1-2 years. Maybe even that is too small. We still > have old_mmap syscall around :) > > Jeff > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 15:23 ` Adrian Bunk ` (2 preceding siblings ...) 2005-12-03 18:43 ` Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) Jeff Garzik @ 2005-12-05 3:23 ` Rob Landley 2005-12-10 19:48 ` Ryan Anderson 4 siblings, 0 replies; 639+ messages in thread From: Rob Landley @ 2005-12-05 3:23 UTC (permalink / raw) To: Adrian Bunk; +Cc: Arjan van de Ven, linux-kernel [-- Attachment #1: Type: text/plain, Size: 2559 bytes --] On Saturday 03 December 2005 09:23, Adrian Bunk wrote: > IOW, we should e.g. ensure that today's udev will still work flawlessly > with kernel 2.6.30 (sic)? A) udev changing its interface every three months isn't the kernel's fault, it's udev's. Heck, udev doesn't even accept a dependency on an external libsys but instead bundles its own copy in there because obviously the proper way to use a shared library is to block copy it into the source tree of every potential user. Its config file format keeps changing. What commands you run to invoke it keeps changing. What does that have to do with the kernel? Attached is a shell script that does the basics of what udev does. (No, it doesn't handle permissions or persistent naming. But it also doesn't show a lot of version dependencies, does it?) B) When I install new packages I have to update dependencies like SDL or zlib all the time, yet you believe the kernel isn't allowed to have dependencies? Not even on things like modprobe or glibc that interface to the kernel not just graphically but "with tongue", as it were? (Despite that, they're pretty darn good at staying compatible anyway.) > This could work, but it should be officially announced that e.g. a > userspace running kernel 2.6.15 must work flawlessly with _any_ future > 2.6 kernel. Oh don't start throwing around "must" and "officially announced" as if those terms actually mean something. If you can come up with a compelling proposal and implement it and attract followers, fine. You don't need to grab a flag and get blessed by somebody else to do anything. (Whose flag and blessing did Linus get way back when? And where the heck did Ubuntu or Gentoo come from?) The _right_ way to do this would have been to announce that you are maintaining a new tree, a -stable fork of whatever release, and give a URL to where it can be downloaded. Announce code, not intentions. (Announcing intentions never works. Code attracts code and talk attracts talk.) And that way the difficulty and sustainability of your task becomes self-apparent. By the way, I'll guarantee you I can configure a 2.6.15 kernel that your userspace won't work with, with no code changes. (To start with, I'd yank elf and force you to use a.out executable format...) > For how many years do you think we will be able to ensure that this will > stay true? Who is "we", kemosabe? > cu > Adrian Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. [-- Attachment #2: setupdev.sh --] [-- Type: application/x-shellscript, Size: 413 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 15:23 ` Adrian Bunk ` (3 preceding siblings ...) 2005-12-05 3:23 ` RFC: Starting a stable kernel series off the 2.6 kernel Rob Landley @ 2005-12-10 19:48 ` Ryan Anderson 4 siblings, 0 replies; 639+ messages in thread From: Ryan Anderson @ 2005-12-10 19:48 UTC (permalink / raw) To: Adrian Bunk; +Cc: Arjan van de Ven, linux-kernel On Sat, Dec 03, 2005 at 04:23:39PM +0100, Adrian Bunk wrote: > On Sat, Dec 03, 2005 at 03:36:38PM +0100, Arjan van de Ven wrote: > > > If the current model doesn't work as you claim it doesn't, then maybe > > the model needs finetuning. Right now the biggest pain is the userland > > ABI changes that need new packages; sometimes (often) for no real hard > > reason. Maybe we should just stop doing those bits, they're not in any > > fundamental way blocking general progress (sure there's some code bloat > > due to it, but I guess we'll just have to live with that). > > IOW, we should e.g. ensure that today's udev will still work flawlessly > with kernel 2.6.30 (sic)? > > This could work, but it should be officially announced that e.g. a > userspace running kernel 2.6.15 must work flawlessly with _any_ future > 2.6 kernel. > > For how many years do you think we will be able to ensure that this will > stay true? I'd rather see the statement that if a kernel 2.6.N requires a userspace update (udev, alsa, whatever), that the new userspace works correctly on 2.6.(N-10). I think that is the bit of the problem that has been really frustrating to the people that have run into it. (I think that is the complaint Dave Jones made during his OLS keynote, and I've seen a similar complaint about udev, though the udev issue may have been Debian specific.) -- Ryan Anderson sometimes Pug Majere ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 13:56 RFC: Starting a stable kernel series off the 2.6 kernel Adrian Bunk ` (2 preceding siblings ...) 2005-12-03 14:36 ` Arjan van de Ven @ 2005-12-03 18:39 ` Dr. David Alan Gilbert 2005-12-03 20:59 ` Lars Marowsky-Bree ` (3 subsequent siblings) 7 siblings, 0 replies; 639+ messages in thread From: Dr. David Alan Gilbert @ 2005-12-03 18:39 UTC (permalink / raw) To: Adrian Bunk; +Cc: linux-kernel Hi Adrian, I would really appreciate such a move to a stable series. I've had some really bad luck with instability of 2.6.x - in particular with NFS. Would such a stable kernel keep up to date on basic drivers? I ask since I got into a messy situation on a series of production servers; they were really new Dell servers using standard Intel chipsets but needed SATA stuff that went in recently. Does 2.6.16 have the basic infrastructure for all the current hardware so that if you branch now you aren't going to have to do any really heavy backports to be able to run on 'current' hardware? I hit the situation where I have a 2.6.5 kernel I use on everything else and whose NFS works fine; and 2.6.11 or newer which supports the hardware - but whose NFS is giving me broken locking to some obscure systems. IMHO we've also got into a real mess where it is vendor kernels that have stability fixes in for many things (NFS in particular) - but the lkml doesn't want to know about vendor kernels, but at the same time they aren't up for stabilisation. Good luck with such a branch! Dave -- -----Open up your eyes, open up your mind, open up your code ------- / Dr. David Alan Gilbert | Running GNU/Linux on Alpha,68K| Happy \ \ gro.gilbert @ treblig.org | MIPS,x86,ARM,SPARC,PPC & HPPA | In Hex / \ _________________________|_____ http://www.treblig.org |_______/ ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 13:56 RFC: Starting a stable kernel series off the 2.6 kernel Adrian Bunk ` (3 preceding siblings ...) 2005-12-03 18:39 ` Dr. David Alan Gilbert @ 2005-12-03 20:59 ` Lars Marowsky-Bree 2005-12-03 21:13 ` Dave Jones 2005-12-06 0:14 ` Florian Weimer 2005-12-04 12:56 ` Indrek Kruusa ` (2 subsequent siblings) 7 siblings, 2 replies; 639+ messages in thread From: Lars Marowsky-Bree @ 2005-12-03 20:59 UTC (permalink / raw) To: linux-kernel On 2005-12-03T14:56:08, Adrian Bunk <bunk@stusta.de> wrote: > The current kernel development model is pretty good for people who > always want to use or offer their costumers the maximum amount of the > latest bugs^Wfeatures without having to resort on additional patches for > them. > > Problems of the current development model from a user's point of view > are: > - many regressions in every new release > - kernel updates often require updates for the kernel-related userspace > (e.g. for udev or the pcmcia tools switch) Your problem statement is correct, but you're fixing the symptoms, not the cause. What we need is an easier way for users to pull in kernel updates with the matching kernel-related user-space. This is provided, though with some lag, by Fedora, openSUSE, Debian testing, dare I say gentoo and others. The right way to address this is to work with the distribution of your choice to make these updates available faster. Sincerely, Lars Marowsky-Brée <lmb@suse.de> -- High Availability & Clustering SUSE Labs, Research and Development SUSE LINUX Products GmbH - A Novell Business -- Charles Darwin "Ignorance more frequently begets confidence than does knowledge" ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 20:59 ` Lars Marowsky-Bree @ 2005-12-03 21:13 ` Dave Jones 2005-12-03 21:18 ` Lars Marowsky-Bree 2005-12-03 23:02 ` Matthias Andree 2005-12-06 0:14 ` Florian Weimer 1 sibling, 2 replies; 639+ messages in thread From: Dave Jones @ 2005-12-03 21:13 UTC (permalink / raw) To: Lars Marowsky-Bree; +Cc: linux-kernel On Sat, Dec 03, 2005 at 09:59:11PM +0100, Lars Marowsky-Bree wrote: > On 2005-12-03T14:56:08, Adrian Bunk <bunk@stusta.de> wrote: > > > The current kernel development model is pretty good for people who > > always want to use or offer their costumers the maximum amount of the > > latest bugs^Wfeatures without having to resort on additional patches for > > them. > > > > Problems of the current development model from a user's point of view > > are: > > - many regressions in every new release > > - kernel updates often require updates for the kernel-related userspace > > (e.g. for udev or the pcmcia tools switch) > > Your problem statement is correct, but you're fixing the symptoms, not > the cause. > > What we need is an easier way for users to pull in kernel updates with > the matching kernel-related user-space. > > This is provided, though with some lag, by Fedora, openSUSE, Debian > testing, dare I say gentoo and others. > > The right way to address this is to work with the distribution of your > choice to make these updates available faster. The big problem is though that we don't typically find out that we've regressed until after a kernel update is in the end-users hands. In many cases, submitters of changes know that things are going to break. Maybe we need a policy that says changes requiring userspace updates need to be clearly documented in the mails Linus gets (Especially if its a git pull request), so that when the next point release gets released, Linus can put a section in the announcement detailing what bits of userspace are needed to be updated. It still isn't to solve the problem of regressions in drivers, but that's a problem that's not easily solvable. Dave ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 21:13 ` Dave Jones @ 2005-12-03 21:18 ` Lars Marowsky-Bree 2005-12-03 21:42 ` [OT] " Michael Buesch 2005-12-03 22:40 ` Adrian Bunk 2005-12-03 23:02 ` Matthias Andree 1 sibling, 2 replies; 639+ messages in thread From: Lars Marowsky-Bree @ 2005-12-03 21:18 UTC (permalink / raw) To: Dave Jones, linux-kernel On 2005-12-03T16:13:29, Dave Jones <davej@redhat.com> wrote: > The big problem is though that we don't typically find out that > we've regressed until after a kernel update is in the end-users hands. > > In many cases, submitters of changes know that things are going > to break. Maybe we need a policy that says changes requiring userspace updates > need to be clearly documented in the mails Linus gets (Especially if its > a git pull request), so that when the next point release gets released, > Linus can put a section in the announcement detailing what bits > of userspace are needed to be updated. True, but this first block doesn't really qualify as a "regression". Yes, a clearer-than-crystal documentation of "this kernel requires user-space component foo to be at least x.y.z if feature bar is used" would go a long way. And if then user-space itself was tolerant of at least version N and N-1, then users could even roll back one kernel version if problems arise. Both of these are documentation and user-space issues, and don't much depend on changes to kernel development model. > It still isn't to solve the problem of regressions in drivers, but > that's a problem that's not easily solvable. True. Regressions will always occur when driver updates happen. There'll always be the next bug. I don't think anyone introduces these on purpose ;-) Sincerely, Lars Marowsky-Brée <lmb@suse.de> -- High Availability & Clustering SUSE Labs, Research and Development SUSE LINUX Products GmbH - A Novell Business -- Charles Darwin "Ignorance more frequently begets confidence than does knowledge" ^ permalink raw reply [flat|nested] 639+ messages in thread
* [OT] Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 21:18 ` Lars Marowsky-Bree @ 2005-12-03 21:42 ` Michael Buesch 2005-12-03 22:40 ` Adrian Bunk 1 sibling, 0 replies; 639+ messages in thread From: Michael Buesch @ 2005-12-03 21:42 UTC (permalink / raw) To: Lars Marowsky-Bree, linux-kernel [-- Attachment #1: Type: text/plain, Size: 605 bytes --] On Saturday 03 December 2005 22:18, you wrote: > > It still isn't to solve the problem of regressions in drivers, but > > that's a problem that's not easily solvable. > > True. Regressions will always occur when driver updates happen. There'll > always be the next bug. I don't think anyone introduces these on purpose > ;-) From the dmesg of the openwrt project: [snip] 802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com> All bugs added by David S. Miller <davem@redhat.com> VFS: Mounted root (squashfs filesystem) readonly. [snap] Smash him. :D -- Greetings Michael. [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 21:18 ` Lars Marowsky-Bree 2005-12-03 21:42 ` [OT] " Michael Buesch @ 2005-12-03 22:40 ` Adrian Bunk 1 sibling, 0 replies; 639+ messages in thread From: Adrian Bunk @ 2005-12-03 22:40 UTC (permalink / raw) To: Lars Marowsky-Bree; +Cc: Dave Jones, linux-kernel On Sat, Dec 03, 2005 at 10:18:10PM +0100, Lars Marowsky-Bree wrote: > On 2005-12-03T16:13:29, Dave Jones <davej@redhat.com> wrote: > > > The big problem is though that we don't typically find out that > > we've regressed until after a kernel update is in the end-users hands. > > > > In many cases, submitters of changes know that things are going > > to break. Maybe we need a policy that says changes requiring userspace updates > > need to be clearly documented in the mails Linus gets (Especially if its > > a git pull request), so that when the next point release gets released, > > Linus can put a section in the announcement detailing what bits > > of userspace are needed to be updated. > > True, but this first block doesn't really qualify as a "regression". > Yes, a clearer-than-crystal documentation of "this kernel requires > user-space component foo to be at least x.y.z if feature bar is used" > would go a long way. > > And if then user-space itself was tolerant of at least version N and > N-1, then users could even roll back one kernel version if problems > arise. > > Both of these are documentation and user-space issues, and don't much > depend on changes to kernel development model. >... One effect that you mustn't underestimate is that if people had to got to N-1 for two or three different N due to different regressions in different kernels, they might decide to stay with kernel version M even though kernel M+10 might have been released and version M lacks tons of security fixes. > Sincerely, > Lars Marowsky-Brée <lmb@suse.de> cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 21:13 ` Dave Jones 2005-12-03 21:18 ` Lars Marowsky-Bree @ 2005-12-03 23:02 ` Matthias Andree 2005-12-03 23:09 ` Dave Jones 1 sibling, 1 reply; 639+ messages in thread From: Matthias Andree @ 2005-12-03 23:02 UTC (permalink / raw) To: linux-kernel; +Cc: Dave Jones, Lars Marowsky-Bree On Sat, 03 Dec 2005, Dave Jones wrote: > In many cases, submitters of changes know that things are going > to break. Maybe we need a policy that says changes requiring userspace updates > need to be clearly documented in the mails Linus gets (Especially if its > a git pull request), so that when the next point release gets released, > Linus can put a section in the announcement detailing what bits > of userspace are needed to be updated. This isn't acceptable in stable kernels. FreeBSD has a very tight policy, newer kernels off the same branch support older user space. The upgrade path is clear, reboot into new kernel, have it spit a few reminders that your userspace needs update (Linux also has this, for instance, with SG_IO and its predecessors) but still everything works. Requiring new userspace at a patchlevel kernel upgrade is nothing but impertinent, unless that updated userspace ships as part of the kernel. > It still isn't to solve the problem of regressions in drivers, but > that's a problem that's not easily solvable. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 23:02 ` Matthias Andree @ 2005-12-03 23:09 ` Dave Jones 0 siblings, 0 replies; 639+ messages in thread From: Dave Jones @ 2005-12-03 23:09 UTC (permalink / raw) To: linux-kernel, Lars Marowsky-Bree On Sun, Dec 04, 2005 at 12:02:54AM +0100, Matthias Andree wrote: > On Sat, 03 Dec 2005, Dave Jones wrote: > > > In many cases, submitters of changes know that things are going > > to break. Maybe we need a policy that says changes requiring userspace updates > > need to be clearly documented in the mails Linus gets (Especially if its > > a git pull request), so that when the next point release gets released, > > Linus can put a section in the announcement detailing what bits > > of userspace are needed to be updated. > > This isn't acceptable in stable kernels. FreeBSD has a very tight > policy, newer kernels off the same branch support older user space. The BSDs have an advantage in that their userspace & kernels are closely coupled. When kernel changes need a userspace change, it gets done at the same time in the same repository. Dave ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 20:59 ` Lars Marowsky-Bree 2005-12-03 21:13 ` Dave Jones @ 2005-12-06 0:14 ` Florian Weimer 2005-12-06 13:20 ` Lars Marowsky-Bree 1 sibling, 1 reply; 639+ messages in thread From: Florian Weimer @ 2005-12-06 0:14 UTC (permalink / raw) To: Lars Marowsky-Bree; +Cc: linux-kernel * Lars Marowsky-Bree: > The right way to address this is to work with the distribution of your > choice to make these updates available faster. Working with a distribution benefits that distribution alone. Working on (e.g.) kernel security advisories would benefit everyone. It's not a speed issue, it's more about coverage. And full coverage is very hard to get without support from the real developers. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 0:14 ` Florian Weimer @ 2005-12-06 13:20 ` Lars Marowsky-Bree 2005-12-06 13:37 ` Florian Weimer 0 siblings, 1 reply; 639+ messages in thread From: Lars Marowsky-Bree @ 2005-12-06 13:20 UTC (permalink / raw) To: Florian Weimer; +Cc: linux-kernel On 2005-12-06T01:14:23, Florian Weimer <fw@deneb.enyo.de> wrote: > > The right way to address this is to work with the distribution of your > > choice to make these updates available faster. > Working with a distribution benefits that distribution alone. Working > on (e.g.) kernel security advisories would benefit everyone. It's not > a speed issue, it's more about coverage. And full coverage is very > hard to get without support from the real developers. The distributions differ from another in their sync and branch points from the main kernel, and there's no way before hell freezes over this is going to change. So, you essentially need to maintain the kernel your distribution branched from. This means: backport fixes/features relevant to your release, and make sure the rest of the system stays in sync. This puts the effort there where it belongs: to those people benefitting from it. The current model actually works _better_ for the existing distributions, because they get to choose their branchpoint - with all the features up to that point - instead of having, say, 2.6.x as the stable base and then already starting out with having to backport major features from 2.7 (because of user demand). A single stable branch beneficial to all users means frozen innovation for the distributions, and they still have to significant QA on the releases and the updates to that kernel (to stay on that issue, it applies to other major components too). Even with 2.4.x, a distribution couldn't simply stick in newer 2.4.x+n releases instead of 2.4.x, because, as someone already so well said, one users bugfix is another's regression. And all the distributors would have to agree on the same policy for kernel changes and sync even updates! Thus, more effort for less gain. The truth is that right now we have _several_ stable branches maintained by the distributors (be they commercial or free) together with the kernel-related user-land. I daresay this is a feature and works pretty well. If someone wants to maintain a stable 2.6.x release, nobody will stop them from maintaining 2.6.x.y until y overflows, or until the 6 months are full and then they can release their new major update and plot a transition path with the updates to all required user-land. The fact people are complaining about stems from the fact that the Linux kernel itself is useless; it is intimately tied to various components which reside in user-space, and so it is inherent to the process that a major kernel update very likely maps to a distribution update. The components are developed separately, but they do not have a stable modular interface, at essentially no level but the POSIX/system call interfaces and sometimes, glibc or what is specified in the LSB. This is a _feature_! It allows us to more quickly move and adapt. The BSD model is, as Dave pointed out, even further along this road, and every distribution basically does exactly that, because our user community is big enough to sustain it. Sincerely, Lars Marowsky-Brée -- High Availability & Clustering SUSE Labs, Research and Development SUSE LINUX Products GmbH - A Novell Business -- Charles Darwin "Ignorance more frequently begets confidence than does knowledge" ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 13:20 ` Lars Marowsky-Bree @ 2005-12-06 13:37 ` Florian Weimer 0 siblings, 0 replies; 639+ messages in thread From: Florian Weimer @ 2005-12-06 13:37 UTC (permalink / raw) To: Lars Marowsky-Bree; +Cc: linux-kernel * Lars Marowsky-Bree: > On 2005-12-06T01:14:23, Florian Weimer <fw@deneb.enyo.de> wrote: > >> > The right way to address this is to work with the distribution of your >> > choice to make these updates available faster. >> Working with a distribution benefits that distribution alone. Working >> on (e.g.) kernel security advisories would benefit everyone. It's not >> a speed issue, it's more about coverage. And full coverage is very >> hard to get without support from the real developers. > > The distributions differ from another in their sync and branch points > from the main kernel, and there's no way before hell freezes over this > is going to change. > > So, you essentially need to maintain the kernel your distribution > branched from. This means: backport fixes/features relevant to your > release, and make sure the rest of the system stays in sync. This puts > the effort there where it belongs: to those people benefitting from it. Lars, please read again what I wrote. It's not about branch maintenance ("here's the patch"), it's about providing basic information which can guide branch maintenance ("here's an issue you need to look at if you use code from the IPv6 stack in version 2.6.10 to 2.6.12"). ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 13:56 RFC: Starting a stable kernel series off the 2.6 kernel Adrian Bunk ` (4 preceding siblings ...) 2005-12-03 20:59 ` Lars Marowsky-Bree @ 2005-12-04 12:56 ` Indrek Kruusa 2005-12-04 13:05 ` Arjan van de Ven 2005-12-05 23:43 ` Rob Landley 2005-12-05 19:30 ` Bill Davidsen 2005-12-12 14:45 ` Felix Oxley 7 siblings, 2 replies; 639+ messages in thread From: Indrek Kruusa @ 2005-12-04 12:56 UTC (permalink / raw) To: Adrian Bunk; +Cc: linux-kernel >These problems follow from the development model. > > Hi! I am not evil or unsatisfied end-user by any means, it is just quite interesting to follow today's lkml posts. After reading "Linux 2.6.15-rc5: off-line for a week" from Torvalds it seems like this:. a) Torvalds thinks that nobody cares about kernel testing b) other gurus (they are also only "on-line" testers nowadays) doesn't feel good with development model (or at least they have no resources to do testing [Torvalds]) c) end-users (or those who are not kernel maintainers) are directed permanently to distros kernels and "stay away from kernel.org you wanna-bees!" I can't say is it good or bad but this is interesting at least. I hope you guys keep going! Have a nice day! Indrek ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 12:56 ` Indrek Kruusa @ 2005-12-04 13:05 ` Arjan van de Ven 2005-12-04 15:26 ` Indrek Kruusa 2005-12-05 23:43 ` Rob Landley 1 sibling, 1 reply; 639+ messages in thread From: Arjan van de Ven @ 2005-12-04 13:05 UTC (permalink / raw) To: Indrek Kruusa; +Cc: Adrian Bunk, linux-kernel > nt model (or at least they have no resources to > do testing [Torvalds]) > c) end-users (or those who are not kernel maintainers) are directed > permanently to distros kernels and "stay away from kernel.org you > wanna-bees! this is not what is being said. What is being said is that if you can't deal with occasional breakage, you're better off using vendor kernels. But.. if you can't deal with occasional breakage, you wouldn't test test kernels EITHER. If you can deal with an occasional breakage, I hope you and everyone else who can, will run and test kernel.org kernels, especially the -rc ones. Most of the "instability" people complain about with the new 2.6 model is caused by people not testing the -rc kernels before they are released, so that they end up being released with regressions. But... that will happen no matter what model you use actually. Before july there was a problem where -rc kernels kept changing bigtime, so it was hard to know which one to test if you only had time to test one, but that is now changed... Maybe it's a bit extremist, but I'd like to even state it like this: "If you can't be bothered to test -rc kernels, you have no right to complain about the final ones", sort of as analogy to the "if you don't go voting you have no right to complain about the politicians". ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 13:05 ` Arjan van de Ven @ 2005-12-04 15:26 ` Indrek Kruusa 0 siblings, 0 replies; 639+ messages in thread From: Indrek Kruusa @ 2005-12-04 15:26 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Adrian Bunk, linux-kernel Arjan van de Ven wrote: >>nt model (or at least they have no resources to >>do testing [Torvalds]) >>c) end-users (or those who are not kernel maintainers) are directed >>permanently to distros kernels and "stay away from kernel.org you >>wanna-bees! >> >> > >this is not what is being said. What is being said is that if you can't >deal with occasional breakage, you're better off using vendor kernels. >But.. if you can't deal with occasional breakage, you wouldn't test test >kernels EITHER. If you can deal with an occasional breakage, I hope you >and everyone else who can, will run and test kernel.org kernels, >especially the -rc ones. > >Most of the "instability" people complain about with the new 2.6 model >is caused by people not testing the -rc kernels before they are >released, so that they end up being released with regressions. > I think I have seen special live-cd distribution for KDE beta testers. Kernel is not a KDE but such a very broken distribution with -rc kernel could be more easily maintained than "udev forever!". Live-cd (or live-usb) wouldn't be too flexible - you can't say there "can you give a whirl to this patch, please" but I bet you will have more testers. thanks, Indrek ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-04 12:56 ` Indrek Kruusa 2005-12-04 13:05 ` Arjan van de Ven @ 2005-12-05 23:43 ` Rob Landley 1 sibling, 0 replies; 639+ messages in thread From: Rob Landley @ 2005-12-05 23:43 UTC (permalink / raw) To: Indrek Kruusa; +Cc: Adrian Bunk, linux-kernel On Sunday 04 December 2005 06:56, Indrek Kruusa wrote: > After reading "Linux 2.6.15-rc5: off-line for a week" from Torvalds it > seems like this:. > > a) Torvalds thinks that nobody cares about kernel testing > b) other gurus (they are also only "on-line" testers nowadays) doesn't > feel good with development model (or at least they have no resources to > do testing [Torvalds]) > c) end-users (or those who are not kernel maintainers) are directed > permanently to distros kernels and "stay away from kernel.org you > wanna-bees!" Yeah, normally this would mean it's a tuesday. We're running a little early. Rob -- Steve Ballmer: Innovation! Inigo Montoya: You keep using that word. I do not think it means what you think it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 13:56 RFC: Starting a stable kernel series off the 2.6 kernel Adrian Bunk ` (5 preceding siblings ...) 2005-12-04 12:56 ` Indrek Kruusa @ 2005-12-05 19:30 ` Bill Davidsen 2005-12-05 23:25 ` Adrian Bunk ` (2 more replies) 2005-12-12 14:45 ` Felix Oxley 7 siblings, 3 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-05 19:30 UTC (permalink / raw) To: Adrian Bunk, Linux Kernel Mailing List Adrian Bunk wrote: > The current kernel development model is pretty good for people who > always want to use or offer their costumers the maximum amount of the > latest bugs^Wfeatures without having to resort on additional patches for > them. > > Problems of the current development model from a user's point of view > are: > - many regressions in every new release > - kernel updates often require updates for the kernel-related userspace > (e.g. for udev or the pcmcia tools switch) > > One problem following from this is that people continue to use older > kernels with known security holes because the amount of work for kernel > upgrades is too high. Depending on where you work, "not working" may be acceptable vs. "working with a known security hole." > > These problems follow from the development model. > > The latest stable kernel series without these problems is 2.4, but 2.4 > is becoming more and more obsolete and might e.g. lack driver support > for some recent hardware you want to use. > > Since Andrew and Linus do AFAIK not plan to change the development > model, what about the following for getting a stable kernel series > without leaving the current development model: > > > Kernel 2.6.16 will be the base for a stable series. > > After 2.6.16, there will be a 2.6.16.y series with the usual stable > rules. > > After the release of 2.6.17, this 2.6.16.y series will be continued with > more relaxed rules similar to the rules in kernel 2.4 since the release > of kernel 2.6.0 (e.g. driver updates will be allowed). Actually I would be happy with the stability of this series if people would stop trying to take working features OUT of it! That's the largest problem I see, not that the existing features are unstable, and we have a -stable branch to cover that, but that I can't count on features I use and which are required for useful work. If a firm policy of not removing supported features until 2.7 was adopted I don't see a problem. The bulk of the instability (not absolutely all, I grant), is in new features, or features which aren't working all that well in any case. But if existing features suddenly drop out from beneath the user, then you will find people doing what you mentioned, staying with old kernels with holes rather than moving to kernels which are simply no longer functional. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 19:30 ` Bill Davidsen @ 2005-12-05 23:25 ` Adrian Bunk 2005-12-06 11:28 ` Matthias Andree 2005-12-06 13:25 ` Lars Marowsky-Bree 2 siblings, 0 replies; 639+ messages in thread From: Adrian Bunk @ 2005-12-05 23:25 UTC (permalink / raw) To: Bill Davidsen; +Cc: Linux Kernel Mailing List On Mon, Dec 05, 2005 at 02:30:09PM -0500, Bill Davidsen wrote: > > Actually I would be happy with the stability of this series if people > would stop trying to take working features OUT of it! That's the largest > problem I see, not that the existing features are unstable, and we have > a -stable branch to cover that, but that I can't count on features I use > and which are required for useful work. > > If a firm policy of not removing supported features until 2.7 was > adopted I don't see a problem. The bulk of the instability (not > absolutely all, I grant), is in new features, or features which aren't > working all that well in any case. But if existing features suddenly > drop out from beneath the user, then you will find people doing what you > mentioned, staying with old kernels with holes rather than moving to > kernels which are simply no longer functional. You are thinking in terms of the old development model. This is not an option since the current development model says that there might never be a 2.7 kernel series. We might like it or not, but this is the current development model and Andrew and Linus don't seem to want to change it. cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 19:30 ` Bill Davidsen 2005-12-05 23:25 ` Adrian Bunk @ 2005-12-06 11:28 ` Matthias Andree 2005-12-06 13:25 ` Lars Marowsky-Bree 2 siblings, 0 replies; 639+ messages in thread From: Matthias Andree @ 2005-12-06 11:28 UTC (permalink / raw) To: Linux Kernel Mailing List On Mon, 05 Dec 2005, Bill Davidsen wrote: > If a firm policy of not removing supported features until 2.7 was > adopted I don't see a problem. The bulk of the instability (not I do - the problem is someone will let it bit-rot for a few releases and then declare it broken. Remember ATAPI CD writing vs. DMA? -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-05 19:30 ` Bill Davidsen 2005-12-05 23:25 ` Adrian Bunk 2005-12-06 11:28 ` Matthias Andree @ 2005-12-06 13:25 ` Lars Marowsky-Bree 2005-12-06 20:46 ` Bill Davidsen 2 siblings, 1 reply; 639+ messages in thread From: Lars Marowsky-Bree @ 2005-12-06 13:25 UTC (permalink / raw) To: Bill Davidsen, Adrian Bunk, Linux Kernel Mailing List On 2005-12-05T14:30:09, Bill Davidsen <davidsen@tmr.com> wrote: > Actually I would be happy with the stability of this series if people > would stop trying to take working features OUT of it! Features are removed when they are no longer features, but design irritations in a new and improved design, and usually, equivalent or better (or at least thought to be) functionality is available still in the big picture (which includes user-space), hopefully in a cleaner place. Now, design is often a holy war, and people disagree. That's fine and to be expected. And sometimes, the whole solution takes a while to materialize and be implemented from the kernel up to all user-space and even longer until it has been implemented in the brains of the admins. This, too, is fine and expected. It's called "innovation" and "development", sometimes iterative. > working all that well in any case. But if existing features suddenly > drop out from beneath the user, then you will find people doing what you > mentioned, staying with old kernels with holes rather than moving to > kernels which are simply no longer functional. You're assuming the kernel is both "static" design-wise as well as independent (or at least basically eternally backwards compatible) from user-space. Both assumptions are no longer true. Get over it. Sincerely, Lars Marowsky-Brée -- High Availability & Clustering SUSE Labs, Research and Development SUSE LINUX Products GmbH - A Novell Business -- Charles Darwin "Ignorance more frequently begets confidence than does knowledge" ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-06 13:25 ` Lars Marowsky-Bree @ 2005-12-06 20:46 ` Bill Davidsen 0 siblings, 0 replies; 639+ messages in thread From: Bill Davidsen @ 2005-12-06 20:46 UTC (permalink / raw) To: Lars Marowsky-Bree, Linux Kernel Mailing List Lars Marowsky-Bree wrote: > On 2005-12-05T14:30:09, Bill Davidsen <davidsen@tmr.com> wrote: > > >>Actually I would be happy with the stability of this series if people >>would stop trying to take working features OUT of it! > > > Features are removed when they are no longer features, but design > irritations in a new and improved design, and usually, equivalent or > better (or at least thought to be) functionality is available still in > the big picture (which includes user-space), hopefully in a cleaner > place. > > Now, design is often a holy war, and people disagree. That's fine and to > be expected. And sometimes, the whole solution takes a while to > materialize and be implemented from the kernel up to all user-space and > even longer until it has been implemented in the brains of the admins. > This, too, is fine and expected. It's called "innovation" and > "development", sometimes iterative. Removing features because there are better solutions is one thing, although it has been done at kernel tree changes for a decade. Removing features for reasons of religion is rather a case of developers removing a useful and unbroken feature for which there is no replacement purely because someone doesn't like it, or it saves a dozen lines of code. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-03 13:56 RFC: Starting a stable kernel series off the 2.6 kernel Adrian Bunk ` (6 preceding siblings ...) 2005-12-05 19:30 ` Bill Davidsen @ 2005-12-12 14:45 ` Felix Oxley 2005-12-12 17:17 ` Horst von Brand 7 siblings, 1 reply; 639+ messages in thread From: Felix Oxley @ 2005-12-12 14:45 UTC (permalink / raw) To: Adrian Bunk; +Cc: linux-kernel On 3 Dec 2005, at 13:56, Adrian Bunk wrote: > The current kernel development model is pretty good for people who > always want to use or offer their costumers the maximum amount of the > latest bugs^Wfeatures without having to resort on additional > patches for > them. > > Problems of the current development model from a user's point of view > are: > - many regressions in every new release > - kernel updates often require updates for the kernel-related > userspace > (e.g. for udev or the pcmcia tools switch) > > One problem following from this is that people continue to use older > kernels with known security holes because the amount of work for > kernel > upgrades is too high. > > These problems follow from the development model. > > The latest stable kernel series without these problems is 2.4, but 2.4 > is becoming more and more obsolete and might e.g. lack driver support > for some recent hardware you want to use. > > Since Andrew and Linus do AFAIK not plan to change the development > model, what about the following for getting a stable kernel series > without leaving the current development model: > > > Kernel 2.6.16 will be the base for a stable series. > > After 2.6.16, there will be a 2.6.16.y series with the usual stable > rules. > > After the release of 2.6.17, this 2.6.16.y series will be continued > with > more relaxed rules similar to the rules in kernel 2.4 since the > release > of kernel 2.6.0 (e.g. driver updates will be allowed). > > > Q: > What is the target audience for this 2.6.16 series? > > A: > The target audience are users still using 2.4 (or who'd still use > kernel > 2.4 if they weren't forced to upgrade to 2.6 for some reason) who > want a > stable kernel series including security fixes but excluding many > regressions. > It might also be interesting for distributions that prefer stability > over always using the latest stuff. > > > Q: > Does this proposal imply anything for the development between > 2.6.15 and > 2.6.16? > > A: > In theory not. > In practice, it would be a big advantage if some of the bigger > changes that might go into 2.6.16 would be postponed to 2.6.17. > > > Q: > Why not start with the more relaxed rules before the release of > 2.6.17? > > A: > After 2.6.16.y following the usual stable rules, the kernel should be > relatively stable and well-tested giving the best possible basis for a > long-living series. > > > Q: > How long should this 2.6.16 series be maintained? > > A: > Time will tell, but if people use it I'd expect 2 or 3 years. > > > Q: > Stable API/ABI for external modules? > > A: > No. > > > Q: > Who will maintain this branch? > > A: > I could do it, but if someone more experienced wants to do it that > would > be even better. > - > To unsubscribe from this list: send the line "unsubscribe linux- > kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ What if ... 1. When people make a patch set, if they have encountered any 'bugs' they split them out as separate items. 2. The submitter would identify through GIT when the error had been introduced so that the the person responsible could be CC'ed, also anybody who had worked on the code recently would be CCed, therefore the programmers who were most familiar with that section of code would be made aware of it. 3. When the patch is posted to LKML, it is tagged [PATCH][FIX] in the subject line. In the body of the fix would be noted each kernel to which the fix applied e.g [FIX 2.6.11][FIX 2.6.12][FIX 2.6.13][FIX 2.6.14] 4. The programmers mentioned in (2) would ACK the patch which would then become part of an 'official' fixes list. 5. If a volunteer wanted to maintain, say, 2.6.14 + fixes, they could build and test it and be a point of contact regarding any problems. These could hopefully be tracked down and submitted as a new fix patch. regards, Felix ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-12 14:45 ` Felix Oxley @ 2005-12-12 17:17 ` Horst von Brand 2005-12-12 18:53 ` Felix Oxley 0 siblings, 1 reply; 639+ messages in thread From: Horst von Brand @ 2005-12-12 17:17 UTC (permalink / raw) To: Felix Oxley; +Cc: Adrian Bunk, linux-kernel Felix Oxley <lkml@oxley.org> wrote: [...] > What if ... > > 1. When people make a patch set, if they have encountered any 'bugs' > they split them out as separate items. No need. Patches are either (a) bug fixes, or (b) infrastructure changes, or (c) additions (mostly drivers). You only need to pick (a) items. Check. > 2. The submitter would identify through GIT when the error had been > introduced Hard to find out. Nobody will do so. > so that the the person responsible could be CC'ed, also > anybody who had worked on the code recently would be CCed, therefore > the programmers who were most familiar with that section of code > would be made aware of it. Cc:ing them is part of the development anyway (in reality, Cc:ing anybody interested in the area). Check. > 3. When the patch is posted to LKML, it is tagged [PATCH][FIX] in the > subject line. > In the body of the fix would be noted each kernel to which the > fix applied e.g [FIX 2.6.11][FIX 2.6.12][FIX 2.6.13][FIX 2.6.14] No do. Problem are the (b) and (c) patches above, they change whatever the patch applies to and make it not apply anymore. The effort of finding out if the patch is (a) or (c) class, seeing if it is really needed, and modifying it so it applies to your source base is called "backporting". And it remains hard, thankless work. > 4. The programmers mentioned in (2) would ACK the patch which would > then become part of an 'official' fixes list. Won't happen. > 5. If a volunteer wanted to maintain, say, 2.6.14 + fixes, they could > build and test it and be a point of contact regarding any problems. > These could hopefully be tracked down and submitted as a new fix patch. Go right ahead. Just be warned that distributions hired a small army of kernel specialists to do exactly this, and got tired of doing so. Among others because the patches deemed necessary were different from one distributuion to the next, and then usually incompatible with one another and with what turned out to be the standard solution. This gave rise to the current development model... Armchair software engineering is much like armchair $SPORT. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-12 17:17 ` Horst von Brand @ 2005-12-12 18:53 ` Felix Oxley 2005-12-13 13:17 ` Horst von Brand 0 siblings, 1 reply; 639+ messages in thread From: Felix Oxley @ 2005-12-12 18:53 UTC (permalink / raw) To: Horst von Brand; +Cc: Adrian Bunk, linux-kernel On 12 Dec 2005, at 17:17, Horst von Brand wrote: > Felix Oxley <lkml@oxley.org> wrote: > > [...] > >> What if ... >> >> 1. When people make a patch set, if they have encountered any 'bugs' >> they split them out as separate items. > > No need. Patches are either (a) bug fixes, or (b) infrastructure > changes, > or (c) additions (mostly drivers). You only need to pick (a) items. > Check. > >> 2. The submitter would identify through GIT when the error had been >> introduced > > Hard to find out. Nobody will do so. > >> so that the the person responsible could be CC'ed, also >> anybody who had worked on the code recently would be CCed, therefore >> the programmers who were most familiar with that section of code >> would be made aware of it. > > Cc:ing them is part of the development anyway (in reality, Cc:ing > anybody > interested in the area). Check. > >> 3. When the patch is posted to LKML, it is tagged [PATCH][FIX] in the >> subject line. >> In the body of the fix would be noted each kernel to which the >> fix applied e.g [FIX 2.6.11][FIX 2.6.12][FIX 2.6.13][FIX 2.6.14] > > No do. Problem are the (b) and (c) patches above, they change > whatever the > patch applies to and make it not apply anymore. The effort of > finding out > if the patch is (a) or (c) class, seeing if it is really needed, and > modifying it so it applies to your source base is called > "backporting". And > it remains hard, thankless work. If this was done for 'trivial' patches of type (a): 1. Would that make it simple enough for people to actually do it? 2. Would it be worthwhile? (Are there enough 'trivial fixes'?) I envisaged something like the current Stable series, just for longer than a single release cycle. >> 4. The programmers mentioned in (2) would ACK the patch which would >> then become part of an 'official' fixes list. > > Won't happen. > >> 5. If a volunteer wanted to maintain, say, 2.6.14 + fixes, they could >> build and test it and be a point of contact regarding any problems. >> These could hopefully be tracked down and submitted as a new fix >> patch. > > Go right ahead. Just be warned that distributions hired a small > army of > kernel specialists to do exactly this, and got tired of doing so. > Among > others because the patches deemed necessary were different from one > distributuion to the next, and then usually incompatible with one > another > and with what turned out to be the standard solution. This gave > rise to the > current development model... > > Armchair software engineering is much like armchair $SPORT. I am guilty :-) Thanks for your reply. regards, Felix ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-12 18:53 ` Felix Oxley @ 2005-12-13 13:17 ` Horst von Brand 2005-12-14 0:09 ` Felix Oxley 0 siblings, 1 reply; 639+ messages in thread From: Horst von Brand @ 2005-12-13 13:17 UTC (permalink / raw) To: Felix Oxley; +Cc: Horst von Brand, Adrian Bunk, linux-kernel Felix Oxley <lkml@oxley.org> wrote: > On 12 Dec 2005, at 17:17, Horst von Brand wrote: > > Felix Oxley <lkml@oxley.org> wrote: > > > > [...] > > > >> What if ... > >> > >> 1. When people make a patch set, if they have encountered any 'bugs' > >> they split them out as separate items. > > No need. Patches are either (a) bug fixes, or (b) infrastructure > > changes, or (c) additions (mostly drivers). You only need to pick (a) > > items. Check. [...] > >> 3. When the patch is posted to LKML, it is tagged [PATCH][FIX] in the > >> subject line. > >> In the body of the fix would be noted each kernel to which the > >> fix applied e.g [FIX 2.6.11][FIX 2.6.12][FIX 2.6.13][FIX 2.6.14] > > No do. Problem are the (b) and (c) patches above, they change > > whatever the patch applies to and make it not apply anymore. The effort > > of finding out if the patch is (a) or (c) class, seeing if it is really > > needed, and modifying it so it applies to your source base is called > > "backporting". And it remains hard, thankless work. > > If this was done for 'trivial' patches of type (a): > 1. Would that make it simple enough for people to actually do it? > 2. Would it be worthwhile? (Are there enough 'trivial fixes'?) Not all important fixes are "trivial", far from it; so this is rather suspect in any case. Changes to the underlying source make even "trivial" patches soon not apply anymore. And there still is the job of finding out if some patch is or is not necessary... > I envisaged something like the current Stable series, just for longer > than a single release cycle. Go right ahead. If enough people get interested and work on it, it might turn out useful. I rather doubt it, as the current development model is exactly geared towards keeping people up to date, not running ancient kernels and then jumping a few versions ahead. The problem with doing that is that instead of one problem at a time you see a dozen, and then it is hard to pin down /when/ it broke (and thus what change is responsible). Plus the drift from backported patches, where you can't be sure it /seemed/ to work because of some random patch. Again, this development model was tried /hard/ for some 12 years by the distributions, and found sorely lacking (and essentially unfixable). -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: RFC: Starting a stable kernel series off the 2.6 kernel 2005-12-13 13:17 ` Horst von Brand @ 2005-12-14 0:09 ` Felix Oxley 0 siblings, 0 replies; 639+ messages in thread From: Felix Oxley @ 2005-12-14 0:09 UTC (permalink / raw) To: Horst von Brand; +Cc: Adrian Bunk, linux-kernel On 13 Dec 2005, at 13:17, Horst von Brand wrote: > Felix Oxley <lkml@oxley.org> wrote: >> On 12 Dec 2005, at 17:17, Horst von Brand wrote: >>> Felix Oxley <lkml@oxley.org> wrote: >>> >>> [...] >>> >>>> What if ... >>>> >>>> 1. When people make a patch set, if they have encountered any >>>> 'bugs' >>>> they split them out as separate items. > >>> No need. Patches are either (a) bug fixes, or (b) infrastructure >>> changes, or (c) additions (mostly drivers). You only need to pick >>> (a) >>> items. Check. > > [...] > >>>> 3. When the patch is posted to LKML, it is tagged [PATCH][FIX] >>>> in the >>>> subject line. >>>> In the body of the fix would be noted each kernel to which the >>>> fix applied e.g [FIX 2.6.11][FIX 2.6.12][FIX 2.6.13][FIX >>>> 2.6.14] > >>> No do. Problem are the (b) and (c) patches above, they change >>> whatever the patch applies to and make it not apply anymore. The >>> effort >>> of finding out if the patch is (a) or (c) class, seeing if it is >>> really >>> needed, and modifying it so it applies to your source base is called >>> "backporting". And it remains hard, thankless work. >> >> If this was done for 'trivial' patches of type (a): >> 1. Would that make it simple enough for people to actually do it? >> 2. Would it be worthwhile? (Are there enough 'trivial fixes'?) > > Not all important fixes are "trivial", far from it; so this is rather > suspect in any case. Changes to the underlying source make even > "trivial" > patches soon not apply anymore. And there still is the job of > finding out > if some patch is or is not necessary... > >> I envisaged something like the current Stable series, just for longer >> than a single release cycle. > > Go right ahead. If enough people get interested and work on it, it > might > turn out useful. I rather doubt it, as the current development > model is > exactly geared towards keeping people up to date, not running ancient > kernels and then jumping a few versions ahead. The problem with > doing that > is that instead of one problem at a time you see a dozen, and then > it is > hard to pin down /when/ it broke (and thus what change is > responsible). > Plus the drift from backported patches, where you can't be sure it / > seemed/ > to work because of some random patch. > > Again, this development model was tried /hard/ for some 12 years by > the > distributions, and found sorely lacking (and essentially unfixable). Thank you for your explanation. I will retire to lurk quietly in my armchair. :-) regards, Felix ^ permalink raw reply [flat|nested] 639+ messages in thread
* the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion @ 2006-07-21 19:46 Hans Reiser 2006-07-22 0:11 ` Losing Technologists [was: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Neil Brown ` (3 more replies) 0 siblings, 4 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-21 19:46 UTC (permalink / raw) To: LKML Is http://wiki.kernelnewbies.org/WhyReiser4IsNotIn truly the "official" point of view as claimed by its author? An interesting method of expression for it. I heard about it from a user who suggested that I respond before it got slashdotted. Let me ask that one compare and contrast the ext4 integration procedure outlined by Ted Tso, with the procedure experienced by other filesystems. The code isn't even written, benchmarked, or tested yet, and it is going into the kernel already so that its developers don't have to deal with maintaining patches separate from the tree. Wow. Kind of hard to argue that it is not politically differentiated, isn't it? Consider what happened with XFS as the article writer mentions. I met the original XFS team, led by two very senior developers (Jim Grey, and another fellow whose name I am blanking on, forgive me, I learned much from him in just a few conversations). They were kind enough to instruct me on what ideas I should take from XFS, and you know what, I listened. Reiser4's allocate on flush is the result of their kind instruction. I then took it a bit further, like a good student, and Reiser4 also has balance on flush, compress on flush, etc. These guys wanted to port XFS to Linux, but there was a problem, which was that IRIX was better in some ways than Linux, and XFS depended on those advantages. Now I met them, and I have to tell you that it was pretty obvious that these guys knew what they were doing. Suggest that these guys needed supervision --- sorry, no way, we needed their supervision. What happened? They got hassled. Instead of learning from them, welcoming into our community two very senior developers who knew a lot more than any of us about the topics they chose to speak about, they got hassled, they get ignored, they felt rejected, and left the Linux community forever, never to return. XFS is still with us, but the loss of those two could only have been devastating for their project. I think the whole kernel community suffered from their loss. Linux has a problem, which is that with success it is attracting people with more skill than what it started with, and it is not doing a very good job of handling that. In fact, it downright stinks at it, behaving in the worst way it could choose for handling that. We have lost quite a number of FS developers who just don't want to deal with people who know less than they do but are obnoxious and disrespectful to submissions because they enjoy powertripping. We lost David Mazieres, for example, because he is very very bright, is one of DARPA's most promising security researchers, and he does not want to be bothered with Viro et al. so he develops for BSD instead. Linus, if you really want to prove that Linux welcomes talented people, go sweet talk Mazieres into giving Linux another try, you might succeed if you try. The odd thing is that Viro is not obnoxious at all in person. lkml suffers from email disease, and we need to make conscious efforts to reduce that. Regarding distros accepting filesystems first, that is just completely backwards from what it ought to be. Linus, I respect you a lot, but I know this one is your idea, and some things I disagree with you on. Distros are marketed towards people who do not know how to tar and untar if an FS is dropped. A reasonable approach would be to say that any filesystem marked as experimental can be dropped at any time, so if you aren't able to tar and untar the partition it is on when a new kernel comes out, you should not use experimental filesystems. Then most distros will not make the experimental FS visible to users who don't press three buttons acknowledging that they were warned.... Linspire's view is pretty simple, they need to know that Reiser4 will be accepted BEFORE they make their distro depend on it. This is being responsible to their users. I could go ask Debian, etc., to include Reiser4, but it is the wrong way, so I am shy about it. I am not saying that ext4 should not be accepted as an experimental FS, I don't even really believe that ext4 should only be accepted when it is higher performance than Reiser4, I am saying that the process should be the same for everyone. Reiser4 is the upgrade for ReiserFS V3, in which we fix all of V3's flaws without disturbing the mission critical servers using V3 by changing the V3 code underneath them. (Things like the bug affecting MythTV users on V3 at the moment just should not be happening. Experiments belong in V4, and I wish there was more respect for my views on this.). V4 contains bug fixes for several V3 bugs that are too deep to fix without deep rewrite, and since V3 does not have plugins, disk format changes should not get added to a stable branch. When submitted Reiser4 was more stable than V3 was when it was accepted. (This is because we now have a much better test suite. I would never submit code that I know has a bug unfixed. At the moment we can crash Reiser4 using our test suite, as some of the linux kernel inclusion related changes made recently were extensive, I hope we have that bug fixed by next week.) We should develop a culture in which acceptance is more based on whose code measurably performs well than on who is friends with whom. We should not think that such a culture will develop without an effort being made to grow it. Actually, if we just had a few more akpms to go around, things would be a lot better..... oh well. We need to recruit more people like him. You can't really have code reviewed by persons less experienced and proven than those being reviewed, it just doesn't work too well. Linux needs to look outward more, and welcome persons who have proven themselves in other arenas as though we were lucky to get their time. Because we are. Maybe when we don't have people with the expertise to review something, we should go outside the Linux community, like the way academic journals will solicit outside reviewers for particular articles as they need them. Our current attitude resembles that of BSD before it lost the market to Linux, I remember it well, there was a reason why I developed for Linux instead. Avoiding the problems that some large corporations have with politics does not happen automatically as a result of it being free software, it requires as much effort as it does in the successful large corporations. Non-profits are in no way immune to being harmed by internal politics. If it is true that Reiser4 is likely to go in for 2.6.19, this is good to hear, though an odd source to hear it from. If it is true, then I will skip lobbying distros to accept Reiser4 before the kernel does, because really it makes little sense for them to do so. Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Losing Technologists [was: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-21 19:46 the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Hans Reiser @ 2006-07-22 0:11 ` Neil Brown 2006-07-22 0:18 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Adrian Bunk ` (2 subsequent siblings) 3 siblings, 0 replies; 639+ messages in thread From: Neil Brown @ 2006-07-22 0:11 UTC (permalink / raw) To: Hans Reiser; +Cc: LKML Hi Hans, On Friday July 21, reiser@namesys.com wrote: > > Linux has a problem, which is that with success it is attracting people > with more skill than what it started with, and it is not doing a very > good job of handling that. You seem to be saying that technological skills are the most important thing for Linux. I disagree. Linux is a success because of a great community as much (probably more) than because it has great technology. And by "great community", I don't mean "completely harmonious" but rather "board and effective". Good community members follow the rule "Be liberal in what you accept, conservative in what you transmit". And the first if these is the more important. Not everyone in the community is perfect at this, but it is clear to me that those who excel at this are what makes the community work. > We have lost quite > a number of FS developers who just don't want to deal with people who > know less than they do but are obnoxious and disrespectful to > submissions because they enjoy powertripping. This just tells me that they wouldn't make good community members - they are not sufficiently liberal in what they accept. It really isn't that hard to filter out the rubbish and look for the genuine technical content. For some people there isn't any of the later so you just ignore them. For others there is real technical content in what they say and you can respond to that while ignoring the rubbish. > lkml suffers from > email disease, and we need to make conscious efforts to reduce that. Yes. But as this is a free and open community, we cannot exclude or censor people. The only emails that we can directly control are our own. We should expect the general standard of the community to be below the standard we present, and aim accordingly. I see this happening. It works. The community isn't perfect, but it could be a lot worse. > Actually, if we just had a few more akpms to go around, things would be > a lot better..... oh well. We need to recruit more people like > him. Yes, and no. Certainly a few more akpms would improve the technology, and would allow our current akpm to catch up with his workload. But I don't think a few more of anything would dramatically change the community. There will always be people who attract flames, and others who are too willing to provide them. New members to the community really need to sit back for a while and figure out who is who. Who is worth listening to. Who is best ignored (except for entertainment value). Who to ask questions of, etc. It really is worth the effort. There is plenty of historical evidence that the best technology doesn't always win out - there are plenty of other factors to success. So don't be too concerned if we lose a few brilliant technologists - it isn't the end of the world (domination). NeilBrown ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-21 19:46 the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Hans Reiser 2006-07-22 0:11 ` Losing Technologists [was: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Neil Brown @ 2006-07-22 0:18 ` Adrian Bunk 2006-07-22 19:19 ` Diego Calleja 2006-07-22 13:02 ` Theodore Tso 2006-07-24 22:17 ` Paul Jackson 3 siblings, 1 reply; 639+ messages in thread From: Adrian Bunk @ 2006-07-22 0:18 UTC (permalink / raw) To: Hans Reiser; +Cc: LKML, Diego Calleja On Fri, Jul 21, 2006 at 01:46:18PM -0600, Hans Reiser wrote: > Is http://wiki.kernelnewbies.org/WhyReiser4IsNotIn truly the "official" > point of view as claimed by its author? An interesting method of > expression for it. I heard about it from a user who suggested that I > respond before it got slashdotted. After two users asked the "Why is Reiser4 still not included?" question within a very short amount of time on this list (mailing lists aren't write-only, and this issue has been discussed often before...), Diego wrote an FAQ for this issue. Emails by people like Linus, Andrew, Christoph or Al are what comes nearest to an official statement. If there are factual errors or things you consider offensive in the FAQ, please try to discuss them with Diego privately first. But changing contents of this FAQ wouldn't change anything regarding the Reiser4 inclusion - the FAQ is only a high level description of the Reiser4 situation for end users. > Let me ask that one compare and contrast the ext4 integration procedure > outlined by Ted Tso, with the procedure experienced by other > filesystems. The code isn't even written, benchmarked, or tested yet, > and it is going into the kernel already so that its developers don't > have to deal with maintaining patches separate from the tree. Wow. > Kind of hard to argue that it is not politically differentiated, isn't it? The main difference seems to be that ext4 is being developed by people that have already shown and are trusted to develop a filesystem that follows the Linux way in all respects. Note that I didn't say "right way" but "Linux way". No matter whether it's coding style or the discussion which functionality belongs to the VFS level, the Linux kernel has it's (often unwritten) rules - but the same is true with other rules for any other operating system. >... > Actually, if we just had a few more akpms to go around, things would be > a lot better..... oh well. We need to recruit more people like him. > You can't really have code reviewed by persons less experienced and > proven than those being reviewed, it just doesn't work too well. Linux > needs to look outward more, and welcome persons who have proven > themselves in other arenas as though we were lucky to get their time. > Because we are. Maybe when we don't have people with the expertise to > review something, we should go outside the Linux community, like the way > academic journals will solicit outside reviewers for particular articles > as they need them. Our current attitude resembles that of BSD before it > lost the market to Linux, I remember it well, there was a reason why I > developed for Linux instead. A very important part of a review is whether it follows the "Linux way" that might be quite different from what someone who comes from outwards has to learn before he can start doing a review. Finding people for the cool stuff like developing a new filesystem is relatively easy compared to finding people why are both capable and willing to do the boring work of reviewing other people's code. So we'd need people who are already acknowleged experts in the area, who are willing to learn the Linux way, and who will then do the not-fun work of reviewing other people's code. How should this work? Someone has to offer well-paid jobs for such people? > Avoiding the problems that some large corporations have with politics > does not happen automatically as a result of it being free software, it > requires as much effort as it does in the successful large > corporations. Non-profits are in no way immune to being harmed by > internal politics. In my experience, the Linux kernel is a big open source project with a working structure. The Linux kernel sometimes looses developers. That seems to unavoidable, there are there are always problems like: - Some developers leave the projects if their code was rejected because it didn't match the standards and policies of the project. - Some developers leave the project if other people's code that didn't match the standards and policies of the project was accepted. And it's not as if open source projects had in any respect better prerequisites than large corporations - open source lacks the "it's our job" glue forcing people to work together in large corporations. >... > Hans cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-22 0:18 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Adrian Bunk @ 2006-07-22 19:19 ` Diego Calleja 0 siblings, 0 replies; 639+ messages in thread From: Diego Calleja @ 2006-07-22 19:19 UTC (permalink / raw) To: Adrian Bunk; +Cc: reiser, linux-kernel El Sat, 22 Jul 2006 02:18:19 +0200, Adrian Bunk <bunk@stusta.de> escribió: > After two users asked the "Why is Reiser4 still not included?" question > within a very short amount of time on this list (mailing lists aren't > write-only, and this issue has been discussed often before...), Diego > wrote an FAQ for this issue. > > Emails by people like Linus, Andrew, Christoph or Al are what comes > nearest to an official statement. [I'm the guy who wrote the doc] I didn't write "It could possibly be ready as soon as 2.6.19" literally, that was someone that reworked my (ugly) english. Being fair what make me wrote that pages was the huge amount of people FUDing about linux developers in online forums (including this list) > > You can't really have code reviewed by persons less experienced and > > proven than those being reviewed, it just doesn't work too well. Linux Hans, stop that "we're smarter than you" attitude. It's not surprising that reiser 4 creates so many flames and that it's getting so hard to make progress, with such strong arguments. As far as I can tell, most of the kernel hackers trust Al Viro and hch on their opinions, specially in fs-related issues - and for good reasons. They know linux and they know how a linux fs must behave. As long as a fs plays well with the rest of the system and the code doesn't suck, I doubt they care too much if it's very advanced or arcane, and the huge variety of filesystem available in linux confirms that. > > as they need them. Our current attitude resembles that of BSD before it > > lost the market to Linux, I remember it well, there was a reason why I > > developed for Linux instead. Hans, I don't agree. If anything, the problem is that right now there's not a "development" stage: people just takes more care about what goes in, it wouldn't happen the same under a development stage. That certainly could make the job of big projects like reiser 4 much harder. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-21 19:46 the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Hans Reiser 2006-07-22 0:11 ` Losing Technologists [was: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Neil Brown 2006-07-22 0:18 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Adrian Bunk @ 2006-07-22 13:02 ` Theodore Tso 2006-07-22 14:30 ` Eric Sandeen ` (3 more replies) 2006-07-24 22:17 ` Paul Jackson 3 siblings, 4 replies; 639+ messages in thread From: Theodore Tso @ 2006-07-22 13:02 UTC (permalink / raw) To: Hans Reiser; +Cc: LKML On Fri, Jul 21, 2006 at 01:46:18PM -0600, Hans Reiser wrote: > Let me ask that one compare and contrast the ext4 integration procedure > outlined by Ted Tso "integration procedure" is hardly an accurate description, rather it is a development procedure that was developed after discussion and consensus building across LKML and the ext2/3/4 development team. It was not the original plan put forth by the ext2 developers, but after listening to the concerns and suggestions, we did not question the motives of the people making suggestions; we listened. > The code isn't even written, benchmarked, or tested yet, Actually, the first bits that we plan to merge have already been written and in use by hundreds of clusterfs customers, posted to LKML for comments (and we don't attack our reviewers, we thank them for their comments), and in fact they were written about at last year's OLS complete with benchmarks and graphs. (http://ext2.sourceforge.net/2005-ols/2005-ols-ext3.html) > Consider what happened with XFS as the article writer mentions. I met > the original XFS team, led by two very senior developers (Jim Grey, and > another fellow whose name I am blanking on, forgive me, I learned much > from him in just a few conversations). I believe you are referring to Jim Mostek and Steve Lord, and yes, they were very talented developers and engineers. I very much enjoyed talking to them at various filesystem and Linux conferences and workshops. > supervision. What happened? They got hassled. Instead of learning > from them, welcoming into our community two very senior developers who > knew a lot more than any of us about the topics they chose to speak > about, they got hassled, they get ignored, they felt rejected, and left > the Linux community forever, never to return. That's hardly what happened. SGI went through layoffs, and they were hit. See: http://slashdot.org/articles/01/05/26/0743254.shtml > A reasonable approach would be to say that any > filesystem marked as experimental can be dropped at any time, so if you > aren't able to tar and untar the partition it is on when a new kernel > comes out, you should not use experimental filesystems. Then most > distros will not make the experimental FS visible to users who don't > press three buttons acknowledging that they were warned.... Linspire's > view is pretty simple, they need to know that Reiser4 will be accepted > BEFORE they make their distro depend on it. You do realize these two statements are completely contradictory, don't you? If an experimental filesystem can be dropped at any time, then Linspire can't depend on it from the point of view of supporting their users. If there is a huge user base, then someone is going to have to maintain it, even if the developer community has moved on to supporting the next new exciting filesystem thing. Hence, it is critical that the resulting filesystem be fully maintainable before it is integrated. To put it in your words, it wouldn't be responsible to the user base to do otherwise. - Ted ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-22 13:02 ` Theodore Tso @ 2006-07-22 14:30 ` Eric Sandeen 2006-07-22 18:33 ` Hans Reiser ` (2 subsequent siblings) 3 siblings, 0 replies; 639+ messages in thread From: Eric Sandeen @ 2006-07-22 14:30 UTC (permalink / raw) To: Theodore Tso, Hans Reiser, LKML Theodore Tso wrote: > On Fri, Jul 21, 2006 at 01:46:18PM -0600, Hans Reiser wrote: >> Consider what happened with XFS as the article writer mentions. I met >> the original XFS team, led by two very senior developers (Jim Grey, and >> another fellow whose name I am blanking on, forgive me, I learned much >> from him in just a few conversations). > > I believe you are referring to Jim Mostek and Steve Lord, and yes, > they were very talented developers and engineers. I very much enjoyed > talking to them at various filesystem and Linux conferences and > workshops. > >> supervision. What happened? They got hassled. Instead of learning >> from them, welcoming into our community two very senior developers who >> knew a lot more than any of us about the topics they chose to speak >> about, they got hassled, they get ignored, they felt rejected, and left >> the Linux community forever, never to return. > > That's hardly what happened. SGI went through layoffs, and they were > hit. See: http://slashdot.org/articles/01/05/26/0743254.shtml Jim & Steve were never laid off from SGI. SGI mgmt -was- smarter than that ;) Neither account above is 100% correct, but I won't speak further on behalf of these gentlemen... -Eric ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-22 13:02 ` Theodore Tso 2006-07-22 14:30 ` Eric Sandeen @ 2006-07-22 18:33 ` Hans Reiser 2006-07-22 20:29 ` Jeff Garzik 2006-07-23 7:20 ` Rene Rebe 2006-07-24 2:08 ` Steve Lord 3 siblings, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-07-22 18:33 UTC (permalink / raw) To: Theodore Tso; +Cc: LKML Theodore Tso wrote: > >Actually, the first bits > yes, the first bits.... other people send in completed filesystems.... > that we plan to merge > I don't actually think that your merge approach is the wrong one, I think that it being exclusive to you is what is wrong. > > > >>Consider what happened with XFS as the article writer mentions. I met >>the original XFS team, led by two very senior developers (Jim Grey, and >>another fellow whose name I am blanking on, forgive me, I learned much >>from him in just a few conversations). >> >> > >I believe you are referring to Jim Mostek > Ah, Jim Mostek and Jim Gray. (Steve Lord was not a senior guy back then, and he is still with SGI last I heard.... I actually don't know Steve very well, hmm, maybe some future conference....) Thanks. >That's hardly what happened. SGI went through layoffs, and they were >hit. See: http://slashdot.org/articles/01/05/26/0743254.shtml > > As the other poster mentioned, they went off to startups, and did not become part of our community. How much of that was because their contributions were more hassled than welcomed, I cannot say with certainty, I can only say that they were discouraged by the difficulty of getting their stuff in, and this was not as it should have been. They were more knowledgeable than we were on the topics they spoke on, and this was not recognized and acknowledged. Outsiders are not respected by the kernel community. This means we miss a lot. > > >>A reasonable approach would be to say that any >>filesystem marked as experimental can be dropped at any time, so if you >>aren't able to tar and untar the partition it is on when a new kernel >>comes out, you should not use experimental filesystems. Then most >>distros will not make the experimental FS visible to users who don't >>press three buttons acknowledging that they were warned.... Linspire's >>view is pretty simple, they need to know that Reiser4 will be accepted >>BEFORE they make their distro depend on it. >> >> > >You do realize these two statements are completely contradictory, >don't you? > No, because distros would wait until it is not experimental before giving it to their users by default, in my proposed release model. lkml is populated with people FAR more suited to experimenting with experimental filesystems than typical distro customer lists are. It is commercial and political reasons that motivate distros being the first with patches not tried yet by lkml, not the interests of the users. Now, for other patches these commercial and political reasons may need to be catered to as the price of getting the Redhats of the world to fund kernel development, but that logic does not apply to Reiser4's particulars. Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-22 18:33 ` Hans Reiser @ 2006-07-22 20:29 ` Jeff Garzik 2006-07-23 7:20 ` Hans Reiser 2006-07-24 8:41 ` Matthias Andree 0 siblings, 2 replies; 639+ messages in thread From: Jeff Garzik @ 2006-07-22 20:29 UTC (permalink / raw) To: Hans Reiser; +Cc: Theodore Tso, LKML Hans Reiser wrote: > Theodore Tso wrote: > >> Actually, the first bits >> > yes, the first bits.... other people send in completed filesystems.... Completed filesystems have a much higher barrier to entry, because they require a fresh review. ext4 will go upstream MUCH faster, because it follows the standard process of Linux evolution, building on top of existing code with progressive changes: cp -a ext3 ext4 update ext4 update ext4 update ext4 ... This process builds upon existing reviews and knowledge of existing code. This process also guarantees a higher degree of stability during development, because the interim changes must always form a complete, working, usable filesystem. > As the other poster mentioned, they went off to startups, and did not > become part of our community. How much of that was because their > contributions were more hassled than welcomed, I cannot say with > certainty, I can only say that they were discouraged by the difficulty > of getting their stuff in, and this was not as it should have been. > They were more knowledgeable than we were on the topics they spoke on, > and this was not recognized and acknowledged. > > Outsiders are not respected by the kernel community. This means we miss > a lot. Anyone who fails to respect the kernel development process, the process of building consensus, is turn not respected, flamed, and/or ignored. If you don't respect us, why should we respect you? > No, because distros would wait until it is not experimental before > giving it to their users by default, in my proposed release model. lkml Distros follow their own release model, and don't have a care about what Hans Reiser thinks they should do. <vendor hat on> Red Hat has a pipeline in place for offering new technologies to users: Fedora Core -> RHEL, and sometimes RHEL technology previews. SuSE presumably does something similar with OpenSUSE. </vendor hat> There is PLENTY of opportunity to be experimental. > is populated with people FAR more suited to experimenting with > experimental filesystems than typical distro customer lists are. It is > commercial and political reasons that motivate distros being the first > with patches not tried yet by lkml, not the interests of the users. > Now, for other patches these commercial and political reasons may need > to be catered to as the price of getting the Redhats of the world to > fund kernel development, but that logic does not apply to Reiser4's > particulars. I always feel sad to hear technologists wail about politics. In my experience, the cause of such is almost always the fault of the submittor, ignoring consensus. But once the submittor has decided that "politics" are cause of their troubles, the submittor focuses on that rather than addressing the technology objections that were raised. With you in particular, you demonstrated NO interest in maintaining reiser3, once reiser4 began to make a splash. Linux kernel code exists for DECADES, and as such, long term maintenance is a CRITICAL aspect of development. Regardless of whatever new whiz-bang technology exists in reiser4, there is a very real worry that you will abandon reiser4 once its in the tree for a few years, just like what happened with reiser3. Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-22 20:29 ` Jeff Garzik @ 2006-07-23 7:20 ` Hans Reiser 2006-07-23 9:12 ` Matt Heler ` (5 more replies) 2006-07-24 8:41 ` Matthias Andree 1 sibling, 6 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-23 7:20 UTC (permalink / raw) To: Jeff Garzik; +Cc: Theodore Tso, LKML, ReiserFS List Jeff, I think that a large part of what is going on is that any patch that can be read in 15 minutes gets reviewed immediately, and any patch that is worked on for 5 years and then takes a week to read gets neglected. This is true even if line for line the 1 week to read patch is more valuable. What is more is that people know this is irrational, but aren't able to cure it in themselves. Even I have a problem of paying too much attention to endless 5 minute emails when I know I should instead, say, read the compression plugin from beginning to end. There is nothing about small patches that makes them better code. There is no reason we should favor them, if the developers are willing to work on something for 5 years to escape a local optimum, that is often the RIGHT thing to do. It is importand that we embrace our diversity, and be happy for the strength it gives us. Some of us are good at small patches that evolve, and some are good at escaping local optimums. We all have value, both trees and grass have their place in the world. > > > With you in particular, you demonstrated NO interest in maintaining > reiser3, once reiser4 began to make a splash. Linux kernel code > exists for DECADES, and as such, long term maintenance is a CRITICAL > aspect of development. You are rejecting the development model which is based on stable branches getting only bugfixes. V3 is a stable branch. It just had a feature added to it which added a bug that MythTV users are hitting. Some of them are responding to it by walking away from Reiser3, and no doubt muttering about what an unstable pile of shit our code is. On monday one of my guys is stopping work on V4 to send in a bug fix for a feature that should have gone into V4 first, and then maybe gotten backported after it was proven in V4. So, given that Jeff and Chris can often be gotten to fix bugs, do I ask them to do it whenever there is a bug to fix and they will fix it? Oh yes! The despiriting thing though is that there is usually another reason to let them fix it, which is that almost all v3 bugs are in features they have added to what ought to have been a stable branch, and since it is their code, they should be the ones to fix it. We might, maybe, get one bug report a year in code written by Namesys before I announced code freeze on V3. I just got an email from the programmer who wrote the MythTV bug saying that he is just too busy to bother fixing the bug in his code..... so my response is that a Namesys programmer is going to fix it on Monday. All this talk about how you guys worry that code is going to be abandoned, you know, try policing the kids in their 20's who do it, not those who have been working since 1984 on developing the thing you somehow are worried they will abandon. I am not 20 something anymore, I am getting fat no matter how much I exercise, and I stick with things, and I only wish some things didn't stick so much with my middle.... > > Regardless of whatever new whiz-bang technology exists in reiser4, > there is a very real worry that you will abandon reiser4 once its in > the tree for a few years, just like what happened with reiser3. And look at how Linus abandoned 2.4! Users of 2.4 needed so many features that were put into 2.6 instead, and they were just abandoned and neglected and.... Do you think he will abandon 2.6.18 also? The stable branch of code getting only bugfixes and the development branch getting all the new features model of development is something most release management professionals agree is the right way to do things. I worked with release management teams some, and I have to say that the dominant paradigm in the software industry is, in this case, the best one yet. Of course, I want to make it a little better, you know how I am, and as I was just discussing on the reiserfs-list, with plugins we can now move to a model in which if you mount reiser4 using the -o reiser4.1-beta mount option, it changes what the default plugin is, and that is how we do releases, we put our beta code in different plugins, and let the user choose whether to upgrade to a new release by just choosing what plugins to use as his default. Now that we paid the 5 year development price tag to get everything as plugins, we can now upgrade in littler pieces than any other FS. Hmm, I need a buzz phrase, its not extreme programming, maybe "moderate programming". Does that sound exciting to others.;-) Seriously though, I am curious to see whether plugin based release management works out as pleasantly for users as I am hoping it will. Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-23 7:20 ` Hans Reiser @ 2006-07-23 9:12 ` Matt Heler 2006-07-24 4:01 ` Hans Reiser 2006-07-23 11:48 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Jan-Benedict Glaw ` (4 subsequent siblings) 5 siblings, 1 reply; 639+ messages in thread From: Matt Heler @ 2006-07-23 9:12 UTC (permalink / raw) To: Hans Reiser; +Cc: Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Sunday 23 July 2006 12:20 am, Hans Reiser wrote: > I just got an email from the programmer who wrote the MythTV bug saying > that he is just too busy to bother fixing the bug in his code..... so > my response is that a Namesys programmer is going to fix it on Monday. The way you wrote this, makes it sound like a userspace issue, and _not_ a problem with reiserfs. > And look at how Linus abandoned 2.4! Users of 2.4 needed so many > features that were put into 2.6 instead, and they were just abandoned > and neglected and.... Do you think he will abandon 2.6.18 also? Not entirely true, he did not abandon the 2.4 kernel branch, he passed on maintainership to Marcelo. Similar to how he passed the torch on the 2.2 kernel branch to Alan Cox. Also on a side note, many new features ( and a ton of bug fixes !! ) were added to the 2.4 series _after_ Linus started working on the 2.5 branch. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-23 9:12 ` Matt Heler @ 2006-07-24 4:01 ` Hans Reiser 2006-07-24 8:54 ` Matthias Andree 0 siblings, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-07-24 4:01 UTC (permalink / raw) To: lkml; +Cc: Jeff Garzik, Theodore Tso, LKML, ReiserFS List Matt Heler wrote: >On Sunday 23 July 2006 12:20 am, Hans Reiser wrote: > > > >The way you wrote this, makes it sound like a userspace issue, and _not_ a >problem with reiserfs. > > It was a problem with reiserfs. Code was added to search for the perfect spot to fit a file. If there is no perfect spot, it searches every bitmap for that spot before giving up. However, Jeff kindly gave us a little patch to fix this and made the whole issue moot. It also seems I was in error, and we actually have had this problem since 2002. Now some past remarks from users about fragmentation make more sense. What can I say, since I have no MP3s I never get anywhere near full on my personal hard drive. > > >>And look at how Linus abandoned 2.4! Users of 2.4 needed so many >>features that were put into 2.6 instead, and they were just abandoned >>and neglected and.... Do you think he will abandon 2.6.18 also? >> >> > >Not entirely true, he did not abandon the 2.4 kernel branch, he passed on >maintainership to Marcelo. Similar to how he passed the torch on the 2.2 >kernel branch to Alan Cox. Also on a side note, many new features ( and a ton >of bug fixes !! ) were added to the 2.4 series _after_ Linus started working >on the 2.5 branch. > > You missed the sarcasm in my voice, my apologies, it is the trouble I have with email. Just to balance everything with some nuance, let me add that when a development branch is first opened, there is usually a bit of gray as to whether particular small features should go into the development branch or the stable branch. As the stable branch gets more stable the incentive to not destabilize it increases, and as a development branch becomes usable, the delay to users due to putting features only there reduces. I want reiserfs to be the filesystem that professional system administrators view as the one with both the fastest technological pace, and the most conservative release management. I apologize to users that the technology required a 5 year gap between releases. It just did, an outsider may not realize how deep the changes we made were. Things like per node locking based on a whole new approach to tree locking that goes bottom up instead of the usual top down are big tasks. Dancing trees are a big change, getting rid of blobs is a big change, wandering logs..... We did a lot of things like that, and got very fortunate with them. If we had tried to add such changes to V3, the code would have been unstable the whole 5 years, and would not have come out right. Experienced writers know that often, if you want to fix a passage, even a passage that is quite good in some parts, sometimes it is better to write the whole passage again without looking at the text of the first draft of the old passage, because sometimes your muse just needs the freedom, and without the freedom the awkwardness of the old passage is incurable. Probably there is some very sophisticated neurological reason why that is. Code can be the same. Sometimes. I knew that reiser4 HAD to be written from scratch without reference to the old code if it was to come out right. If I cannot be a great artist, at least I can try to have the temperament of one, yes? :-) I sincerely hope that using mount options to select default plugins, and making development code go into new plugins means that releases after this can be roughly quarterly, and that we can start doing a whole bunch of quick little plugins. Technically, I think it is going to be downhill skiing from here, and some very visible bits of functionality will get added much more easily than this difficult infrastructure we just coded. Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 4:01 ` Hans Reiser @ 2006-07-24 8:54 ` Matthias Andree 2006-07-24 8:13 ` Hans Reiser 0 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-07-24 8:54 UTC (permalink / raw) To: Hans Reiser; +Cc: lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Sun, 23 Jul 2006, Hans Reiser wrote: > I want reiserfs to be the filesystem that professional system > administrators view as the one with both the fastest technological pace, > and the most conservative release management. Well, I, with the administrator hat on, phased out all reiserfs file systems and replaced them by ext3. This got me rid of silent corruptions, immature reiserfsprogs and hash collision chain limits. > I apologize to users that the technology required a 5 year gap between > releases. It just did, an outsider may not realize how deep the > changes we made were. Things like per node locking based on a whole new > approach to tree locking that goes bottom up instead of the usual top > down are big tasks. Dancing trees are a big change, getting rid of > blobs is a big change, wandering logs..... We did a lot of things like > that, and got very fortunate with them. If we had tried to add such > changes to V3, the code would have been unstable the whole 5 years, and > would not have come out right. And that is something that an administrator does not care the least about. It must simply work, and the tools must simply work. Once I hit issues like "xfs_check believes / were mounted R/W (not ignoring rootfs) and refuses the R/O check", "reiserfsck can't fix a R/O file system" (I believed this one got fixed before 3.6.19) or particularly silent corruptions that show up later in a routine fsck --check after a kernel update, the filesystem and its tools appear in a bad light. I've never had such troubles with ext2fs or ext3fs or FreeBSD's or Solaris's ufs. I'm not sure what patches Chris added to SUSE's reiserfs, nor do I care any more. The father declared his child unsupported, and that's the end of the story for me. There's nothing wrong about focusing on newer code, but the old code needs to be cared for, too, to fix remaining issues such as the "can only have N files with the same hash value". (I am well aware this is exploiting worst-case behavior in a malicious sense but I simply cannot risk such nonsense on a 270 GB RAID5 if users have shared work directories.) -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 8:54 ` Matthias Andree @ 2006-07-24 8:13 ` Hans Reiser 2006-07-24 10:25 ` Matthias Andree 2006-07-26 13:17 ` Pavel Machek 0 siblings, 2 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-24 8:13 UTC (permalink / raw) To: Matthias Andree; +Cc: lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List Matthias Andree wrote: > The father declared his child unsupported, > I never did that. >and that's the end >of the story for me. There's nothing wrong about focusing on newer code, >but the old code needs to be cared for, too, to fix remaining issues >such as the "can only have N files with the same hash value". > Requires a disk format change, in a filesystem without plugins, to fix it. >(I am well >aware this is exploiting worst-case behavior in a malicious sense but I >simply cannot risk such nonsense on a 270 GB RAID5 if users have shared >work directories.) > > > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 8:13 ` Hans Reiser @ 2006-07-24 10:25 ` Matthias Andree 2006-07-24 11:34 ` Christian Iversen ` (2 more replies) 2006-07-26 13:17 ` Pavel Machek 1 sibling, 3 replies; 639+ messages in thread From: Matthias Andree @ 2006-07-24 10:25 UTC (permalink / raw) To: Hans Reiser Cc: Matthias Andree, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Mon, 24 Jul 2006, Hans Reiser wrote: > >and that's the end > >of the story for me. There's nothing wrong about focusing on newer code, > >but the old code needs to be cared for, too, to fix remaining issues > >such as the "can only have N files with the same hash value". > > Requires a disk format change, in a filesystem without plugins, to fix it. You see, I don't care a iota about "plugins" or other implementation details. The bottom line is reiserfs 3.6 imposes practial limits that ext3fs doesn't impose and that's reason enough for an administrator not to install reiserfs 3.6. Sorry. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 10:25 ` Matthias Andree @ 2006-07-24 11:34 ` Christian Iversen 2006-07-24 12:37 ` Erik Mouw 2006-07-24 16:57 ` Mike Benoit 2006-07-24 20:28 ` Lex Lyamin 2 siblings, 1 reply; 639+ messages in thread From: Christian Iversen @ 2006-07-24 11:34 UTC (permalink / raw) To: Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Monday 24 July 2006 12:25, Matthias Andree wrote: > On Mon, 24 Jul 2006, Hans Reiser wrote: > > >and that's the end > > >of the story for me. There's nothing wrong about focusing on newer code, > > >but the old code needs to be cared for, too, to fix remaining issues > > >such as the "can only have N files with the same hash value". > > > > Requires a disk format change, in a filesystem without plugins, to fix > > it. > > You see, I don't care a iota about "plugins" or other implementation > details. > > The bottom line is reiserfs 3.6 imposes practial limits that ext3fs > doesn't impose and that's reason enough for an administrator not to > install reiserfs 3.6. Sorry. And what do you do if you, say, run of of inodes on ext3? Do you think the users will care about that? Or what if the number of files in your mail queue or proxy cache* become large enough for your fs operations to slow to a crawl? * Yes I know most programs work around this by using many subdirs, but that's really a bandaid solution. -- Regards, Christian Iversen ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 11:34 ` Christian Iversen @ 2006-07-24 12:37 ` Erik Mouw 0 siblings, 0 replies; 639+ messages in thread From: Erik Mouw @ 2006-07-24 12:37 UTC (permalink / raw) To: Christian Iversen Cc: Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Mon, Jul 24, 2006 at 01:34:11PM +0200, Christian Iversen wrote: > On Monday 24 July 2006 12:25, Matthias Andree wrote: > > The bottom line is reiserfs 3.6 imposes practial limits that ext3fs > > doesn't impose and that's reason enough for an administrator not to > > install reiserfs 3.6. Sorry. > > And what do you do if you, say, run of of inodes on ext3? Do you think the > users will care about that? >From what I've seen from our customers, that never happens. Yes, there are sometimes people with a million inodes in use, and we've seen four million once, but that's never been a problem, even not with a huge mail server with thousands of users having mailboxes in maildir format. We usually limit our own filesystems to 12 million inodes and it's never been a problem to store files from our customers. > Or what if the number of files in your mail queue > or proxy cache* become large enough for your fs operations to slow to a > crawl? Not a problem anymore with htree dirextory indexing. If it's not yet enabled (dumpe2fs the filesystem and look for the "dir_index" feature), enable it with: tune2fs -O dir_index /dev/whatever After the next mount the filesystem will use it for new directories. To optimize existing directories, run e2fsck -D. Erik -- +-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 -- | Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion @ 2006-07-24 12:37 ` Erik Mouw 0 siblings, 0 replies; 639+ messages in thread From: Erik Mouw @ 2006-07-24 12:37 UTC (permalink / raw) To: Christian Iversen Cc: Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Mon, Jul 24, 2006 at 01:34:11PM +0200, Christian Iversen wrote: > On Monday 24 July 2006 12:25, Matthias Andree wrote: > > The bottom line is reiserfs 3.6 imposes practial limits that ext3fs > > doesn't impose and that's reason enough for an administrator not to > > install reiserfs 3.6. Sorry. > > And what do you do if you, say, run of of inodes on ext3? Do you think the > users will care about that? From what I've seen from our customers, that never happens. Yes, there are sometimes people with a million inodes in use, and we've seen four million once, but that's never been a problem, even not with a huge mail server with thousands of users having mailboxes in maildir format. We usually limit our own filesystems to 12 million inodes and it's never been a problem to store files from our customers. > Or what if the number of files in your mail queue > or proxy cache* become large enough for your fs operations to slow to a > crawl? Not a problem anymore with htree dirextory indexing. If it's not yet enabled (dumpe2fs the filesystem and look for the "dir_index" feature), enable it with: tune2fs -O dir_index /dev/whatever After the next mount the filesystem will use it for new directories. To optimize existing directories, run e2fsck -D. Erik -- +-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 -- | Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 10:25 ` Matthias Andree 2006-07-24 11:34 ` Christian Iversen @ 2006-07-24 16:57 ` Mike Benoit 2006-07-24 17:35 ` Matthias Andree 2006-07-24 18:06 ` Horst H. von Brand 2006-07-24 20:28 ` Lex Lyamin 2 siblings, 2 replies; 639+ messages in thread From: Mike Benoit @ 2006-07-24 16:57 UTC (permalink / raw) To: Matthias Andree Cc: Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1708 bytes --] On Mon, 2006-07-24 at 12:25 +0200, Matthias Andree wrote: > On Mon, 24 Jul 2006, Hans Reiser wrote: > > > >and that's the end > > >of the story for me. There's nothing wrong about focusing on newer code, > > >but the old code needs to be cared for, too, to fix remaining issues > > >such as the "can only have N files with the same hash value". > > > > Requires a disk format change, in a filesystem without plugins, to fix it. > > You see, I don't care a iota about "plugins" or other implementation details. > > The bottom line is reiserfs 3.6 imposes practial limits that ext3fs > doesn't impose and that's reason enough for an administrator not to > install reiserfs 3.6. Sorry. > And EXT3 imposes practical limits that ReiserFS doesn't as well. The big one being a fixed number of inodes that can't be adjusted on the fly, which was reason enough for me to not use EXT3 and use ReiserFS instead. Do you consider the EXT3 developers to have "abandoned" it because they haven't fixed this issue? I don't, I just think of it as using the right tool for the job. I've been bitten by running out of inodes on several occasions, and by switching to ReiserFS it saved one company I worked for over $250,000 because they didn't need to buy a totally new piece of software. I haven't been able to use EXT3 on a backup server for the last ~5 years due to inode limitations. Instead, ReiserFS has been filling that spot like a champ. The bottom line is that every file system imposes some sort of limits that bite someone. In your case it sounds like EXT3 limits weren't an issue for you, in my case they were. Thats life. -- Mike Benoit <ipso@snappymail.ca> [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 16:57 ` Mike Benoit @ 2006-07-24 17:35 ` Matthias Andree 2006-07-24 18:28 ` Valdis.Kletnieks 2006-07-24 18:06 ` Horst H. von Brand 1 sibling, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-07-24 17:35 UTC (permalink / raw) To: Mike Benoit Cc: Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List Mike Benoit wrote: > I've been bitten by running out of inodes on several occasions, and by > switching to ReiserFS it saved one company I worked for over $250,000 > because they didn't need to buy a totally new piece of software. ext3fs's inode density is configurable, reiserfs's hash overflow chain length is not, and it doesn't show in df -i either. If you need lots of inodes, mkfs for lots. That's old Unix lore. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 17:35 ` Matthias Andree @ 2006-07-24 18:28 ` Valdis.Kletnieks 0 siblings, 0 replies; 639+ messages in thread From: Valdis.Kletnieks @ 2006-07-24 18:28 UTC (permalink / raw) To: Matthias Andree Cc: Mike Benoit, Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 573 bytes --] On Mon, 24 Jul 2006 19:35:24 +0200, Matthias Andree said: > Mike Benoit wrote: > > > I've been bitten by running out of inodes on several occasions, and by > > switching to ReiserFS it saved one company I worked for over $250,000 > > because they didn't need to buy a totally new piece of software. > > ext3fs's inode density is configurable, reiserfs's hash overflow chain > length is not, and it doesn't show in df -i either. Equally important - you can usually *see* "out of inodes" coming on a 'df -i' long before a reiser3 filesystem hits the wall on a hash issue. [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 16:57 ` Mike Benoit 2006-07-24 17:35 ` Matthias Andree @ 2006-07-24 18:06 ` Horst H. von Brand 2006-07-24 20:37 ` Mike Benoit 2006-07-31 10:58 ` Adrian Ulrich 1 sibling, 2 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-07-24 18:06 UTC (permalink / raw) To: Mike Benoit Cc: Matthias Andree, Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List Mike Benoit <ipso@snappymail.ca> wrote: > On Mon, 2006-07-24 at 12:25 +0200, Matthias Andree wrote: > > On Mon, 24 Jul 2006, Hans Reiser wrote: > > > > > >and that's the end > > > >of the story for me. There's nothing wrong about focusing on newer code, > > > >but the old code needs to be cared for, too, to fix remaining issues > > > >such as the "can only have N files with the same hash value". > > > > > > Requires a disk format change, in a filesystem without plugins, to fix it. > > You see, I don't care a iota about "plugins" or other implementation details. > > > > The bottom line is reiserfs 3.6 imposes practial limits that ext3fs > > doesn't impose and that's reason enough for an administrator not to > > install reiserfs 3.6. Sorry. > And EXT3 imposes practical limits that ReiserFS doesn't as well. The big > one being a fixed number of inodes that can't be adjusted on the fly, Right. Plan ahead. > which was reason enough for me to not use EXT3 and use ReiserFS > instead. I don't see this following in any way. > Do you consider the EXT3 developers to have "abandoned" it because they > haven't fixed this issue? I don't, I just think of it as using the right > tool for the job. Dangerous parallel, that one... > I've been bitten by running out of inodes on several occasions, Me too. It was rather painful each time, but fixable (and in hindsight, dumb user (setup) error). > and by > switching to ReiserFS it saved one company I worked for over $250,000 > because they didn't need to buy a totally new piece of software. How can a filesystem (which by basic requirements and design is almost transparent to applications) make such a difference?! > I haven't been able to use EXT3 on a backup server for the last ~5 years > due to inode limitations. See comment above. Read mke2fs(8) with care. > Instead, ReiserFS has been filling that spot > like a champ. Nice for you. > The bottom line is that every file system imposes some sort of limits > that bite someone. Mostly that infinite disks are hard to come by ;-) > In your case it sounds like EXT3 limits weren't an > issue for you, in my case they were. I'd suspect the limits you ran into weren't exactly in ext3. > Thats life. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 18:06 ` Horst H. von Brand @ 2006-07-24 20:37 ` Mike Benoit 2006-07-24 21:22 ` Jan-Benedict Glaw 2006-07-24 21:51 ` Horst H. von Brand 2006-07-31 10:58 ` Adrian Ulrich 1 sibling, 2 replies; 639+ messages in thread From: Mike Benoit @ 2006-07-24 20:37 UTC (permalink / raw) To: Horst H. von Brand Cc: Matthias Andree, Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 2406 bytes --] On Mon, 2006-07-24 at 14:06 -0400, Horst H. von Brand wrote: > > And EXT3 imposes practical limits that ReiserFS doesn't as well. The big > > one being a fixed number of inodes that can't be adjusted on the fly, > > Right. Plan ahead. That is great in theory. But back to reality, when your working for a company that is growing by leaps and bounds that isn't always possible. Why would I want to intentionally limit myself to a set number of inodes when I can get a performance increase and not have to worry about it by simply using ReiserFS? It all boils down to using the right tool for the job, ReiserFS was the right tool for this job. > > I've been bitten by running out of inodes on several occasions, > > Me too. It was rather painful each time, but fixable (and in hindsight, > dumb user (setup) error). > > > and by > > switching to ReiserFS it saved one company I worked for over $250,000 > > because they didn't need to buy a totally new piece of software. > > How can a filesystem (which by basic requirements and design is almost > transparent to applications) make such a difference?! Very easily, the backup software the company had spent ~$75,000 on before I started working there used tiny little files as its "database". Once the inodes ran out the entire system pretty much came to a screeching halt. We basically had two options, use ReiserFS, or find another piece of software that didn't use tiny little files as its database. If I recall correctly the database went from about 2million files to over 40 million in the span of just a few months. No one could have predicted that. I believe this database was on an 18GB SCSI drive, so even using 1K blocks wouldn't have worked. According to the mke2fs man page: "-i bytes-per-inode This value generally shouldn't be smaller than the blocksize of the filesystem, since then too many inodes will be made." The only other option at that time was to purchase Veritas backup and its not cheap. We ended up switching to ReiserFS, it increased our backup/restore time immensely and also bought us about 1year before the system finally outgrew itself for good. By that time the company could afford to drop $250,000 on high end backup software so we could grow past 10TB. -- Mike Benoit <ipso@snappymail.ca> [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 20:37 ` Mike Benoit @ 2006-07-24 21:22 ` Jan-Benedict Glaw 2006-07-24 21:51 ` Horst H. von Brand 1 sibling, 0 replies; 639+ messages in thread From: Jan-Benedict Glaw @ 2006-07-24 21:22 UTC (permalink / raw) To: Mike Benoit Cc: Horst H. von Brand, Matthias Andree, Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 887 bytes --] On Mon, 2006-07-24 13:37:13 -0700, Mike Benoit <ipso@snappymail.ca> wrote: > On Mon, 2006-07-24 at 14:06 -0400, Horst H. von Brand wrote: > The only other option at that time was to purchase Veritas backup and > its not cheap. We ended up switching to ReiserFS, it increased our > backup/restore time immensely and also bought us about 1year before the > system finally outgrew itself for good. By that time the company could > afford to drop $250,000 on high end backup software so we could grow > past 10TB. Erm, how does your data look like and why does it require a $250,000 backup solution? Sounds like I'm in the wrong business... But that's for sure not lkml related... MfG, JBG -- Jan-Benedict Glaw jbglaw@lug-owl.de +49-172-7608481 Signature of: Don't believe in miracles: Rely on them! the second : [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 20:37 ` Mike Benoit 2006-07-24 21:22 ` Jan-Benedict Glaw @ 2006-07-24 21:51 ` Horst H. von Brand 2006-07-24 22:51 ` Mike Benoit ` (2 more replies) 1 sibling, 3 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-07-24 21:51 UTC (permalink / raw) To: Mike Benoit Cc: Horst H. von Brand, Matthias Andree, Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List Mike Benoit <ipso@snappymail.ca> wrote: > On Mon, 2006-07-24 at 14:06 -0400, Horst H. von Brand wrote: > > > And EXT3 imposes practical limits that ReiserFS doesn't as well. The big > > > one being a fixed number of inodes that can't be adjusted on the fly, > > Right. Plan ahead. > That is great in theory. But back to reality, when your working for a > company that is growing by leaps and bounds that isn't always possible. > Why would I want to intentionally limit myself to a set number of inodes > when I can get a performance increase and not have to worry about it by > simply using ReiserFS? Place your filesystems on LVN, when they grow the number of inodes grows to match. Or did you run out of inodes and not of diskspace? Only ever happened to me on (static) /dev, or (wrong configured) newsspools... > It all boils down to using the right tool for the job, ReiserFS was the > right tool for this job. /One/ tool that did the job. Not "the" right one, perhaps not even the best one. > > > I've been bitten by running out of inodes on several occasions, > > Me too. It was rather painful each time, but fixable (and in hindsight, > > dumb user (setup) error). > > > and by > > > switching to ReiserFS it saved one company I worked for over $250,000 > > > because they didn't need to buy a totally new piece of software. > > How can a filesystem (which by basic requirements and design is almost > > transparent to applications) make such a difference?! > Very easily, the backup software the company had spent ~$75,000 on > before I started working there used tiny little files as its "database". If you know that, you configure your filesystem like a newsspool or some such. > Once the inodes ran out the entire system pretty much came to a > screeching halt. Get a clue by for, apply to the vendor (for the design, or at the very least for not warning unsuspecting users)? > We basically had two options, use ReiserFS, or find > another piece of software that didn't use tiny little files as its > database. Or reconfigure the filesystem with more inodes (you were willing to rebuild the filesystem in any case, so...) > If I recall correctly the database went from about 2million > files to over 40 million in the span of just a few months. No one could > have predicted that. I believe this database was on an 18GB SCSI drive, > so even using 1K blocks wouldn't have worked. According to the mke2fs > man page: > "-i bytes-per-inode > This value generally shouldn't be smaller than the blocksize of > the filesystem, since then too many inodes will be made." 18GiB = 18 million KiB, you do have a point there. But 40 million files on that, with some space to spare, just doesn't add up. > The only other option at that time was to purchase Veritas backup ... or a larger disk... > and > its not cheap. We ended up switching to ReiserFS, it increased our > backup/restore time immensely and also bought us about 1year before the > system finally outgrew itself for good. By that time the company could > afford to drop $250,000 on high end backup software so we could grow > past 10TB. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 21:51 ` Horst H. von Brand @ 2006-07-24 22:51 ` Mike Benoit 2006-07-25 15:08 ` Denis Vlasenko 2006-07-26 0:29 ` David Masover 2 siblings, 0 replies; 639+ messages in thread From: Mike Benoit @ 2006-07-24 22:51 UTC (permalink / raw) To: Horst H. von Brand; +Cc: reiserfs-list [-- Attachment #1: Type: text/plain, Size: 1964 bytes --] On Mon, 2006-07-24 at 17:51 -0400, Horst H. von Brand wrote: > > It all boils down to using the right tool for the job, ReiserFS was the > > right tool for this job. > > /One/ tool that did the job. Not "the" right one, perhaps not even the best > one. Please, enlighten me as to what "the best" tool for the job would be in your opinion. Keep in mind this was around 2002-2003, which if I recall was before dir_index for EXT3 was around and stable too. > > If I recall correctly the database went from about 2million > > files to over 40 million in the span of just a few months. No one could > > have predicted that. I believe this database was on an 18GB SCSI drive, > > so even using 1K blocks wouldn't have worked. According to the mke2fs > > man page: > > > "-i bytes-per-inode > > This value generally shouldn't be smaller than the blocksize of > > the filesystem, since then too many inodes will be made." > > 18GiB = 18 million KiB, you do have a point there. But 40 million files on > that, with some space to spare, just doesn't add up. > > > The only other option at that time was to purchase Veritas backup > > ... or a larger disk... Why would we waste money purchasing larger disks just to _temporarily_ work around an EXT3 limitation? Even if we doubled the size of the disks (36gb) we would have run in to the same problem within months. I wouldn't consider this to be smart "planning ahead" as you put it. Its not just an inode issue either, its a performance issue. Benchmarks we ran with just 100,000 files making up 1GB of data, ReiserFS blew EXT3 out of the water where EXT3 was over 50% slower. If I recall the "database" was about 5GB at 25million files. When you have a limited window of time to do the backups in, performance is critical. http://fsbench.netnation.com/new_hardware/2.6.0-test9/scsi/bonnie.html (4th table down) -- Mike Benoit <ipso@snappymail.ca> [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 21:51 ` Horst H. von Brand 2006-07-24 22:51 ` Mike Benoit @ 2006-07-25 15:08 ` Denis Vlasenko 2006-07-25 20:49 ` Matthias Andree 2006-07-26 0:29 ` David Masover 2 siblings, 1 reply; 639+ messages in thread From: Denis Vlasenko @ 2006-07-25 15:08 UTC (permalink / raw) To: Horst H. von Brand Cc: Mike Benoit, Matthias Andree, Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Monday 24 July 2006 23:51, Horst H. von Brand wrote: > > Once the inodes ran out the entire system pretty much came to a > > screeching halt. > > Get a clue by for, apply to the vendor (for the design, or at the very > least for not warning unsuspecting users)? > > > We basically had two options, use ReiserFS, or find > > another piece of software that didn't use tiny little files as its > > database. > > Or reconfigure the filesystem with more inodes (you were willing to rebuild > the filesystem in any case, so...) The fact that _many_ UNIX filesystem formats have inode number limit (configurable only at mkfs time) doesn't make it a good design. By the same virtue you may declare that it is okay to smoke and drink lots of vodka. Why not? Sizable portion if population does that... I, on the contrary, want software to impose as few limits on me as possible. -- vda ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 15:08 ` Denis Vlasenko @ 2006-07-25 20:49 ` Matthias Andree 2006-07-25 23:04 ` David Masover 0 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-07-25 20:49 UTC (permalink / raw) To: Denis Vlasenko Cc: Horst H. von Brand, Mike Benoit, Matthias Andree, Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Tue, 25 Jul 2006, Denis Vlasenko wrote: > I, on the contrary, want software to impose as few limits on me > as possible. As long as it's choosing some limit, I'll pick the one with fewer surprises. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 20:49 ` Matthias Andree @ 2006-07-25 23:04 ` David Masover 2006-07-26 11:20 ` Matthias Andree 0 siblings, 1 reply; 639+ messages in thread From: David Masover @ 2006-07-25 23:04 UTC (permalink / raw) To: Denis Vlasenko, Horst H. von Brand, Mike Benoit, Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 383 bytes --] Matthias Andree wrote: > On Tue, 25 Jul 2006, Denis Vlasenko wrote: > >> I, on the contrary, want software to impose as few limits on me >> as possible. > > As long as it's choosing some limit, I'll pick the one with fewer > surprises. Running out of inodes would be pretty surprising for me. But then, I guess it's a good thing I don't admin for a living anymore. [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 890 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 23:04 ` David Masover @ 2006-07-26 11:20 ` Matthias Andree 2006-07-26 11:26 ` Matthias Andree ` (2 more replies) 0 siblings, 3 replies; 639+ messages in thread From: Matthias Andree @ 2006-07-26 11:20 UTC (permalink / raw) To: David Masover; +Cc: LKML, ReiserFS List On Tue, 25 Jul 2006, David Masover wrote: > Matthias Andree wrote: > > On Tue, 25 Jul 2006, Denis Vlasenko wrote: > > > >> I, on the contrary, want software to impose as few limits on me > >> as possible. > > > > As long as it's choosing some limit, I'll pick the one with fewer > > surprises. > > Running out of inodes would be pretty surprising for me. No offense: Then it was a surprise for you because you closed your eyes and didn't look at df -i or didn't have monitors in place. There is no way to ask how many files with particular hash values you can still stuff into a reiserfs 3.X. There, you're running into a brick wall that only your forehead will "see" when you touch it. Of course, different sites have different needs and if you need gazillions of inodes or file names, you may see trouble. But the assertion that some backup was the cause for inode exhaustion on ext? is not very plausible since hard links do not take up inodes, symlinks are not backups and everything else requires disk blocks. So, since reformatting ext2/ext3 to one inode per block is possible (regardless of disk capacity), I see no way how a reformatted file system might run out of inodes before it runs out of blocks. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 11:20 ` Matthias Andree @ 2006-07-26 11:26 ` Matthias Andree 2006-07-26 13:02 ` Bernd Eckenfels 2006-07-27 1:29 ` David Masover 2 siblings, 0 replies; 639+ messages in thread From: Matthias Andree @ 2006-07-26 11:26 UTC (permalink / raw) To: David Masover, LKML, ReiserFS List On Wed, 26 Jul 2006, Matthias Andree wrote: > But the assertion that some backup was the cause for inode exhaustion on > ext? is not very plausible since hard links do not take up inodes, > symlinks are not backups and everything else requires disk blocks. So, > since reformatting ext2/ext3 to one inode per block is possible > (regardless of disk capacity), I see no way how a reformatted file > system might run out of inodes before it runs out of blocks. OK; ext2/ext3 require 1k blocks, but still you need heaps of files < 1k to run out of inodes without running of space. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 11:20 ` Matthias Andree 2006-07-26 11:26 ` Matthias Andree @ 2006-07-26 13:02 ` Bernd Eckenfels 2006-07-26 18:54 ` Buddy Lucas 2006-07-27 1:29 ` David Masover 2 siblings, 1 reply; 639+ messages in thread From: Bernd Eckenfels @ 2006-07-26 13:02 UTC (permalink / raw) To: linux-kernel Hello, I know thats not relevant for the discussion, but I wanted to share my experiences anyway (to emphasis how important df-i monitoring on smaller filesystems is): Matthias Andree <matthias.andree@gmx.de> wrote: > But the assertion that some backup was the cause for inode exhaustion on > ext? is not very plausible since hard links do not take up inodes, > symlinks are not backups and everything else requires disk blocks. So, > since reformatting ext2/ext3 to one inode per block is possible > (regardless of disk capacity), I see no way how a reformatted file > system might run out of inodes before it runs out of blocks. Well I had actually the problem on a tmpfs where I had too many zero byte files... Gruss Bernd > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 13:02 ` Bernd Eckenfels @ 2006-07-26 18:54 ` Buddy Lucas 0 siblings, 0 replies; 639+ messages in thread From: Buddy Lucas @ 2006-07-26 18:54 UTC (permalink / raw) To: Bernd Eckenfels; +Cc: linux-kernel On 7/26/06, Bernd Eckenfels <be-news06@lina.inka.de> wrote: > > Matthias Andree <matthias.andree@gmx.de> wrote: > > But the assertion that some backup was the cause for inode exhaustion on > > ext? is not very plausible since hard links do not take up inodes, > > symlinks are not backups and everything else requires disk blocks. So, > > since reformatting ext2/ext3 to one inode per block is possible > > (regardless of disk capacity), I see no way how a reformatted file > > system might run out of inodes before it runs out of blocks. > > Well I had actually the problem on a tmpfs where I had too many zero byte > files... Yes, I once ran out of inodes because logrotate kept rotating and compressing already compressed and empty logfiles. I can't remember how many seconds it took me to add 'df -i' to our monitoring system. This, however, was not a feature of the software. I assume. So, any company that considers the remote possibility of seeking a $250,000 solution, where the alternative is to buy a 36GB hard drive, please give me a call. Cheers, Buddy ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 11:20 ` Matthias Andree 2006-07-26 11:26 ` Matthias Andree 2006-07-26 13:02 ` Bernd Eckenfels @ 2006-07-27 1:29 ` David Masover 2 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-07-27 1:29 UTC (permalink / raw) To: David Masover, LKML, ReiserFS List Matthias Andree wrote: > On Tue, 25 Jul 2006, David Masover wrote: > >> Matthias Andree wrote: >>> On Tue, 25 Jul 2006, Denis Vlasenko wrote: >>> >>>> I, on the contrary, want software to impose as few limits on me >>>> as possible. >>> As long as it's choosing some limit, I'll pick the one with fewer >>> surprises. >> Running out of inodes would be pretty surprising for me. > > No offense: Then it was a surprise for you because you closed your eyes > and didn't look at df -i or didn't have monitors in place. Or because my (hypothetical) business exploded before I had the chance. After all, you could make the same argument about bandwidth, until you get Slashdotted. Surprise! > There is no way to ask how many files with particular hash values you > can still stuff into a reiserfs 3.X. There, you're running into a brick > wall that only your forehead will "see" when you touch it. That's true, so you may be correct about "less" surprises. So, it depends which is more valuable -- fewer surprises, or fewer limits? That's not a hypothetical statement, and I don't really know. I can see both sides of this one. But I do hope that once Reiser4 is stable enough for you, it will be predictable enough. > But the assertion that some backup was the cause for inode exhaustion on > ext? is not very plausible since hard links do not take up inodes, > symlinks are not backups and everything else requires disk blocks. So, Ok, where's the assertion that symlinks are not backups? Or not used in backup software? What about directories full of hardlinks -- the dirs themselves must use something, right? Anyway, it wasn't my project that hit this limit. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 21:51 ` Horst H. von Brand 2006-07-24 22:51 ` Mike Benoit 2006-07-25 15:08 ` Denis Vlasenko @ 2006-07-26 0:29 ` David Masover 2006-07-26 0:36 ` David Lang 2 siblings, 1 reply; 639+ messages in thread From: David Masover @ 2006-07-26 0:29 UTC (permalink / raw) To: Horst H. von Brand Cc: Mike Benoit, Matthias Andree, Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1747 bytes --] Horst H. von Brand wrote: > 18GiB = 18 million KiB, you do have a point there. But 40 million files on > that, with some space to spare, just doesn't add up. Right, ok... Here's a quick check of my box. I've explicitly stated which root-level directories to search, to avoid nfs mounts, chrooted OSes, and virtual filesystems like /proc and /sys. elite ~ # find /bin/ /boot/ /dev/ /emul/ /etc/ /home /lib32 /lib64 /opt /root /sbin /tmp /usr /var -type f -size 1 | wc -l 246127 According to the "find" manpage: -size n[bckw] File uses n units of space. The units are 512-byte blocks by default or if `b' follows n, bytes if `c' follows n, kilobytes if `k' follows n, or 2-byte words if `w' follows n. The size does not count indirect blocks, but it does count blocks in sparse files that are not actually allocated. And I certainly didn't plan it that way. And this is my desktop box, and I'm just one user. Most of the space is taken up by movies. And yet, I have almost 250k files at the moment whose size is less than 512 bytes. And this is a normal usage pattern. It's not hard to imagine something prone to creating lots of tiny files, combined with thousands of users, easily hitting some 40 mil files -- and since none of them are movies, it could fit in 18 gigs. I mean, just for fun: elite ~ # find /bin/ /boot/ /dev/ /emul/ /etc/ /home /lib32 /lib64 /opt /root /sbin /tmp /usr /var | wc -l 866160 It may not be a good idea, but it's possible. And one of the larger reasons it's not a good idea is that most filesystems can't handle it. Kind of like how BitTorrent is a very bad idea on dialup, but a very good idea on broadband. [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 890 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 0:29 ` David Masover @ 2006-07-26 0:36 ` David Lang 2006-07-26 0:47 ` David Masover 0 siblings, 1 reply; 639+ messages in thread From: David Lang @ 2006-07-26 0:36 UTC (permalink / raw) To: David Masover Cc: Horst H. von Brand, Mike Benoit, Matthias Andree, Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Tue, 25 Jul 2006, David Masover wrote: > Horst H. von Brand wrote: > >> 18GiB = 18 million KiB, you do have a point there. But 40 million files on >> that, with some space to spare, just doesn't add up. if you have 18 million KiB and each file is a single block (512 Bytes = 0.5 Kib) then assuming zero overhead you could fit 18 Million KiB / 0.5 KiB = 36 Million files on the drive. thus being scheptical about 40 million files on a 18G drive. this is only possible if you are abel to have multiple files per 512 byte block. David Lang > Right, ok... > > Here's a quick check of my box. I've explicitly stated which root-level > directories to search, to avoid nfs mounts, chrooted OSes, and virtual > filesystems like /proc and /sys. > > elite ~ # find /bin/ /boot/ /dev/ /emul/ /etc/ /home /lib32 /lib64 /opt > /root /sbin /tmp /usr /var -type f -size 1 | wc -l > 246127 > > According to the "find" manpage: > > -size n[bckw] > File uses n units of space. The units are 512-byte blocks by > default or if `b' follows n, bytes if `c' follows n, kilobytes > if `k' follows n, or 2-byte words if `w' follows n. The size > does not count indirect blocks, but it does count blocks in > sparse files that are not actually allocated. > > > And I certainly didn't plan it that way. And this is my desktop box, > and I'm just one user. Most of the space is taken up by movies. > > And yet, I have almost 250k files at the moment whose size is less than > 512 bytes. And this is a normal usage pattern. It's not hard to > imagine something prone to creating lots of tiny files, combined with > thousands of users, easily hitting some 40 mil files -- and since none > of them are movies, it could fit in 18 gigs. > > I mean, just for fun: > > elite ~ # find /bin/ /boot/ /dev/ /emul/ /etc/ /home /lib32 /lib64 /opt > /root /sbin /tmp /usr /var | wc -l > 866160 > > It may not be a good idea, but it's possible. And one of the larger > reasons it's not a good idea is that most filesystems can't handle it. > Kind of like how BitTorrent is a very bad idea on dialup, but a very > good idea on broadband. > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 0:36 ` David Lang @ 2006-07-26 0:47 ` David Masover 0 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-07-26 0:47 UTC (permalink / raw) To: David Lang Cc: Horst H. von Brand, Mike Benoit, Matthias Andree, Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 736 bytes --] David Lang wrote: > On Tue, 25 Jul 2006, David Masover wrote: > >> Horst H. von Brand wrote: >> >>> 18GiB = 18 million KiB, you do have a point there. But 40 million >>> files on >>> that, with some space to spare, just doesn't add up. > > if you have 18 million KiB and each file is a single block (512 Bytes = > 0.5 Kib) then assuming zero overhead you could fit 18 Million KiB / 0.5 > KiB = 36 Million files on the drive. > > thus being scheptical about 40 million files on a 18G drive. > > this is only possible if you are abel to have multiple files per 512 > byte block. I believe Reiser4 does this. Does ext3? I know I heard somewhere in this thread that you can't set the blocksize lower than 1k... [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 890 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 18:06 ` Horst H. von Brand 2006-07-24 20:37 ` Mike Benoit @ 2006-07-31 10:58 ` Adrian Ulrich 2006-07-31 14:47 ` Matthias Andree 1 sibling, 1 reply; 639+ messages in thread From: Adrian Ulrich @ 2006-07-31 10:58 UTC (permalink / raw) To: Horst H. von Brand Cc: ipso, matthias.andree, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list > > And EXT3 imposes practical limits that ReiserFS doesn't as well. The big > > one being a fixed number of inodes that can't be adjusted on the fly, > > Right. Plan ahead. Ok: Assume that i've read the mke2fs manpage and added more inodes to my filesystem. So: What happens if i need to grow my filesystem by 200% after 1-2 years? Can i add more inodes to Ext3 on-the-fly ? A filesystem with a fixed number of inodes (= not readjustable while mounted) is ehr.. somewhat unuseable for a lot of people with big and *flexible* storage needs (Talking about NetApp/EMC owners) Why are a lot of Solaris-people using (buying) VxFS? Maybe because UFS also has such silly limitations? (..and performs awkward with trillions of files..?..) Ext3 may be a fine and stable Filesystem and works well for a lot of people. But there are also a lot of people who need 'something' better like: VxFS, WAFL and Reiser4. Btw: Do you know Adics 'StorNext Filesystem' ? IMHO Ext3 will never be able to do such things... But with Reiser4.. .. if someone wrote a plugin .. ;-) Regards, Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 10:58 ` Adrian Ulrich @ 2006-07-31 14:47 ` Matthias Andree 2006-07-31 15:59 ` Adrian Ulrich ` (2 more replies) 0 siblings, 3 replies; 639+ messages in thread From: Matthias Andree @ 2006-07-31 14:47 UTC (permalink / raw) To: Adrian Ulrich Cc: Horst H. von Brand, ipso, matthias.andree, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Adrian Ulrich schrieb am 2006-07-31: > > > And EXT3 imposes practical limits that ReiserFS doesn't as well. The big > > > one being a fixed number of inodes that can't be adjusted on the fly, > > > > Right. Plan ahead. > > Ok: Assume that i've read the mke2fs manpage and added more inodes to > my filesystem. > > So: What happens if i need to grow my filesystem by 200% after 1-2 > years? Can i add more inodes to Ext3 on-the-fly ? Since you "grow", you'll be using resize2fs (or growfs or mkfs -G for UFS). resize2fs and the other tools do exactly that: add inodes - and you could easily have told this either from reading the resize2fs code or just trying it on a temp file: -- create file system dd if=/dev/zero of=/tmp/foo bs=1k count=50000 /sbin/mke2fs -F -j /tmp/foo -- check no. of inodes /sbin/tune2fs -l /tmp/foo | grep -i inode | head -2 # Inode count: 12544 # Free inodes: 12533 -- resize /sbin/e2fsck -f /tmp/foo dd if=/dev/zero bs=1k count=50000 >>/tmp/foo /sbin/resize2fs /tmp/foo -- check no. of inodes /sbin/tune2fs -l /tmp/foo | grep -i inode # Inode count: 23296 # Free inodes: 23285 Trying the same after mke2fs -b 1024 -i 1024 shows that the inode density will continue to be respected. FreeBSD 6.1's growfs(8) increases the number of inodes. This is documented to work since 4.4. Solaris 8's mkfs -G also increases the number of inodes and apparently also works for mounted file systems. This looks rather like an education issue rather than a technical limit. > A filesystem with a fixed number of inodes (= not readjustable while > mounted) is ehr.. somewhat unuseable for a lot of people with > big and *flexible* storage needs (Talking about NetApp/EMC owners) Which is untrue at least for Solaris, which allows resizing a life file system. FreeBSD and Linux require an unmount. > Why are a lot of Solaris-people using (buying) VxFS? Maybe because UFS > also has such silly limitations? (..and performs awkward with trillions > of files..?..) Well, such "silly limitations"... looks like they are mostly hot air spewn by marketroids that need to justify people spending money on their new filesystem. The only problem remains if you grossly overestimate the average file size and with it underestimate the number of inodes needed. But even then, I'd be interested to know if that's a real problem for systems such as ZFS. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 14:47 ` Matthias Andree @ 2006-07-31 15:59 ` Adrian Ulrich 2006-07-31 16:52 ` David Masover 2006-08-01 6:22 ` Jan Engelhardt 2 siblings, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-07-31 15:59 UTC (permalink / raw) To: Matthias Andree Cc: vonbrand, ipso, matthias.andree, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Hello Matthias, > This looks rather like an education issue rather than a technical limit. We aren't talking about the same issue: I was asking to do it on-the-fly. Umounting the filesystem, running e2fsck and resize2fs is something different ;-) > Which is untrue at least for Solaris, which allows resizing a life file > system. FreeBSD and Linux require an unmount. Correct: You can add more inodes to a Solaris UFS on-the-fly if you are lucky enough to have some free space available. A colleague of mine happened to create a ~300gb filesystem and started to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) to the new LUN. At about 70% the filesystem ran out of inodes; Not a big deal with VxFS because such a problem is fixable within seconds. What would have happened if he had used UFS? mkfs -G wouldn't work because he had no additional Diskspace left... *ouch*.. > Well, such "silly limitations"... looks like they are mostly hot air > spewn by marketroids that need to justify people spending money on their > new filesystem. Have you ever seen VxFS or WAFL in action? > But even then, I'd be interested to know if that's a real problem for systems > such as ZFS. ZFS uses 'dnodes'. The dnodes are allocated on demand from your available space so running out of [di]nodes is impossible. Great to see that Sun ships a state-of-the-art Filesystem with Solaris... I think linux should do the same... Regards, Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion @ 2006-07-31 15:59 ` Adrian Ulrich 0 siblings, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-07-31 15:59 UTC (permalink / raw) Cc: vonbrand, ipso, matthias.andree, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Hello Matthias, > This looks rather like an education issue rather than a technical limit. We aren't talking about the same issue: I was asking to do it on-the-fly. Umounting the filesystem, running e2fsck and resize2fs is something different ;-) > Which is untrue at least for Solaris, which allows resizing a life file > system. FreeBSD and Linux require an unmount. Correct: You can add more inodes to a Solaris UFS on-the-fly if you are lucky enough to have some free space available. A colleague of mine happened to create a ~300gb filesystem and started to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) to the new LUN. At about 70% the filesystem ran out of inodes; Not a big deal with VxFS because such a problem is fixable within seconds. What would have happened if he had used UFS? mkfs -G wouldn't work because he had no additional Diskspace left... *ouch*.. > Well, such "silly limitations"... looks like they are mostly hot air > spewn by marketroids that need to justify people spending money on their > new filesystem. Have you ever seen VxFS or WAFL in action? > But even then, I'd be interested to know if that's a real problem for systems > such as ZFS. ZFS uses 'dnodes'. The dnodes are allocated on demand from your available space so running out of [di]nodes is impossible. Great to see that Sun ships a state-of-the-art Filesystem with Solaris... I think linux should do the same... Regards, Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 15:59 ` Adrian Ulrich (?) @ 2006-07-31 16:22 ` Jan-Benedict Glaw 2006-07-31 16:44 ` David Masover ` (2 more replies) -1 siblings, 3 replies; 639+ messages in thread From: Jan-Benedict Glaw @ 2006-07-31 16:22 UTC (permalink / raw) To: Adrian Ulrich Cc: Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 610 bytes --] On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich <reiser4@blinkenlights.ch> wrote: > A colleague of mine happened to create a ~300gb filesystem and started > to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) > to the new LUN. At about 70% the filesystem ran out of inodes; Not a So preparation work wasn't done. MfG, JBG -- Jan-Benedict Glaw jbglaw@lug-owl.de +49-172-7608481 Signature of: Ich hatte in letzter Zeit ein bisschen viel Realitycheck. the second : Langsam möchte ich mal wieder weiterträumen können. [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 16:22 ` Jan-Benedict Glaw @ 2006-07-31 16:44 ` David Masover 2006-07-31 17:34 ` Bernd Eckenfels 2006-07-31 18:36 ` Jan-Benedict Glaw 2006-07-31 16:44 ` Rudy Zijlstra 2006-07-31 16:47 ` Dan Oglesby 2 siblings, 2 replies; 639+ messages in thread From: David Masover @ 2006-07-31 16:44 UTC (permalink / raw) To: Adrian Ulrich, Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Jan-Benedict Glaw wrote: > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich <reiser4@blinkenlights.ch> wrote: >> A colleague of mine happened to create a ~300gb filesystem and started >> to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) >> to the new LUN. At about 70% the filesystem ran out of inodes; Not a > > So preparation work wasn't done. So what? Yes, you need to do preparation. But it is really nice if the filesystem can do that work for you. Let me put it this way -- You're back in college, and it's time to write a thesis. You have a choice of software packages: Package A: You have to specify how many pages, and how many words, you're likely to use before you start typing. Guess too high, and you'll print out a bunch of blank pages at the end. Guess too low, and you'll run out of space and have to start over, copy and paste your document back in, and hope it gets all the formatting right, which it probably won't. Package B: Your document grows as you type. When it's time to print, only the pages you've actually written something on -- but all of the pages you've actually written something on -- are printed. All other things being equal, which would you choose? Which one seems more modern? Look, I understand the argument against ReiserFS v3 -- it has another limitation that you don't even know about. That other limitation is scary -- that's like being able to type as many words as you want, but once you type enough pages (no way of knowing how many), pages start randomly disappearing from the middle of your document. But the argument that no one cares about inode limits? Really, stop kidding yourselves. It's 2006. The limits are starting to look ridiculous. Just because they're workable doesn't mean we should have to live with them. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 16:44 ` David Masover @ 2006-07-31 17:34 ` Bernd Eckenfels 2006-07-31 18:36 ` Jan-Benedict Glaw 1 sibling, 0 replies; 639+ messages in thread From: Bernd Eckenfels @ 2006-07-31 17:34 UTC (permalink / raw) To: linux-kernel David Masover <ninja@slaphack.com> wrote: > Yes, you need to do preparation. But it is really nice if the > filesystem can do that work for you. The filesystem defaults are more than useable for most situations. You simply dont migrate a filesystem of that size without doing some calculation. Gruss Bernd ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 16:44 ` David Masover 2006-07-31 17:34 ` Bernd Eckenfels @ 2006-07-31 18:36 ` Jan-Benedict Glaw 1 sibling, 0 replies; 639+ messages in thread From: Jan-Benedict Glaw @ 2006-07-31 18:36 UTC (permalink / raw) To: David Masover Cc: Adrian Ulrich, Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 1729 bytes --] On Mon, 2006-07-31 11:44:25 -0500, David Masover <ninja@slaphack.com> wrote: > Jan-Benedict Glaw wrote: > > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich > > <reiser4@blinkenlights.ch> wrote: > > > A colleague of mine happened to create a ~300gb filesystem and started > > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) > > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a > > > > So preparation work wasn't done. > > Let me put it this way -- You're back in college, and it's time to write > a thesis. You have a choice of software packages: > > Package A: You have to specify how many pages, and how many words, > you're likely to use before you start typing. Guess too high, and > you'll print out a bunch of blank pages at the end. Guess too low, and > you'll run out of space and have to start over, copy and paste your > document back in, and hope it gets all the formatting right, which it > probably won't. > > Package B: Your document grows as you type. When it's time to print, > only the pages you've actually written something on -- but all of the > pages you've actually written something on -- are printed. > > All other things being equal, which would you choose? Which one seems > more modern? :) Well, given that TeX needs two (or even three!) runs to get all page references right, why should I choose MS Word, where you won't see that problem at all? MfG, JBG -- Jan-Benedict Glaw jbglaw@lug-owl.de +49-172-7608481 Signature of: If it doesn't work, force it. the second : If it breaks, it needed replacing anyway. [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 16:22 ` Jan-Benedict Glaw 2006-07-31 16:44 ` David Masover @ 2006-07-31 16:44 ` Rudy Zijlstra 2006-07-31 17:20 ` Jan-Benedict Glaw 2006-07-31 17:32 ` Jan-Benedict Glaw 2006-07-31 16:47 ` Dan Oglesby 2 siblings, 2 replies; 639+ messages in thread From: Rudy Zijlstra @ 2006-07-31 16:44 UTC (permalink / raw) To: Jan-Benedict Glaw Cc: Adrian Ulrich, Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote: > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich <reiser4@blinkenlights.ch> wrote: >> A colleague of mine happened to create a ~300gb filesystem and started >> to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) >> to the new LUN. At about 70% the filesystem ran out of inodes; Not a > > So preparation work wasn't done. > > MfG, JBG Of course you are right. Preparation work was not fully done. And using ext1 would also have been possible. I suspect you are still using ext1, cause with proper preparation it is perfectly usable. Cheers, Rudy ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 16:44 ` Rudy Zijlstra @ 2006-07-31 17:20 ` Jan-Benedict Glaw 2006-07-31 17:32 ` Jan-Benedict Glaw 1 sibling, 0 replies; 639+ messages in thread From: Jan-Benedict Glaw @ 2006-07-31 17:20 UTC (permalink / raw) To: Rudy Zijlstra Cc: Adrian Ulrich, Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 1357 bytes --] On Mon, 2006-07-31 18:44:33 +0200, Rudy Zijlstra <rudy@edsons.demon.nl> wrote: > On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote: > > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich > > <reiser4@blinkenlights.ch> wrote: > > > A colleague of mine happened to create a ~300gb filesystem and started > > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) > > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a > > > > So preparation work wasn't done. > > Of course you are right. Preparation work was not fully done. And using > ext1 would also have been possible. I suspect you are still using ext1, > cause with proper preparation it is perfectly usable. Nope, I'm a quite happy ext3 user for quite divergent workloads. Starting with storing loads of small files in a hugh number of directories (mkfs -T news) over hugh numbers of files in few directories (mkfs -O dir_index) to only a small number of really hugh files (mkfs -T largefile4). All that I still need to catch up with is 4TB sized files. But I guess this is really an uncommon workload, but ISTR that somebody even works on this. MfG, JBG -- Jan-Benedict Glaw jbglaw@lug-owl.de +49-172-7608481 Signature of: Don't believe in miracles: Rely on them! the second : [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 16:44 ` Rudy Zijlstra 2006-07-31 17:20 ` Jan-Benedict Glaw @ 2006-07-31 17:32 ` Jan-Benedict Glaw 2006-07-31 17:46 ` Dan Oglesby ` (2 more replies) 1 sibling, 3 replies; 639+ messages in thread From: Jan-Benedict Glaw @ 2006-07-31 17:32 UTC (permalink / raw) To: Rudy Zijlstra Cc: Adrian Ulrich, Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 2352 bytes --] On Mon, 2006-07-31 18:44:33 +0200, Rudy Zijlstra <rudy@edsons.demon.nl> wrote: > On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote: > > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich > > <reiser4@blinkenlights.ch> wrote: > > > A colleague of mine happened to create a ~300gb filesystem and started > > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) > > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a > > > > So preparation work wasn't done. > > Of course you are right. Preparation work was not fully done. And using > ext1 would also have been possible. I suspect you are still using ext1, > cause with proper preparation it is perfectly usable. Oh, and before people start laughing at me, here are some personal or friend's experiences with different filesystems: * reiser3: A HDD containing a reiser3 filesystem was tried to be booted on a machine that fucked up DMA writes. Fortunately, it crashed really soon (right after going for read-write.) After rebooting the HDD on a sane PeeCee, it refused to boot. Starting off some rescue system showed an _empty_ root filesystem. * A friend's XFS data partition (portable USB/FireWire HDD) once crashed due to being hot-unplugged off the USB. The in-kernel XFS driver refused to mount that thing again, and the tools also refused to fix any errors. (Don't ask, no details at my hands...) * JFS just always worked for me. Though I've never ever had a broken HDD where it (or it's tools) could have shown how well-done they were, so from a crash-recovery point of view, it's untested. * Being a regular ext3 user, I had lots of broken HDDs containing ext3 filesystems. For every single case, it has been easy fixing the filesystem after cloning. Just _once_, fsck wasn't able to fix something, so I did it manually with some disk editor. This worked well because the on-disk data structures are actually as simple as they are. ext3 always worked well for me, so why should I abandon it? MfG, JBG -- Jan-Benedict Glaw jbglaw@lug-owl.de +49-172-7608481 Signature of: If it doesn't work, force it. the second : If it breaks, it needed replacing anyway. [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 17:32 ` Jan-Benedict Glaw @ 2006-07-31 17:46 ` Dan Oglesby 2006-07-31 18:11 ` Matthias Andree 2006-07-31 21:21 ` Łukasz Mierzwa 2 siblings, 0 replies; 639+ messages in thread From: Dan Oglesby @ 2006-07-31 17:46 UTC (permalink / raw) To: Jan-Benedict Glaw Cc: Rudy Zijlstra, Adrian Ulrich, Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On Mon, 2006-07-31 at 19:32 +0200, Jan-Benedict Glaw wrote: > On Mon, 2006-07-31 18:44:33 +0200, Rudy Zijlstra <rudy@edsons.demon.nl> wrote: > > On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote: > > > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich > > > <reiser4@blinkenlights.ch> wrote: > > > > A colleague of mine happened to create a ~300gb filesystem and started > > > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) > > > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a > > > > > > So preparation work wasn't done. > > > > Of course you are right. Preparation work was not fully done. And using > > ext1 would also have been possible. I suspect you are still using ext1, > > cause with proper preparation it is perfectly usable. > > Oh, and before people start laughing at me, here are some personal or > friend's experiences with different filesystems: > > * reiser3: A HDD containing a reiser3 filesystem was tried to be > booted on a machine that fucked up DMA writes. Fortunately, it > crashed really soon (right after going for read-write.) After > rebooting the HDD on a sane PeeCee, it refused to boot. Starting > off some rescue system showed an _empty_ root filesystem. > > * A friend's XFS data partition (portable USB/FireWire HDD) once > crashed due to being hot-unplugged off the USB. The in-kernel XFS > driver refused to mount that thing again, and the tools also > refused to fix any errors. (Don't ask, no details at my hands...) > > * JFS just always worked for me. Though I've never ever had a broken > HDD where it (or it's tools) could have shown how well-done they > were, so from a crash-recovery point of view, it's untested. > > * Being a regular ext3 user, I had lots of broken HDDs containing > ext3 filesystems. For every single case, it has been easy fixing > the filesystem after cloning. Just _once_, fsck wasn't able to fix > something, so I did it manually with some disk editor. This worked > well because the on-disk data structures are actually as simple as > they are. > > ext3 always worked well for me, so why should I abandon it? > > MfG, JBG I've lost EXT2 and EXT3 filesystems from machines with no bad hardware (power loss during writes). I've recovered all but a handful of files from a RAID-5 array running ReiserFS v3 that had two drives fail in rapid succession with bad sectors. Sometimes you're lucky, sometimes you're not. --Dan ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 17:32 ` Jan-Benedict Glaw 2006-07-31 17:46 ` Dan Oglesby @ 2006-07-31 18:11 ` Matthias Andree 2006-07-31 18:43 ` Jan-Benedict Glaw 2006-07-31 21:21 ` Łukasz Mierzwa 2 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-07-31 18:11 UTC (permalink / raw) To: Rudy Zijlstra, Adrian Ulrich, Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Jan-Benedict Glaw schrieb am 2006-07-31: > On Mon, 2006-07-31 18:44:33 +0200, Rudy Zijlstra <rudy@edsons.demon.nl> wrote: > > On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote: > > > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich > > > <reiser4@blinkenlights.ch> wrote: > > > > A colleague of mine happened to create a ~300gb filesystem and started > > > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) > > > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a > > > > > > So preparation work wasn't done. > > > > Of course you are right. Preparation work was not fully done. And using > > ext1 would also have been possible. I suspect you are still using ext1, > > cause with proper preparation it is perfectly usable. > > Oh, and before people start laughing at me, here are some personal or > friend's experiences with different filesystems: > > * reiser3: A HDD containing a reiser3 filesystem was tried to be > booted on a machine that fucked up DMA writes. Fortunately, it > crashed really soon (right after going for read-write.) After > rebooting the HDD on a sane PeeCee, it refused to boot. Starting > off some rescue system showed an _empty_ root filesystem. Massive hardware problems don't count. ext2/ext3 doesn't look much better in such cases. I had a machine with RAM gone bad (no ECC - I wonder what idiot ordered a machine without ECC for a server, but anyways) and it fucked up every 64th bit - only in a certain region. Guess what happened to the fs when it went into e2fsck after a reboot. Boom. Same with a dead DPTA that lost every 16th block or so, the rescue in the first case was swapping the RAM and "amrecover" and in the second swapping the drive and "dsmc restore". OTOH, kernel panics on bad blocks are a no-no of course. > * A friend's XFS data partition (portable USB/FireWire HDD) once > crashed due to being hot-unplugged off the USB. The in-kernel XFS > driver refused to mount that thing again, and the tools also > refused to fix any errors. (Don't ask, no details at my hands...) Don't use write caches then. (Though I've seen NUL-filled blocks in new files or appended to files after in 2001 or 2002.) > * JFS just always worked for me. Though I've never ever had a broken > HDD where it (or it's tools) could have shown how well-done they > were, so from a crash-recovery point of view, it's untested. SUSE removed JFS support from their installation tool for "technical reasons" they didn't specify in the release notes. Whatever. > ext3 always worked well for me, so why should I abandon it? Plus, it and its tools are maintained. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 18:11 ` Matthias Andree @ 2006-07-31 18:43 ` Jan-Benedict Glaw 2006-07-31 19:17 ` Clay Barnes 2006-07-31 22:17 ` Matthias Andree 0 siblings, 2 replies; 639+ messages in thread From: Jan-Benedict Glaw @ 2006-07-31 18:43 UTC (permalink / raw) To: Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 1447 bytes --] On Mon, 2006-07-31 20:11:20 +0200, Matthias Andree <matthias.andree@gmx.de> wrote: > Jan-Benedict Glaw schrieb am 2006-07-31: > > * reiser3: A HDD containing a reiser3 filesystem was tried to be > > booted on a machine that fucked up DMA writes. Fortunately, it > > crashed really soon (right after going for read-write.) After > > rebooting the HDD on a sane PeeCee, it refused to boot. Starting > > off some rescue system showed an _empty_ root filesystem. > > Massive hardware problems don't count. ext2/ext3 doesn't look much better in > such cases. I had a machine with RAM gone bad (no ECC - I wonder what They do! Very much, actually. These happen In Real Life, so I have to pay attention to them. Once you're in setups with > 10000 machines, everything counts. At some certain point, you can even use HDD's temperature sensors in old machines to diagnose dead fans. Everything that eases recovery for whatever reason is something you have to pay attention to. The simplicity of ext{2,3} is something I really fail to find proper words for. As well as the really good fsck. Once seen a SIGSEGV'ing fsck, you really don't want to go there. MfG, JBG -- Jan-Benedict Glaw jbglaw@lug-owl.de +49-172-7608481 Signature of: Eine Freie Meinung in einem Freien Kopf the second : für einen Freien Staat voll Freier Bürger. [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 18:43 ` Jan-Benedict Glaw @ 2006-07-31 19:17 ` Clay Barnes 2006-07-31 19:29 ` Jan-Benedict Glaw 2006-07-31 19:42 ` Alan Cox 2006-07-31 22:17 ` Matthias Andree 1 sibling, 2 replies; 639+ messages in thread From: Clay Barnes @ 2006-07-31 19:17 UTC (permalink / raw) To: Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On 20:43 Mon 31 Jul , Jan-Benedict Glaw wrote: > On Mon, 2006-07-31 20:11:20 +0200, Matthias Andree <matthias.andree@gmx.de> wrote: > > Jan-Benedict Glaw schrieb am 2006-07-31: > > > * reiser3: A HDD containing a reiser3 filesystem was tried to be > > > booted on a machine that fucked up DMA writes. Fortunately, it > > > crashed really soon (right after going for read-write.) After > > > rebooting the HDD on a sane PeeCee, it refused to boot. Starting > > > off some rescue system showed an _empty_ root filesystem. > > > > Massive hardware problems don't count. ext2/ext3 doesn't look much better in > > such cases. I had a machine with RAM gone bad (no ECC - I wonder what > > They do! Very much, actually. These happen In Real Life, so I have to > pay attention to them. Once you're in setups with > 10000 machines, > everything counts. At some certain point, you can even use HDD's > temperature sensors in old machines to diagnose dead fans. I think what he meant was that it is unfair to blame reiser3 for data loss in a massive failure situation as a case example by itself. What would be more appropriate (and fair) is saying something along the lines of "in a machine with DMA failure, reiser3 hosed a drive, while ext3 only lost x data (or nothing). In my situation, every bit of massive failure robustness counts... " This of course assumes you actually had the *exact* same problem with hardware under ext3, pretty much in every detail. Of course, so many subtleties interact in massive ways with filesystems and hardware problems, so we have to keep in mind how much difference that can make (all the difference in the world), and that, really, without a statistically significant sample size and some real analysis, hardware failure comparisons are impossible to do fairly. Of course, if ext3 were proven to be more robust against failures, I bet the reiser team would be very interested in all the forensic data you can offer, since, from what I've seen, they are always trying to make reiser as good as possible---in speed, flexability, *and* robustness. If they didn't care about the last, they'd benifit greatly from not journaling! :-P --Clay > > Everything that eases recovery for whatever reason is something you > have to pay attention to. The simplicity of ext{2,3} is something I > really fail to find proper words for. As well as the really good fsck. > Once seen a SIGSEGV'ing fsck, you really don't want to go there. > > MfG, JBG > > -- > Jan-Benedict Glaw jbglaw@lug-owl.de +49-172-7608481 > Signature of: Eine Freie Meinung in einem Freien Kopf > the second : für einen Freien Staat voll Freier Bürger. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 19:17 ` Clay Barnes @ 2006-07-31 19:29 ` Jan-Benedict Glaw 2006-07-31 20:00 ` David Masover 2006-07-31 21:14 ` the " 'official' point of view" expressed by kernelnewbies.org regarding " Bernd Schubert 2006-07-31 19:42 ` Alan Cox 1 sibling, 2 replies; 639+ messages in thread From: Jan-Benedict Glaw @ 2006-07-31 19:29 UTC (permalink / raw) To: Clay Barnes Cc: Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 1426 bytes --] On Mon, 2006-07-31 12:17:12 -0700, Clay Barnes <clay.barnes@gmail.com> wrote: > On 20:43 Mon 31 Jul , Jan-Benedict Glaw wrote: > > On Mon, 2006-07-31 20:11:20 +0200, Matthias Andree <matthias.andree@gmx.de> wrote: > > > Jan-Benedict Glaw schrieb am 2006-07-31: [Crippled DMA writes] > > > Massive hardware problems don't count. ext2/ext3 doesn't look much better in > > > such cases. I had a machine with RAM gone bad (no ECC - I wonder what > > > > They do! Very much, actually. These happen In Real Life, so I have to > > I think what he meant was that it is unfair to blame reiser3 for data > loss in a massive failure situation as a case example by itself. What Crippling a few KB of metadata in the ext{2,3} case probably wouldn't fobar the filesystem... > failure robustness counts... " This of course assumes you actually had > the *exact* same problem with hardware under ext3, pretty much in every > detail. Of course, so many subtleties interact in massive ways with The point is that it's quite hard to really fuck up ext{2,3} with only some KB being written while it seems (due to the fragile^Wsophisticated on-disk data structures) that it's just easy to kill a reiser3 filesystem. MfG, JBG -- Jan-Benedict Glaw jbglaw@lug-owl.de +49-172-7608481 Signature of: Zensur im Internet? Nein danke! the second : [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 19:29 ` Jan-Benedict Glaw @ 2006-07-31 20:00 ` David Masover 2006-07-31 20:53 ` the " 'official' point of view" expressed by kernelnewbies.orgregarding " David Lang 2006-07-31 21:14 ` the " 'official' point of view" expressed by kernelnewbies.org regarding " Bernd Schubert 1 sibling, 1 reply; 639+ messages in thread From: David Masover @ 2006-07-31 20:00 UTC (permalink / raw) To: Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Jan-Benedict Glaw wrote: > On Mon, 2006-07-31 12:17:12 -0700, Clay Barnes <clay.barnes@gmail.com> wrote: >> On 20:43 Mon 31 Jul , Jan-Benedict Glaw wrote: >>> On Mon, 2006-07-31 20:11:20 +0200, Matthias Andree <matthias.andree@gmx.de> wrote: >>>> Jan-Benedict Glaw schrieb am 2006-07-31: > [Crippled DMA writes] >>>> Massive hardware problems don't count. ext2/ext3 doesn't look much better in >>>> such cases. I had a machine with RAM gone bad (no ECC - I wonder what >>> They do! Very much, actually. These happen In Real Life, so I have to >> I think what he meant was that it is unfair to blame reiser3 for data >> loss in a massive failure situation as a case example by itself. What > > Crippling a few KB of metadata in the ext{2,3} case probably wouldn't > fobar the filesystem... Probably. By the time a few KB of metadata are corrupted, I'm reaching for my backup. I don't care what filesystem it is or how easy it is to edit the on-disk structures. This isn't to say that having robust on-disk structures isn't a good thing. I have no idea how Reiser4 will hold up either way. But ultimately, what you want is the journaling (so power failure / crashes still leave you in an OK state), backups (so when blocks go bad, you don't care), and performance (so you can spend less money on hardware and more money on backup hardware). ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.orgregarding reiser4 inclusion 2006-07-31 20:00 ` David Masover @ 2006-07-31 20:53 ` David Lang 2006-07-31 21:16 ` David Masover 0 siblings, 1 reply; 639+ messages in thread From: David Lang @ 2006-07-31 20:53 UTC (permalink / raw) To: David Masover Cc: Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On Mon, 31 Jul 2006, David Masover wrote: > Probably. By the time a few KB of metadata are corrupted, I'm reaching for > my backup. I don't care what filesystem it is or how easy it is to edit the > on-disk structures. > > This isn't to say that having robust on-disk structures isn't a good thing. > I have no idea how Reiser4 will hold up either way. But ultimately, what you > want is the journaling (so power failure / crashes still leave you in an OK > state), backups (so when blocks go bad, you don't care), and performance (so > you can spend less money on hardware and more money on backup hardware). please read the discussion that took place at the filesystem summit a couple weeks ago (available on lwn.net) one of the things that they pointed out there is that as disks get larger the ratio of bad spots per Gig of storage is remaining about the same. As is the rate of failures per Gig of storage. As a result of this the idea of only running on perfect disks that never have any failures is becomeing significantly less realistic, instead you need to take measures to survive in the face of minor corruption (including robust filesystems, raid, etc) David Lang ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.orgregarding reiser4 inclusion 2006-07-31 20:53 ` the " 'official' point of view" expressed by kernelnewbies.orgregarding " David Lang @ 2006-07-31 21:16 ` David Masover 0 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-07-31 21:16 UTC (permalink / raw) To: David Lang Cc: Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list David Lang wrote: > On Mon, 31 Jul 2006, David Masover wrote: > >> Probably. By the time a few KB of metadata are corrupted, I'm >> reaching for my backup. I don't care what filesystem it is or how >> easy it is to edit the on-disk structures. >> >> This isn't to say that having robust on-disk structures isn't a good >> thing. I have no idea how Reiser4 will hold up either way. But >> ultimately, what you want is the journaling (so power failure / >> crashes still leave you in an OK state), backups (so when blocks go >> bad, you don't care), and performance (so you can spend less money on >> hardware and more money on backup hardware). > > please read the discussion that took place at the filesystem summit a > couple weeks ago (available on lwn.net) I think I will, but I don't have the time today, so... > one of the things that they pointed out there is that as disks get > larger the ratio of bad spots per Gig of storage is remaining about the > same. As is the rate of failures per Gig of storage. > > As a result of this the idea of only running on perfect disks that never > have any failures is becomeing significantly less realistic, instead you > need to take measures to survive in the face of minor corruption > (including robust filesystems, raid, etc) RAID seems a much more viable solution to me. That and cheaper storage, so that you can actually afford to replace the disk when you find corruption, or have more redundancy so you don't have to. Because "robust filesystems" is nice in theory, but in practice, you really never know what will get hit. RAID, at least, is predictable. When it's not: Backups. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 19:29 ` Jan-Benedict Glaw 2006-07-31 20:00 ` David Masover @ 2006-07-31 21:14 ` Bernd Schubert 2006-08-01 14:28 ` Horst H. von Brand 1 sibling, 1 reply; 639+ messages in thread From: Bernd Schubert @ 2006-07-31 21:14 UTC (permalink / raw) To: reiserfs-list Cc: Jan-Benedict Glaw, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel On Monday 31 July 2006 21:29, Jan-Benedict Glaw wrote: > > The point is that it's quite hard to really fuck up ext{2,3} with only > some KB being written while it seems (due to the > fragile^Wsophisticated on-disk data structures) that it's just easy to > kill a reiser3 filesystem. > Well, I was once very 'luckily' and after a system crash (*) e2fsck put all files into lost+found. Sure, I never experienced this again, but I also never experienced something like this with reiserfs. So please, stop this kind of FUD against reiser3.6. While filesystem speed is nice, it also would be great if reiser4.x would be very robust against any kind of hardware failures. (*) The problem was a specific mainboard + video-card + driver combination. As soon as X started up, the system entirely crashed. I don't believe many bytes were written, but I also can't prove it. -- Bernd Schubert PCI / Theoretische Chemie Universität Heidelberg INF 229 69120 Heidelberg ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 21:14 ` the " 'official' point of view" expressed by kernelnewbies.org regarding " Bernd Schubert @ 2006-08-01 14:28 ` Horst H. von Brand 2006-08-01 14:52 ` Adrian Ulrich 2006-08-01 16:57 ` David Masover 0 siblings, 2 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-08-01 14:28 UTC (permalink / raw) To: Bernd Schubert Cc: reiserfs-list, Jan-Benedict Glaw, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel Bernd Schubert <bernd-schubert@gmx.de> wrote: > On Monday 31 July 2006 21:29, Jan-Benedict Glaw wrote: > > The point is that it's quite hard to really fuck up ext{2,3} with only > > some KB being written while it seems (due to the > > fragile^Wsophisticated on-disk data structures) that it's just easy to > > kill a reiser3 filesystem. > Well, I was once very 'luckily' and after a system crash (*) e2fsck put > all files into lost+found. Sure, I never experienced this again, but I > also never experienced something like this with reiserfs. So please, stop > this kind of FUD against reiser3.6. It isn't FUD. One data point doesn't allow you to draw conclusions. Yes, I've seen/heard of ext2/ext3 failures and data loss too. But at least the same number for ReiserFS. And I know it is outnumbered 10 to 1 or so in my sample, so that would indicate at a 10 fold higher probability of catastrophic data loss, other factors mostly the same. > While filesystem speed is nice, it also would be great if reiser4.x would be > very robust against any kind of hardware failures. Can't have both. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 14:28 ` Horst H. von Brand @ 2006-08-01 14:52 ` Adrian Ulrich 2006-08-01 16:57 ` David Masover 1 sibling, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-08-01 14:52 UTC (permalink / raw) To: Horst H. von Brand Cc: bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel > > While filesystem speed is nice, it also would be great if reiser4.x would be > > very robust against any kind of hardware failures. > > Can't have both. ..and some people simply don't care about this: If you are running a 'big' Storage-System with battery protected WriteCache, Mirroring between 2 Datacenters, snapshotting.. etc.. you don't need your filesystem beeing super-robust against bad sectors and such stuff because: a) You've paid enough money to let the storage care about Hardware issues. b) If your storage is on fire you can do a failover using the mirror. c) And if someone ran dd if=/dev/urandom of=/dev/sda you could even rollback your Snapshot. (Btw: i did this once to a Reiser4 filesystem (overwritten about 1.2gb). fsck.reiser4 --rebuild-sb was able to fix it.) ..but what you really need is a flexible and **fast** filesystem: Like Reiser4. (Yeah.. yeah.. i know: ext3 is also flexible and fast.. but Reiser4 simply is *MUCH* faster than ext3 for 'my' workload/application). Regards, Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion @ 2006-08-01 14:52 ` Adrian Ulrich 0 siblings, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-08-01 14:52 UTC (permalink / raw) Cc: bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel > > While filesystem speed is nice, it also would be great if reiser4.x would be > > very robust against any kind of hardware failures. > > Can't have both. ..and some people simply don't care about this: If you are running a 'big' Storage-System with battery protected WriteCache, Mirroring between 2 Datacenters, snapshotting.. etc.. you don't need your filesystem beeing super-robust against bad sectors and such stuff because: a) You've paid enough money to let the storage care about Hardware issues. b) If your storage is on fire you can do a failover using the mirror. c) And if someone ran dd if=/dev/urandom of=/dev/sda you could even rollback your Snapshot. (Btw: i did this once to a Reiser4 filesystem (overwritten about 1.2gb). fsck.reiser4 --rebuild-sb was able to fix it.) ..but what you really need is a flexible and **fast** filesystem: Like Reiser4. (Yeah.. yeah.. i know: ext3 is also flexible and fast.. but Reiser4 simply is *MUCH* faster than ext3 for 'my' workload/application). Regards, Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 14:52 ` Adrian Ulrich (?) @ 2006-08-01 15:29 ` Alan Cox 2006-08-01 16:44 ` David Masover ` (2 more replies) -1 siblings, 3 replies; 639+ messages in thread From: Alan Cox @ 2006-08-01 15:29 UTC (permalink / raw) To: Adrian Ulrich Cc: Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel Ar Maw, 2006-08-01 am 16:52 +0200, ysgrifennodd Adrian Ulrich: > WriteCache, Mirroring between 2 Datacenters, snapshotting.. etc.. > you don't need your filesystem beeing super-robust against bad sectors > and such stuff because: You do it turns out. Its becoming an issue more and more that the sheer amount of storage means that the undetected error rate from disks, hosts, memory, cables and everything else is rising. There has been a great deal of discussion about this at the filesystem and kernel summits - and data is getting kicked the way of networking - end to end not reliability in the middle. The sort of changes this needs hit the block layer and ever fs. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 15:29 ` Alan Cox @ 2006-08-01 16:44 ` David Masover 2006-08-01 17:04 ` Gregory Maxwell 2006-08-01 17:19 ` Alan Cox 2006-08-01 18:11 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Adrian Ulrich 2006-08-01 18:21 ` Ric Wheeler 2 siblings, 2 replies; 639+ messages in thread From: David Masover @ 2006-08-01 16:44 UTC (permalink / raw) To: Alan Cox Cc: Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel Alan Cox wrote: > Ar Maw, 2006-08-01 am 16:52 +0200, ysgrifennodd Adrian Ulrich: >> WriteCache, Mirroring between 2 Datacenters, snapshotting.. etc.. >> you don't need your filesystem beeing super-robust against bad sectors >> and such stuff because: > > You do it turns out. Its becoming an issue more and more that the sheer > amount of storage means that the undetected error rate from disks, > hosts, memory, cables and everything else is rising. Yikes. Undetected. Wait, what? Disks, at least, would be protected by RAID. Are you telling me RAID won't detect such an error? It just seems wholly alien to me that errors would go undetected, and we're OK with that, so long as our filesystems are robust enough. If it's an _undetected_ error, doesn't that cause way more problems (impossible problems) than FS corruption? Ok, your FS is fine -- but now your bank database shows $1k less on random accounts -- is that ok? > There has been a great deal of discussion about this at the filesystem > and kernel summits - and data is getting kicked the way of networking - > end to end not reliability in the middle. Sounds good, but I've never let discussions by people smarter than me prevent me from asking the stupid questions. > The sort of changes this needs hit the block layer and ever fs. Seems it would need to hit every application also... ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 16:44 ` David Masover @ 2006-08-01 17:04 ` Gregory Maxwell 2006-08-01 12:01 ` Hans Reiser 2006-08-01 17:41 ` David Masover 2006-08-01 17:19 ` Alan Cox 1 sibling, 2 replies; 639+ messages in thread From: Gregory Maxwell @ 2006-08-01 17:04 UTC (permalink / raw) To: David Masover Cc: Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel On 8/1/06, David Masover <ninja@slaphack.com> wrote: > Yikes. Undetected. > > Wait, what? Disks, at least, would be protected by RAID. Are you > telling me RAID won't detect such an error? Unless the disk ECC catches it raid won't know anything is wrong. This is why ZFS offers block checksums... it can then try all the permutations of raid regens to find a solution which gives the right checksum. Every level of the system must be paranoid and take measure to avoid corruption if the system is to avoid it... it's a tough problem. It seems that the ZFS folks have addressed this challenge by building as much of what is classically separate layers into one part. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 17:04 ` Gregory Maxwell @ 2006-08-01 12:01 ` Hans Reiser 2006-08-01 17:41 ` David Masover 1 sibling, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-08-01 12:01 UTC (permalink / raw) To: Gregory Maxwell Cc: David Masover, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel Gregory Maxwell wrote: > This is why ZFS offers block checksums... it can then try all the > permutations of raid regens to find a solution which gives the right > checksum. > ZFS performance is pretty bad in the only benchmark I have seen of it. Does anyone have serious benchmarks of it? I suspect that our compression plugin (with ecc) will outperform it. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 17:04 ` Gregory Maxwell 2006-08-01 12:01 ` Hans Reiser @ 2006-08-01 17:41 ` David Masover 2006-08-01 18:14 ` Adrian Ulrich 1 sibling, 1 reply; 639+ messages in thread From: David Masover @ 2006-08-01 17:41 UTC (permalink / raw) To: Gregory Maxwell Cc: Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel Gregory Maxwell wrote: > On 8/1/06, David Masover <ninja@slaphack.com> wrote: >> Yikes. Undetected. >> >> Wait, what? Disks, at least, would be protected by RAID. Are you >> telling me RAID won't detect such an error? > > Unless the disk ECC catches it raid won't know anything is wrong. > > This is why ZFS offers block checksums... it can then try all the > permutations of raid regens to find a solution which gives the right > checksum. Isn't there a way to do this at the block layer? Something in device-mapper? > Every level of the system must be paranoid and take measure to avoid > corruption if the system is to avoid it... it's a tough problem. It > seems that the ZFS folks have addressed this challenge by building as > much of what is classically separate layers into one part. Sounds like bad design to me, and I can point to the antipattern, but what do I know? ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 17:41 ` David Masover @ 2006-08-01 18:14 ` Adrian Ulrich 0 siblings, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-08-01 18:14 UTC (permalink / raw) To: David Masover Cc: gmaxwell, alan, vonbrand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel > > This is why ZFS offers block checksums... it can then try all the > > permutations of raid regens to find a solution which gives the right > > checksum. > > Isn't there a way to do this at the block layer? Something in > device-mapper? Remember: Suns new Filesystem + Suns new Volume Manager = ZFS ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 16:44 ` David Masover 2006-08-01 17:04 ` Gregory Maxwell @ 2006-08-01 17:19 ` Alan Cox 2006-08-01 11:36 ` Hans Reiser 2006-08-01 17:40 ` David Masover 1 sibling, 2 replies; 639+ messages in thread From: Alan Cox @ 2006-08-01 17:19 UTC (permalink / raw) To: David Masover Cc: Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel Ar Maw, 2006-08-01 am 11:44 -0500, ysgrifennodd David Masover: > Yikes. Undetected. > > Wait, what? Disks, at least, would be protected by RAID. Are you > telling me RAID won't detect such an error? Yes. RAID deals with the case where a device fails. RAID 1 with 2 disks can in theory detect an internal inconsistency but cannot fix it. > we're OK with that, so long as our filesystems are robust enough. If > it's an _undetected_ error, doesn't that cause way more problems > (impossible problems) than FS corruption? Ok, your FS is fine -- but > now your bank database shows $1k less on random accounts -- is that ok? Not really no. Your bank is probably using a machine (hopefully using a machine) with ECC memory, ECC cache and the like. The UDMA and SATA storage subsystems use CRC checksums between the controller and the device. SCSI uses various similar systems - some older ones just use a parity bit so have only a 50/50 chance of noticing a bit error. Similarly the media itself is recorded with a lot of FEC (forward error correction) so will spot most changes. Unfortunately when you throw this lot together with astronomical amounts of data you get burned now and then, especially as most systems are not using ECC ram, do not have ECC on the CPU registers and may not even have ECC on the caches in the disks. > > The sort of changes this needs hit the block layer and ever fs. > > Seems it would need to hit every application also... Depending how far you propogate it. Someone people working with huge data sets already write and check user level CRC values for this reason (in fact bitkeeper does it for one example). It should be relatively cheap to get much of that benefit without doing application to application just as TCP gets most of its benefit without going app to app. Alan ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 17:19 ` Alan Cox @ 2006-08-01 11:36 ` Hans Reiser 2006-08-01 17:40 ` David Masover 1 sibling, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-08-01 11:36 UTC (permalink / raw) To: Alan Cox, Vitaly Fertman Cc: David Masover, Adrian Ulrich, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel Alan, I have seen only anecdotal evidence against reiserfsck, and I have seen formal tests from Vitaly (which it seems a user has replicated) where our fsck did better than ext3s. Note that these tests are of the latest fsck from us: I am sure everyone understands that it takes time for an fsck to mature, and that our early fsck's were poor. I will also say the V4's fsck is more robust than V3's because we made disk format changes specifically to help fsck. Now I am not dismissing your anecdotes as I will never dismiss data I have not seen, and it sounds like you have seen more data than most people, but I must dismiss your explanation of them. Being able to throw away all of the tree but the leaves and twigs with extent pointers and rebuild all of it makes V4 very robust, more so than ext3. This business of inodes not moving, I don't see what the advantage is, we can lose the directory entry and rebuild just as well as ext3, probably better because we can at least figure out what directory it was in. Vitaly can say all of this more expertly than I.... Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 17:19 ` Alan Cox 2006-08-01 11:36 ` Hans Reiser @ 2006-08-01 17:40 ` David Masover 2006-08-01 19:27 ` Krzysztof Halasa 2006-08-03 13:58 ` Matthias Andree 1 sibling, 2 replies; 639+ messages in thread From: David Masover @ 2006-08-01 17:40 UTC (permalink / raw) To: Alan Cox Cc: Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel Alan Cox wrote: > Ar Maw, 2006-08-01 am 11:44 -0500, ysgrifennodd David Masover: >> Yikes. Undetected. >> >> Wait, what? Disks, at least, would be protected by RAID. Are you >> telling me RAID won't detect such an error? > > Yes. > > RAID deals with the case where a device fails. RAID 1 with 2 disks can > in theory detect an internal inconsistency but cannot fix it. Still, if it does that, that should be enough. The scary part wasn't that there's an internal inconsistency, but that you wouldn't know. And it can fix it if you can figure out which disk went. Or give it 3 disks and it should be entirely automatic -- admin gets paged, admin hotswaps in a new disk, done. >> we're OK with that, so long as our filesystems are robust enough. If >> it's an _undetected_ error, doesn't that cause way more problems >> (impossible problems) than FS corruption? Ok, your FS is fine -- but >> now your bank database shows $1k less on random accounts -- is that ok? > > Not really no. Your bank is probably using a machine (hopefully using a > machine) with ECC memory, ECC cache and the like. The UDMA and SATA > storage subsystems use CRC checksums between the controller and the > device. SCSI uses various similar systems - some older ones just use a > parity bit so have only a 50/50 chance of noticing a bit error. > > Similarly the media itself is recorded with a lot of FEC (forward error > correction) so will spot most changes. > > Unfortunately when you throw this lot together with astronomical amounts > of data you get burned now and then, especially as most systems are not > using ECC ram, do not have ECC on the CPU registers and may not even > have ECC on the caches in the disks. It seems like this is the place to fix it, not the software. If the software can fix it easily, great. But I'd much rather rely on the hardware looking after itself, because when hardware goes bad, all bets are off. Specifically, it seems like you do mention lots of hardware solutions, that just aren't always used. It seems like storage itself is getting cheap enough that it's time to step back a year or two in Moore's Law to get the reliability. >>> The sort of changes this needs hit the block layer and ever fs. >> Seems it would need to hit every application also... > > Depending how far you propogate it. Someone people working with huge > data sets already write and check user level CRC values for this reason > (in fact bitkeeper does it for one example). It should be relatively > cheap to get much of that benefit without doing application to > application just as TCP gets most of its benefit without going app to > app. And yet, if you can do that, I'd suspect you can, should, must do it at a lower level than the FS. Again, FS robustness is good, but if the disk itself is going, what good is having your directory (mostly) intact if the files themselves have random corruptions? If you can't trust the disk, you need more than just an FS which can mostly survive hardware failure. You also need the FS itself (or maybe the block layer) to support bad block relocation and all that good stuff, or you need your apps designed to do that job by themselves. It just doesn't make sense to me to do this at the FS level. You mention TCP -- ok, but if TCP is doing its job, I shouldn't also need to implement checksums and other robustness at the protocol layer (http, ftp, ssh), should I? Because in this analogy, it looks like TCP is the "block layer" and a protocol is the "fs". As I understand it, TCP only lets the protocol/application know when something's seriously FUBARed and it has to drop the connection. Similarly, the FS (and the apps) shouldn't have to know about hardware problems until it really can't do anything about it anymore, at which point the right thing to do is for the FS and apps to go "oh shit" and drop what they're doing, and the admin replaces hardware and restores from backup. Or brings a backup server online, or... I guess my main point was that _undetected_ problems are serious, but if you can detect them, and you have at least a bit of redundancy, you should be good. For instance, if your RAID reports errors that it can't fix, you bring that server down and let the backup server run. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 17:40 ` David Masover @ 2006-08-01 19:27 ` Krzysztof Halasa 2006-08-03 13:58 ` Matthias Andree 1 sibling, 0 replies; 639+ messages in thread From: Krzysztof Halasa @ 2006-08-01 19:27 UTC (permalink / raw) To: David Masover Cc: Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel David Masover <ninja@slaphack.com> writes: >> RAID deals with the case where a device fails. RAID 1 with 2 disks >> can >> in theory detect an internal inconsistency but cannot fix it. > > Still, if it does that, that should be enough. The scary part wasn't > that there's an internal inconsistency, but that you wouldn't know. RAID1 can do that in theory but it practice there is no verification, so the other disk can perform another read simultaneously (thus increasing performance). Some high-end systems, maybe. That would be hardly economical. Per-block checksums (like used by the ZFS) are different story, they add only little additional load. > And it can fix it if you can figure out which disk went. Or give it 3 > disks and it should be entirely automatic -- admin gets paged, admin > hotswaps in a new disk, done. Yep, that could be done. Or with 2 disks with block checksums. Actually, while I don't exactly buy their ads, I think ZFS employs some useful ideas. > And yet, if you can do that, I'd suspect you can, should, must do it > at a lower level than the FS. Again, FS robustness is good, but if > the disk itself is going, what good is having your directory (mostly) > intact if the files themselves have random corruptions? With per-block checksum you will know. Of course, that's still not end to end checksum. > If you can't trust the disk, you need more than just an FS which can > mostly survive hardware failure. You also need the FS itself (or > maybe the block layer) to support bad block relocation and all that > good stuff, or you need your apps designed to do that job by > themselves. Drives have internal relocation mechanisms, I don't think the filesystem needs to duplicate them (though it should try to work with bad blocks - relocations are possible on write). > It just doesn't make sense to me to do this at the FS level. You > mention TCP -- ok, but if TCP is doing its job, I shouldn't also need > to implement checksums and other robustness at the protocol layer > (http, ftp, ssh), should I? Sure you have to, if you value your data. > Similarly, the FS (and the apps) shouldn't have to know > about hardware problems until it really can't do anything about it > anymore, at which point the right thing to do is for the FS and apps > to go "oh shit" and drop what they're doing, and the admin replaces > hardware and restores from backup. Or brings a backup server online, > or... I don't think so. Going read-only if the disk returns write error, ok. But taking the fs offline? Why? Continuous backups (or rather transaction logs) are possible but who has them? Do you have them? Would you throw away several hours of work just because some file (or, say, unused area) contained unreadable block (which could probably be transient problem, and/or could be corrected by write)? -- Krzysztof Halasa ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 17:40 ` David Masover 2006-08-01 19:27 ` Krzysztof Halasa @ 2006-08-03 13:58 ` Matthias Andree 2006-08-03 18:48 ` Gregory Maxwell 2006-08-03 23:25 ` Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Russell Leighton 1 sibling, 2 replies; 639+ messages in thread From: Matthias Andree @ 2006-08-03 13:58 UTC (permalink / raw) To: David Masover Cc: Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel On Tue, 01 Aug 2006, David Masover wrote: > >RAID deals with the case where a device fails. RAID 1 with 2 disks can > >in theory detect an internal inconsistency but cannot fix it. > > Still, if it does that, that should be enough. The scary part wasn't > that there's an internal inconsistency, but that you wouldn't know. You won't usually know, unless you run a consistency check: RAID-1 will only read from one of the two drives for speed - except if you make the system check consistency as it goes, which would imply waiting for both disks at the same time. And in that case, you'd better look for drives that allow to synchronize their platter staples in order to avoid the read access penalty that waiting for two drives entails. > And it can fix it if you can figure out which disk went. If it's decent and detects a bad block, it'll log it and rewrite it with data from the mirror and let the drive do the remapping through ARWE. > >Depending how far you propogate it. Someone people working with huge > >data sets already write and check user level CRC values for this reason > >(in fact bitkeeper does it for one example). It should be relatively > >cheap to get much of that benefit without doing application to > >application just as TCP gets most of its benefit without going app to > >app. > > And yet, if you can do that, I'd suspect you can, should, must do it at > a lower level than the FS. Again, FS robustness is good, but if the > disk itself is going, what good is having your directory (mostly) intact > if the files themselves have random corruptions? Berkeley DB can, since version 4.1 (IIRC), write checksums (newer versions document this as SHA1) on its database pages, to detect corruptions and writes that were supposed to be atomic but failed (because you cannot write 4K or 16K atomically on a disk drive). -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-03 13:58 ` Matthias Andree @ 2006-08-03 18:48 ` Gregory Maxwell 2006-08-03 23:25 ` Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Russell Leighton 1 sibling, 0 replies; 639+ messages in thread From: Gregory Maxwell @ 2006-08-03 18:48 UTC (permalink / raw) To: David Masover, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso On 8/3/06, Matthias Andree <matthias.andree@gmx.de> wrote: > Berkeley DB can, since version 4.1 (IIRC), write checksums (newer > versions document this as SHA1) on its database pages, to detect > corruptions and writes that were supposed to be atomic but failed > (because you cannot write 4K or 16K atomically on a disk drive). The drive doesn't provide atomic writes for > 1sector (and I'd hardly call trashing a sector during a failed write atomic for even one)... but a FS can provide such semantics. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-03 13:58 ` Matthias Andree 2006-08-03 18:48 ` Gregory Maxwell @ 2006-08-03 23:25 ` Russell Leighton 2006-08-03 23:35 ` Matthias Andree ` (3 more replies) 1 sibling, 4 replies; 639+ messages in thread From: Russell Leighton @ 2006-08-03 23:25 UTC (permalink / raw) To: Matthias Andree Cc: David Masover, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel If the software (filesystem like ZFS or database like Berkeley DB) finds a mismatch for a checksum on a block read, then what? Is there a recovery mechanism, or do you just be happy you know there is a problem (and go to backup)? Thx Matthias Andree wrote: >Berkeley DB can, since version 4.1 (IIRC), write checksums (newer >versions document this as SHA1) on its database pages, to detect >corruptions and writes that were supposed to be atomic but failed >(because you cannot write 4K or 16K atomically on a disk drive). > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-03 23:25 ` Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Russell Leighton @ 2006-08-03 23:35 ` Matthias Andree 2006-08-04 0:01 ` Russell Leighton 2006-08-04 7:56 ` Toby Thain ` (2 subsequent siblings) 3 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-08-03 23:35 UTC (permalink / raw) To: Russell Leighton; +Cc: linux-kernel (I've stripped the Cc: list down to the bones. No need to shout side topics from the rooftops.) On Thu, 03 Aug 2006, Russell Leighton wrote: > If the software (filesystem like ZFS or database like Berkeley DB) > finds a mismatch for a checksum on a block read, then what? (Note that this assumes a Berkeley DB in transactional mode.) Complain, demand recovery, set the panic flag (refusing further transactions except close and open for recovery). > Is there a recovery mechanism, or do you just be happy you know there is > a problem (and go to backup)? Recoverability depends on log retention policy (set by the user or administrator) and how recently the block was written. There is a recovery mechanism. For applications that don't need their own recovery methods (few do), db_recover can do the job. In typical cases of power loss or kernel panic during write, the broken page will probably either be in the log so it can be restored (recover towards commit), or, if the commit hadn't completed but pages had been written due to cache conflicts, the database will be rolled back to the state before the interrupted transaction, effectively aborting the transaction. The details are in the Berkeley DB documentation, which please see. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-03 23:35 ` Matthias Andree @ 2006-08-04 0:01 ` Russell Leighton 0 siblings, 0 replies; 639+ messages in thread From: Russell Leighton @ 2006-08-04 0:01 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel Thx, for a db this seems natural... I am very curious about ZFS as I think we will seem more protection in the FS layer as disks get larger... If I have a very old file I am now half way throught reading and ZFS finds a bad block, I assume I would get some kind of read() error...but then what? Does anyone know if there are tools with ZFS to inspect the file? Matthias Andree wrote: >(I've stripped the Cc: list down to the bones. >No need to shout side topics from the rooftops.) > >On Thu, 03 Aug 2006, Russell Leighton wrote: > > > >>If the software (filesystem like ZFS or database like Berkeley DB) >>finds a mismatch for a checksum on a block read, then what? >> >> > >(Note that this assumes a Berkeley DB in transactional mode.) Complain, >demand recovery, set the panic flag (refusing further transactions >except close and open for recovery). > > > >>Is there a recovery mechanism, or do you just be happy you know there is >>a problem (and go to backup)? >> >> > >Recoverability depends on log retention policy (set by the user or >administrator) and how recently the block was written. There is a >recovery mechanism. > >For applications that don't need their own recovery methods (few do), >db_recover can do the job. > >In typical cases of power loss or kernel panic during write, the broken >page will probably either be in the log so it can be restored (recover >towards commit), or, if the commit hadn't completed but pages had been >written due to cache conflicts, the database will be rolled back to the >state before the interrupted transaction, effectively aborting the >transaction. > >The details are in the Berkeley DB documentation, which please see. > > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-03 23:25 ` Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Russell Leighton 2006-08-03 23:35 ` Matthias Andree @ 2006-08-04 7:56 ` Toby Thain 2006-08-04 11:55 ` Russell Leighton 2006-08-04 11:41 ` Tomasz Torcz 2006-08-04 20:51 ` David Masover 3 siblings, 1 reply; 639+ messages in thread From: Toby Thain @ 2006-08-04 7:56 UTC (permalink / raw) To: Russell Leighton; +Cc: ReiserFS List On 4-Aug-06, at 3:25 AM, Russell Leighton wrote: > > If the software (filesystem like ZFS or database like Berkeley DB) > finds a mismatch for a checksum on a block read, then what? > > Is there a recovery mechanism, or do you just be happy you know > there is a problem (and go to backup)? ZFS will correct from a good mirror (http://blogs.sun.com/roller/page/ bonwick?entry=zfs_end_to_end_data). --T > > Thx > > Matthias Andree wrote: > >> Berkeley DB can, since version 4.1 (IIRC), write checksums (newer >> versions document this as SHA1) on its database pages, to detect >> corruptions and writes that were supposed to be atomic but failed >> (because you cannot write 4K or 16K atomically on a disk drive). >> > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-04 7:56 ` Toby Thain @ 2006-08-04 11:55 ` Russell Leighton 0 siblings, 0 replies; 639+ messages in thread From: Russell Leighton @ 2006-08-04 11:55 UTC (permalink / raw) To: Toby Thain; +Cc: ReiserFS List That was exactly the summary I was looking for. I would enourage folks to read the referenced link Toby sent: http://blogs.sun.com/roller/page/bonwick?entry=zfs_end_to_end_data ...also the linked RAID-Z summary from this article was very interesting, since something like this is needed for recovery from checksum failures: > Which brings us to the coolest thing about RAID-Z: self-healing data. > In addition to handling whole-disk failure, RAID-Z can also detect and > correct silent data corruption. Whenever you read a RAID-Z block, ZFS > compares it against its checksum. If the data disks didn't return the > right answer, ZFS reads the parity and then does combinatorial > reconstruction to figure out which disk returned bad data. It then > repairs the damaged disk and returns good data to the application. ZFS > also reports the incident through Solaris FMA so that the system > administrator knows that one of the disks is silently failing. > > Finally, note that *RAID-Z doesn't require any special hardware.* It > doesn't need NVRAM for correctness, and it doesn't need write > buffering for good performance. With RAID-Z, ZFS makes good on the > original RAID promise: it provides fast, reliable storage using cheap, > commodity disks. > http://blogs.sun.com/roller/page/bonwick?entry=raid_z Toby Thain wrote: > > On 4-Aug-06, at 3:25 AM, Russell Leighton wrote: > >> >> If the software (filesystem like ZFS or database like Berkeley DB) >> finds a mismatch for a checksum on a block read, then what? >> >> Is there a recovery mechanism, or do you just be happy you know >> there is a problem (and go to backup)? > > > ZFS will correct from a good mirror > (http://blogs.sun.com/roller/page/bonwick?entry=zfs_end_to_end_data). > --T > >> >> Thx >> >> Matthias Andree wrote: >> >>> Berkeley DB can, since version 4.1 (IIRC), write checksums (newer >>> versions document this as SHA1) on its database pages, to detect >>> corruptions and writes that were supposed to be atomic but failed >>> (because you cannot write 4K or 16K atomically on a disk drive). >>> >> > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-03 23:25 ` Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Russell Leighton 2006-08-03 23:35 ` Matthias Andree 2006-08-04 7:56 ` Toby Thain @ 2006-08-04 11:41 ` Tomasz Torcz 2006-08-04 20:42 ` Horst H. von Brand 2006-08-04 20:51 ` David Masover 3 siblings, 1 reply; 639+ messages in thread From: Tomasz Torcz @ 2006-08-04 11:41 UTC (permalink / raw) To: linux-kernel [-- Attachment #1: Type: text/plain, Size: 893 bytes --] On Thu, Aug 03, 2006 at 07:25:19PM -0400, Russell Leighton wrote: > > If the software (filesystem like ZFS or database like Berkeley DB) > finds a mismatch for a checksum on a block read, then what? > > Is there a recovery mechanism, or do you just be happy you know there is > a problem (and go to backup)? ZFS readsthis block again from different mirror, and if checksum is right -- returns good data to userspace and rewrites failed block with good data. Note, that there could be multiple mirrors, either physically (like RAID1) or logically (blocks could be mirrored on different areas of the same disk; some files can be protected with multiple mirrors, some left unprotected without mirrors). -- Tomasz Torcz To co nierealne -- tutaj jest normalne. zdzichu@irc.-nie.spam-.pl Ziomale na życie mają tu patenty specjalne. [-- Attachment #2: Type: application/pgp-signature, Size: 229 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-04 11:41 ` Tomasz Torcz @ 2006-08-04 20:42 ` Horst H. von Brand 0 siblings, 0 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-08-04 20:42 UTC (permalink / raw) To: linux-kernel Tomasz Torcz <zdzichu@irc.pl> wrote: > On Thu, Aug 03, 2006 at 07:25:19PM -0400, Russell Leighton wrote: > > > > If the software (filesystem like ZFS or database like Berkeley DB) > > finds a mismatch for a checksum on a block read, then what? > > > > Is there a recovery mechanism, or do you just be happy you know there is > > a problem (and go to backup)? > > ZFS readsthis block again from different mirror, and if checksum is > right -- returns good data to userspace and rewrites failed block with > good data. > > Note, that there could be multiple mirrors, either physically (like > RAID1) or logically (blocks could be mirrored on different areas of the > same disk; some files can be protected with multiple mirrors, some left > unprotected without mirrors). Murphy's law will ensure that the important files are unprotected. And the 1st Law of Disk Drives (they are always full) will ensure that there are no mirrored pieces anyway... -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-03 23:25 ` Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Russell Leighton ` (2 preceding siblings ...) 2006-08-04 11:41 ` Tomasz Torcz @ 2006-08-04 20:51 ` David Masover 3 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-08-04 20:51 UTC (permalink / raw) To: Russell Leighton Cc: Matthias Andree, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel Russell Leighton wrote: > Is there a recovery mechanism, or do you just be happy you know there is > a problem (and go to backup)? You probably go to backup anyway. The recovery mechanism just means you get to choose the downtime to restore from backup (if there is downtime), versus being suddenly down until you can restore. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 15:29 ` Alan Cox 2006-08-01 16:44 ` David Masover @ 2006-08-01 18:11 ` Adrian Ulrich 2006-08-01 18:21 ` Ric Wheeler 2 siblings, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-08-01 18:11 UTC (permalink / raw) To: Alan Cox Cc: vonbrand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel > You do it turns out. Its becoming an issue more and more that the sheer > amount of storage means that the undetected error rate from disks, > hosts, memory, cables and everything else is rising. IMHO the possibility to hit such a random-so-far-undetected-corruption is very low with one of the big/expensive raid systems as they are doing fancy stuff like 'disk scrubbing' and usually do fail disks at very early stages.. * I've seen storage systems from a BIG vendor die due to firmware bugs * I've seen FC-Cards die.. SAN-switches rebooted.. People used my cables to do rope skipping * We had Fire, non-working UPS and faulty diesel generators.. but so far the FSes (and applications) on the Storage never complained about corrupted data. ..YMMV.. Btw: I don't think that Reiserfs really behaves this bad with broken hardware. So far, Reiser3 survived 2 broken Harddrives without problems while i've seen ext2/3 die 4 times so far... (= everything inside /lost+found). Reiser4 survived # mkisofs . > /dev/sda Lucky me.. maybe.. To get back on-topic: Some people try very hard to claim that the world doesn't need Reiser4 and that you can do everything with ext3. Ext3 may be fine for them but some people (like me) really need Reiser4 because they got applications/workloads that won't work good (fast) on ext3. Why is it such a big thing to include a filesystem? Even if it's unstable: does anyone care? Eg: the HFS+ driver is buggy (corrupted the FS of my OSX installation 3 times so far) but does this buggyness affect people *not* using it? No. Regards, Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 15:29 ` Alan Cox 2006-08-01 16:44 ` David Masover 2006-08-01 18:11 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Adrian Ulrich @ 2006-08-01 18:21 ` Ric Wheeler 2006-08-01 11:41 ` Hans Reiser ` (2 more replies) 2 siblings, 3 replies; 639+ messages in thread From: Ric Wheeler @ 2006-08-01 18:21 UTC (permalink / raw) To: Alan Cox Cc: Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel Alan Cox wrote: > Ar Maw, 2006-08-01 am 16:52 +0200, ysgrifennodd Adrian Ulrich: > >>WriteCache, Mirroring between 2 Datacenters, snapshotting.. etc.. >>you don't need your filesystem beeing super-robust against bad sectors >>and such stuff because: > > > You do it turns out. Its becoming an issue more and more that the sheer > amount of storage means that the undetected error rate from disks, > hosts, memory, cables and everything else is rising. I agree with Alan despite being an enthusiastic supporter of neat array based technologies. Most people use absolutely giant disks in laptops and desktop systems (300GB & 500GB are common, 750GB on the way). File systems need to be as robust as possible for users of these systems as people are commonly storing personal "critical" data like photos mostly on these unprotected drives. Even for the high end users, array based mirroring and so on can only do so much to protect you. Mirroring a corrupt file system to a remote data center will mirror your corruption. Rolling back to a snapshot typically only happens when you notice a corruption which can go undetected for quite a while, so even that will benefit from having "reliability" baked into the file system (i.e., it should grumble about corruption to let you know that you need to roll back or fsck or whatever). An even larger issue is that our tools, like fsck, which are used to uncover these silent corruptions need to scale up to the point that they can uncover issues in minutes instead of days. A lot of the focus at the file system workshop was around how to dramatically reduce the repair time of file systems. In a way, having super reliable storage hardware is only as good as the file system layer on top of it - reliability needs to be baked into the entire IO system stack... ric ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 18:21 ` Ric Wheeler @ 2006-08-01 11:41 ` Hans Reiser 2006-08-03 14:03 ` Matthias Andree 2006-08-01 19:11 ` David Masover 2006-08-03 14:03 ` Matthias Andree 2 siblings, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-08-01 11:41 UTC (permalink / raw) To: ric, Edward Shishkin Cc: Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel Ric Wheeler wrote: > Alan Cox wrote: > >> >> >> You do it turns out. Its becoming an issue more and more that the sheer >> amount of storage means that the undetected error rate from disks, >> hosts, memory, cables and everything else is rising. > > > > I agree with Alan You will want to try our compression plugin, it has an ecc for every 64k.... Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 11:41 ` Hans Reiser @ 2006-08-03 14:03 ` Matthias Andree 2006-08-03 15:44 ` Edward Shishkin 0 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-08-03 14:03 UTC (permalink / raw) To: Hans Reiser Cc: ric, Edward Shishkin, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel On Tue, 01 Aug 2006, Hans Reiser wrote: > You will want to try our compression plugin, it has an ecc for every 64k.... What kind of forward error correction would that be, and how much and what failure patterns can it correct? URL suffices. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-03 14:03 ` Matthias Andree @ 2006-08-03 15:44 ` Edward Shishkin 2006-08-03 17:26 ` Hans Reiser 2006-08-07 7:57 ` Matthias Andree 0 siblings, 2 replies; 639+ messages in thread From: Edward Shishkin @ 2006-08-03 15:44 UTC (permalink / raw) To: Matthias Andree Cc: Hans Reiser, ric, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel Matthias Andree wrote: > On Tue, 01 Aug 2006, Hans Reiser wrote: > > >>You will want to try our compression plugin, it has an ecc for every 64k.... > > > What kind of forward error correction would that be, Actually we use checksums, not ECC. If checksum is wrong, then run fsck - it will remove the whole disk cluster, that represent 64K of data. and how much and > what failure patterns can it correct? URL suffices. > Checksum is checked before unsafe decompression (when trying to decompress incorrect data can lead to fatal things). It can be broken because of many reasons. The main one is tree corruption (for example, when disk cluster became incomplete - ECC can not help here). Perhaps such checksumming is also useful for other things, I didnt classify the patterns.. Edward. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-03 15:44 ` Edward Shishkin @ 2006-08-03 17:26 ` Hans Reiser 2006-08-04 17:04 ` Edward Shishkin 2006-08-07 7:57 ` Matthias Andree 1 sibling, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-08-03 17:26 UTC (permalink / raw) To: Edward Shishkin Cc: Matthias Andree, ric, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel Edward Shishkin wrote: > Matthias Andree wrote: > >> On Tue, 01 Aug 2006, Hans Reiser wrote: >> >> >>> You will want to try our compression plugin, it has an ecc for every >>> 64k.... >> >> >> >> What kind of forward error correction would that be, > > > > Actually we use checksums, not ECC. If checksum is wrong, then run > fsck - it will remove the whole disk cluster, that represent 64K of > data. How about we switch to ecc, which would help with bit rot not sector loss? > > > and how much and > >> what failure patterns can it correct? URL suffices. >> > > Checksum is checked before unsafe decompression (when trying to > decompress incorrect data can lead to fatal things). It can be > broken because of many reasons. The main one is tree corruption > (for example, when disk cluster became incomplete - ECC can not > help here). Perhaps such checksumming is also useful for other > things, I didnt classify the patterns.. > > Edward. > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-03 17:26 ` Hans Reiser @ 2006-08-04 17:04 ` Edward Shishkin 2006-08-04 18:57 ` Antonio Vargas 2006-08-05 0:56 ` Hans Reiser 0 siblings, 2 replies; 639+ messages in thread From: Edward Shishkin @ 2006-08-04 17:04 UTC (permalink / raw) To: Hans Reiser Cc: Matthias Andree, ric, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel Hans Reiser wrote: > Edward Shishkin wrote: > > >>Matthias Andree wrote: >> >> >>>On Tue, 01 Aug 2006, Hans Reiser wrote: >>> >>> >>> >>>>You will want to try our compression plugin, it has an ecc for every >>>>64k.... >>> >>> >>> >>>What kind of forward error correction would that be, >> >> >> >>Actually we use checksums, not ECC. If checksum is wrong, then run >>fsck - it will remove the whole disk cluster, that represent 64K of >>data. > > > How about we switch to ecc, which would help with bit rot not sector loss? Interesting aspect. Yes, we can implement ECC as a special crypto transform that inflates data. As I mentioned earlier, it is possible via translation of key offsets with scale factor > 1. Of course, it is better then nothing, but anyway meta-data remains ecc-unprotected, and, hence, robustness is not increased.. Edward. > >> >> and how much and >> >> >>>what failure patterns can it correct? URL suffices. >>> >> >>Checksum is checked before unsafe decompression (when trying to >>decompress incorrect data can lead to fatal things). It can be >>broken because of many reasons. The main one is tree corruption >>(for example, when disk cluster became incomplete - ECC can not >>help here). Perhaps such checksumming is also useful for other >>things, I didnt classify the patterns.. >> >>Edward. >> >> > > > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-04 17:04 ` Edward Shishkin @ 2006-08-04 18:57 ` Antonio Vargas 2006-08-05 1:02 ` Hans Reiser 2006-08-05 0:56 ` Hans Reiser 1 sibling, 1 reply; 639+ messages in thread From: Antonio Vargas @ 2006-08-04 18:57 UTC (permalink / raw) To: Edward Shishkin Cc: Hans Reiser, Matthias Andree, ric, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel On 8/4/06, Edward Shishkin <edward@namesys.com> wrote: > Hans Reiser wrote: > > Edward Shishkin wrote: > > > > > >>Matthias Andree wrote: > >> > >> > >>>On Tue, 01 Aug 2006, Hans Reiser wrote: > >>> > >>> > >>> > >>>>You will want to try our compression plugin, it has an ecc for every > >>>>64k.... > >>> > >>> > >>> > >>>What kind of forward error correction would that be, > >> > >> > >> > >>Actually we use checksums, not ECC. If checksum is wrong, then run > >>fsck - it will remove the whole disk cluster, that represent 64K of > >>data. > > > > > > How about we switch to ecc, which would help with bit rot not sector loss? > > Interesting aspect. > > Yes, we can implement ECC as a special crypto transform that inflates > data. As I mentioned earlier, it is possible via translation of key > offsets with scale factor > 1. > > Of course, it is better then nothing, but anyway meta-data remains > ecc-unprotected, and, hence, robustness is not increased.. > > Edward. > > > > >> > >> and how much and > >> > >> > >>>what failure patterns can it correct? URL suffices. > >>> > >> > >>Checksum is checked before unsafe decompression (when trying to > >>decompress incorrect data can lead to fatal things). It can be > >>broken because of many reasons. The main one is tree corruption > >>(for example, when disk cluster became incomplete - ECC can not > >>help here). Perhaps such checksumming is also useful for other > >>things, I didnt classify the patterns.. > >> > >>Edward. > >> > >> Would the storage + plugin subsystem support storing >1 copies of the metadata tree? -- Greetz, Antonio Vargas aka winden of network http://network.amigascne.org/ windNOenSPAMntw@gmail.com thesameasabove@amigascne.org Every day, every year you have to work you have to study you have to scene. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-04 18:57 ` Antonio Vargas @ 2006-08-05 1:02 ` Hans Reiser 0 siblings, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-08-05 1:02 UTC (permalink / raw) To: Antonio Vargas Cc: Edward Shishkin, Matthias Andree, ric, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel Antonio Vargas wrote: > On 8/4/06, Edward Shishkin <edward@namesys.com> wrote: > >> Hans Reiser wrote: >> > Edward Shishkin wrote: >> > >> > >> >>Matthias Andree wrote: >> >> >> >> >> >>>On Tue, 01 Aug 2006, Hans Reiser wrote: >> >>> >> >>> >> >>> >> >>>>You will want to try our compression plugin, it has an ecc for every >> >>>>64k.... >> >>> >> >>> >> >>> >> >>>What kind of forward error correction would that be, >> >> >> >> >> >> >> >>Actually we use checksums, not ECC. If checksum is wrong, then run >> >>fsck - it will remove the whole disk cluster, that represent 64K of >> >>data. >> > >> > >> > How about we switch to ecc, which would help with bit rot not >> sector loss? >> >> Interesting aspect. >> >> Yes, we can implement ECC as a special crypto transform that inflates >> data. As I mentioned earlier, it is possible via translation of key >> offsets with scale factor > 1. >> >> Of course, it is better then nothing, but anyway meta-data remains >> ecc-unprotected, and, hence, robustness is not increased.. >> >> Edward. >> >> > >> >> >> >> and how much and >> >> >> >> >> >>>what failure patterns can it correct? URL suffices. >> >>> >> >> >> >>Checksum is checked before unsafe decompression (when trying to >> >>decompress incorrect data can lead to fatal things). It can be >> >>broken because of many reasons. The main one is tree corruption >> >>(for example, when disk cluster became incomplete - ECC can not >> >>help here). Perhaps such checksumming is also useful for other >> >>things, I didnt classify the patterns.. >> >> >> >>Edward. >> >> >> >> > > > Would the storage + plugin subsystem support storing >1 copies of the > metadata tree? > > I suppose.... What would be nice would be to have a plugin that when a node fails its checksum/ecc it knows to get it from another mirror, and which generally handles faults with a graceful understanding of its ability to get copies from a mirror (or RAID parity calculation). I would happily accept such a patch (subject to usual reservation of right to complain about implementation details). ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-04 17:04 ` Edward Shishkin 2006-08-04 18:57 ` Antonio Vargas @ 2006-08-05 0:56 ` Hans Reiser 2006-08-06 22:19 ` Edward Shishkin 1 sibling, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-08-05 0:56 UTC (permalink / raw) To: Edward Shishkin Cc: Matthias Andree, ric, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel Edward Shishkin wrote: > >> >> >> How about we switch to ecc, which would help with bit rot not sector >> loss? > > > Interesting aspect. > > Yes, we can implement ECC as a special crypto transform that inflates > data. As I mentioned earlier, it is possible via translation of key > offsets with scale factor > 1. > > Of course, it is better then nothing, but anyway meta-data remains > ecc-unprotected, and, hence, robustness is not increased.. > > Edward. Would you prefer to do it as a node layout plugin instead, so as to get the metadata? Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-05 0:56 ` Hans Reiser @ 2006-08-06 22:19 ` Edward Shishkin 2006-08-09 8:40 ` Hans Reiser 0 siblings, 1 reply; 639+ messages in thread From: Edward Shishkin @ 2006-08-06 22:19 UTC (permalink / raw) To: Hans Reiser Cc: Matthias Andree, ric, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel Hans Reiser wrote: > Edward Shishkin wrote: > > >>> >>>How about we switch to ecc, which would help with bit rot not sector >>>loss? >> >> >>Interesting aspect. >> >>Yes, we can implement ECC as a special crypto transform that inflates >>data. As I mentioned earlier, it is possible via translation of key >>offsets with scale factor > 1. >> >>Of course, it is better then nothing, but anyway meta-data remains >>ecc-unprotected, and, hence, robustness is not increased.. >> >>Edward. > > > Would you prefer to do it as a node layout plugin instead, so as to get > the metadata? > Yes, it looks like a business of node plugin, but AFAIK, you objected against such checks: currently only bitmap nodes have a protection (checksum); supporting ecc-signatures is more space/cpu expensive. Edward. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-06 22:19 ` Edward Shishkin @ 2006-08-09 8:40 ` Hans Reiser 2006-08-09 11:53 ` Jan Engelhardt 0 siblings, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-08-09 8:40 UTC (permalink / raw) To: Edward Shishkin Cc: Matthias Andree, ric, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel Edward Shishkin wrote: > Hans Reiser wrote: > >> Edward Shishkin wrote: >> >> >>>> >>>> How about we switch to ecc, which would help with bit rot not sector >>>> loss? >>> >>> >>> >>> Interesting aspect. >>> >>> Yes, we can implement ECC as a special crypto transform that inflates >>> data. As I mentioned earlier, it is possible via translation of key >>> offsets with scale factor > 1. >>> >>> Of course, it is better then nothing, but anyway meta-data remains >>> ecc-unprotected, and, hence, robustness is not increased.. >>> >>> Edward. >> >> >> >> Would you prefer to do it as a node layout plugin instead, so as to get >> the metadata? >> > > Yes, it looks like a business of node plugin, but AFAIK, you > objected against such checks: Did I really? Well, I think that allowing users to choose whether to checksum or not is a reasonable thing to allow them. I personally would skip the checksum on my computer, but others.... It could be a useful mkfs option.... > currently only bitmap nodes have > a protection (checksum); supporting ecc-signatures is more > space/cpu expensive. > > Edward. > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-09 8:40 ` Hans Reiser @ 2006-08-09 11:53 ` Jan Engelhardt 2006-08-09 15:48 ` David Masover 0 siblings, 1 reply; 639+ messages in thread From: Jan Engelhardt @ 2006-08-09 11:53 UTC (permalink / raw) To: Hans Reiser Cc: Edward Shishkin, Matthias Andree, ric, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel >> Yes, it looks like a business of node plugin, but AFAIK, you >> objected against such checks: > >Did I really? Well, I think that allowing users to choose whether to >checksum or not is a reasonable thing to allow them. I personally would >skip the checksum on my computer, but others.... > >It could be a useful mkfs option.... It should preferably a runtime tunable variable, at best even per-superblock and (overriding the sb setting), per-file. Jan Engelhardt -- ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-09 11:53 ` Jan Engelhardt @ 2006-08-09 15:48 ` David Masover 0 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-08-09 15:48 UTC (permalink / raw) To: Jan Engelhardt Cc: Hans Reiser, Edward Shishkin, Matthias Andree, ric, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, lkml, jeff, tytso, linux-kernel Jan Engelhardt wrote: >>> Yes, it looks like a business of node plugin, but AFAIK, you >>> objected against such checks: >> Did I really? Well, I think that allowing users to choose whether to >> checksum or not is a reasonable thing to allow them. I personally would >> skip the checksum on my computer, but others.... >> >> It could be a useful mkfs option.... > > It should preferably a runtime tunable variable, at best even > per-superblock and (overriding the sb setting), per-file. Sounds almost exactly like a plugin. And yes, that would be the way to do it, especially considering some files will already have internal consistency checking -- just as we should allow direct disk IO to some files (no journaling) when the files in question are databases that do their own journaling. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-03 15:44 ` Edward Shishkin 2006-08-03 17:26 ` Hans Reiser @ 2006-08-07 7:57 ` Matthias Andree 2006-08-08 11:06 ` Edward Shishkin 1 sibling, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-08-07 7:57 UTC (permalink / raw) To: Edward Shishkin; +Cc: Hans Reiser, reiserfs-list, linux-kernel [stripping Cc: list] On Thu, 03 Aug 2006, Edward Shishkin wrote: > >What kind of forward error correction would that be, > > Actually we use checksums, not ECC. If checksum is wrong, then run > fsck - it will remove the whole disk cluster, that represent 64K of > data. Well, that's quite a difference... > Checksum is checked before unsafe decompression (when trying to > decompress incorrect data can lead to fatal things). Is this sufficient? How about corruptions that lead to the same checksum and can then confuse the decompressor? Is the decompressor safe in that it does not scribble over memory it has not allocated? -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-07 7:57 ` Matthias Andree @ 2006-08-08 11:06 ` Edward Shishkin 0 siblings, 0 replies; 639+ messages in thread From: Edward Shishkin @ 2006-08-08 11:06 UTC (permalink / raw) To: Matthias Andree; +Cc: Hans Reiser, reiserfs-list, linux-kernel Matthias Andree wrote: > [stripping Cc: list] > > On Thu, 03 Aug 2006, Edward Shishkin wrote: > > >>>What kind of forward error correction would that be, >> >>Actually we use checksums, not ECC. If checksum is wrong, then run >>fsck - it will remove the whole disk cluster, that represent 64K of >>data. > > > Well, that's quite a difference... > > >>Checksum is checked before unsafe decompression (when trying to >>decompress incorrect data can lead to fatal things). > > > Is this sufficient? How about corruptions that lead to the same checksum > and can then confuse the decompressor? It is a multiplication of two unlikely events: fs corruption and 32-hash collision. Paranoid people can assign zlib-based transform plugin: afaik everything is safe there. Is the decompressor safe in that > it does not scribble over memory it has not allocated? > yes ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 18:21 ` Ric Wheeler 2006-08-01 11:41 ` Hans Reiser @ 2006-08-01 19:11 ` David Masover 2006-08-03 14:03 ` Matthias Andree 2 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-08-01 19:11 UTC (permalink / raw) To: ric Cc: Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel Ric Wheeler wrote: > Alan Cox wrote: >> Ar Maw, 2006-08-01 am 16:52 +0200, ysgrifennodd Adrian Ulrich: >> >>> WriteCache, Mirroring between 2 Datacenters, snapshotting.. etc.. >>> you don't need your filesystem beeing super-robust against bad sectors >>> and such stuff because: >> >> >> You do it turns out. Its becoming an issue more and more that the sheer >> amount of storage means that the undetected error rate from disks, >> hosts, memory, cables and everything else is rising. > Most people use absolutely giant disks in laptops and desktop systems > (300GB & 500GB are common, 750GB on the way). File systems need to be as > robust as possible for users of these systems as people are commonly > storing personal "critical" data like photos mostly on these unprotected > drives. Their loss. Robust FS is good, but really, if you aren't doing backup, you are going to lose data. End of story. > Even for the high end users, array based mirroring and so on can only do > so much to protect you. > > Mirroring a corrupt file system to a remote data center will mirror your > corruption. Assuming it's undetected. Why would it be undetected? > Rolling back to a snapshot typically only happens when you notice a > corruption which can go undetected for quite a while, so even that will > benefit from having "reliability" baked into the file system (i.e., it > should grumble about corruption to let you know that you need to roll > back or fsck or whatever). Yes, the filesystem should complain about corruption. So should the block layer -- if you don't trust the FS, use a checksum at the block layer. So should... There are just so many other, better places to do this than the FS. The FS should complain, yes, but if the disk is bad, there's going to be corruption. > An even larger issue is that our tools, like fsck, which are used to > uncover these silent corruptions need to scale up to the point that they > can uncover issues in minutes instead of days. A lot of the focus at > the file system workshop was around how to dramatically reduce the > repair time of file systems. That would be interesting. I know from experience that fsck.reiser4 is amazing. Blew away my data with something akin to an rm -rf, and fsck fixed it. Tons of crashing/instability in the early days, but only once -- before they even had a version instead of a date, I think -- did I ever have a case where fsck couldn't fix it. So I guess the next step would be to make fsck faster. Someone mentioned a fsck that repairs the FS in the background? > In a way, having super reliable storage hardware is only as good as the > file system layer on top of it - reliability needs to be baked into the > entire IO system stack... That bit makes no sense. If you have super reliable storage failure (never dies), and your FS is also reliable (never dies unless hardware does, but may go bat-shit insane when hardware dies), then you've got a super reliable system. You're right, running Linux's HFS+ or NTFS write support is generally a bad idea, no matter how reliable your hardware is. But this discussion was not about whether an FS is stable, but how well an FS survives hardware corruption. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 18:21 ` Ric Wheeler 2006-08-01 11:41 ` Hans Reiser 2006-08-01 19:11 ` David Masover @ 2006-08-03 14:03 ` Matthias Andree 2006-08-03 16:50 ` Theodore Tso 2 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-08-03 14:03 UTC (permalink / raw) To: Ric Wheeler Cc: Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, tytso, linux-kernel On Tue, 01 Aug 2006, Ric Wheeler wrote: > Mirroring a corrupt file system to a remote data center will mirror your > corruption. > > Rolling back to a snapshot typically only happens when you notice a > corruption which can go undetected for quite a while, so even that will > benefit from having "reliability" baked into the file system (i.e., it > should grumble about corruption to let you know that you need to roll > back or fsck or whatever). > > An even larger issue is that our tools, like fsck, which are used to > uncover these silent corruptions need to scale up to the point that they > can uncover issues in minutes instead of days. A lot of the focus at > the file system workshop was around how to dramatically reduce the > repair time of file systems. Which makes me wonder if backup systems shouldn't help with this. If they are reading the whole file anyways, they can easily compute strong checksums as they go, and record them for later use, and check so many percent of unchanged files every day to complain about corruptions. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-03 14:03 ` Matthias Andree @ 2006-08-03 16:50 ` Theodore Tso 0 siblings, 0 replies; 639+ messages in thread From: Theodore Tso @ 2006-08-03 16:50 UTC (permalink / raw) To: Ric Wheeler, Alan Cox, Adrian Ulrich, Horst H. von Brand, bernd-schubert, reiserfs-list, jbglaw, clay.barnes, rudy, ipso, reiser, lkml, jeff, linux-kernel, wayned On Thu, Aug 03, 2006 at 04:03:07PM +0200, Matthias Andree wrote: > On Tue, 01 Aug 2006, Ric Wheeler wrote: > > > Mirroring a corrupt file system to a remote data center will mirror your > > corruption. > > > > Which makes me wonder if backup systems shouldn't help with this. If > they are reading the whole file anyways, they can easily compute strong > checksums as they go, and record them for later use, and check so many > percent of unchanged files every day to complain about corruptions. They absolutely should do this sort of thing. Also sounds like yet another option that could be added to rsync. (Only half-joking. :-) - Ted ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 14:28 ` Horst H. von Brand 2006-08-01 14:52 ` Adrian Ulrich @ 2006-08-01 16:57 ` David Masover 2006-08-06 22:59 ` Pavel Machek 1 sibling, 1 reply; 639+ messages in thread From: David Masover @ 2006-08-01 16:57 UTC (permalink / raw) To: Horst H. von Brand Cc: Bernd Schubert, reiserfs-list, Jan-Benedict Glaw, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, ipso, reiser, lkml, jeff, tytso, linux-kernel Horst H. von Brand wrote: > Bernd Schubert <bernd-schubert@gmx.de> wrote: >> While filesystem speed is nice, it also would be great if reiser4.x would be >> very robust against any kind of hardware failures. > > Can't have both. Why not? I mean, other than TANSTAAFL, is there a technical reason for them being mutually exclusive? I suspect it's more "we haven't found a way yet..." ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 16:57 ` David Masover @ 2006-08-06 22:59 ` Pavel Machek 2006-08-06 23:36 ` David Masover 2006-08-09 8:37 ` Hans Reiser 0 siblings, 2 replies; 639+ messages in thread From: Pavel Machek @ 2006-08-06 22:59 UTC (permalink / raw) To: David Masover Cc: Horst H. von Brand, Bernd Schubert, reiserfs-list, Jan-Benedict Glaw, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, ipso, reiser, lkml, jeff, tytso, linux-kernel On Tue 01-08-06 11:57:10, David Masover wrote: > Horst H. von Brand wrote: > >Bernd Schubert <bernd-schubert@gmx.de> wrote: > > >>While filesystem speed is nice, it also would be great > >>if reiser4.x would be very robust against any kind of > >>hardware failures. > > > >Can't have both. > > Why not? I mean, other than TANSTAAFL, is there a > technical reason for them being mutually exclusive? I > suspect it's more "we haven't found a way yet..." What does the acronym mean? Yes, I'm afraid redundancy/checksums kill write speed, and you need that for robustness... You could have filesystem that can be tuned for reliability and tuned for speed... but you can't have both in one filesystem instance. -- Thanks for all the (sleeping) penguins. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-06 22:59 ` Pavel Machek @ 2006-08-06 23:36 ` David Masover 2006-08-09 8:37 ` Hans Reiser 1 sibling, 0 replies; 639+ messages in thread From: David Masover @ 2006-08-06 23:36 UTC (permalink / raw) To: Pavel Machek Cc: Horst H. von Brand, Bernd Schubert, reiserfs-list, Jan-Benedict Glaw, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, ipso, reiser, lkml, jeff, tytso, linux-kernel Pavel Machek wrote: > On Tue 01-08-06 11:57:10, David Masover wrote: >> Horst H. von Brand wrote: >>> Bernd Schubert <bernd-schubert@gmx.de> wrote: >>>> While filesystem speed is nice, it also would be great >>>> if reiser4.x would be very robust against any kind of >>>> hardware failures. >>> Can't have both. >> Why not? I mean, other than TANSTAAFL, is there a >> technical reason for them being mutually exclusive? I >> suspect it's more "we haven't found a way yet..." > > What does the acronym mean? There Ain't No Such Thing As A Free Lunch. > Yes, I'm afraid redundancy/checksums kill write speed, and you need > that for robustness... Not necessarily -- if you do it on flush, and store it near the data it relates to, you can expect a similar impact to compression, except that due to slow disks, the compression can actually speed things up 2x, whereas checksums should be some insignificant amount slower than 1x. Redundancy, sure, but checksums should be easy, and I don't see what robustness (abilities of fsck) has to do with it. > You could have filesystem that can be tuned for reliability and tuned > for speed... but you can't have both in one filesystem instance. That's an example of TANSTAAFL, if it's true. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-06 22:59 ` Pavel Machek 2006-08-06 23:36 ` David Masover @ 2006-08-09 8:37 ` Hans Reiser 2006-08-09 9:48 ` Pavel Machek 2006-08-09 15:52 ` David Masover 1 sibling, 2 replies; 639+ messages in thread From: Hans Reiser @ 2006-08-09 8:37 UTC (permalink / raw) To: Pavel Machek Cc: David Masover, Horst H. von Brand, Bernd Schubert, reiserfs-list, Jan-Benedict Glaw, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, ipso, lkml, jeff, tytso, linux-kernel Pavel Machek wrote: > > >Yes, I'm afraid redundancy/checksums kill write speed, > they kill write speed to cache, but not to disk.... our compression plugin is faster than the uncompressed plugin..... ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-09 8:37 ` Hans Reiser @ 2006-08-09 9:48 ` Pavel Machek 2006-08-09 9:15 ` Hans Reiser 2006-08-09 15:52 ` David Masover 1 sibling, 1 reply; 639+ messages in thread From: Pavel Machek @ 2006-08-09 9:48 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Horst H. von Brand, Bernd Schubert, reiserfs-list, Jan-Benedict Glaw, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, ipso, lkml, jeff, tytso, linux-kernel On Wed 2006-08-09 02:37:45, Hans Reiser wrote: > Pavel Machek wrote: > > > > > > >Yes, I'm afraid redundancy/checksums kill write speed, > > > they kill write speed to cache, but not to disk.... our compression > plugin is faster than the uncompressed plugin..... Yes, you can get clever. But your compression plugin also means that single bit error means whole block is lost, so there _is_ speed vs. stability-against-hw-problems. But you are right that compression will catch same class of errors checksums will, so that it is probably good thing w.r.t. stability. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-09 9:48 ` Pavel Machek @ 2006-08-09 9:15 ` Hans Reiser 0 siblings, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-08-09 9:15 UTC (permalink / raw) To: Pavel Machek Cc: David Masover, Horst H. von Brand, Bernd Schubert, reiserfs-list, Jan-Benedict Glaw, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, ipso, lkml, jeff, tytso, linux-kernel Pavel Machek wrote: >On Wed 2006-08-09 02:37:45, Hans Reiser wrote: > > >>Pavel Machek wrote: >> >> >> >>>Yes, I'm afraid redundancy/checksums kill write speed, >>> >>> >>> >>they kill write speed to cache, but not to disk.... our compression >>plugin is faster than the uncompressed plugin..... >> >> > >Yes, you can get clever. But your compression plugin also means that >single bit error means whole block is lost, so there _is_ speed >vs. stability-against-hw-problems. > >But you are right that compression will catch same class of errors >checksums will, so that it is probably good thing w.r.t. stability. > > Pavel > > So we need to use ecc not checksums if we want to increase reliability. Edward, can you comment in more detail regarding your views and the performance issues for ecc that you see? ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-09 8:37 ` Hans Reiser 2006-08-09 9:48 ` Pavel Machek @ 2006-08-09 15:52 ` David Masover 1 sibling, 0 replies; 639+ messages in thread From: David Masover @ 2006-08-09 15:52 UTC (permalink / raw) To: Hans Reiser Cc: Pavel Machek, Horst H. von Brand, Bernd Schubert, reiserfs-list, Jan-Benedict Glaw, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, ipso, lkml, jeff, tytso, linux-kernel Hans Reiser wrote: > Pavel Machek wrote: > >> >> Yes, I'm afraid redundancy/checksums kill write speed, >> > they kill write speed to cache, but not to disk.... our compression > plugin is faster than the uncompressed plugin..... Regarding cache, do we do any sort of consistency checking for RAM, or do we leave that to some of the stranger kernel patches -- or just an occasional memtest? ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 19:17 ` Clay Barnes 2006-07-31 19:29 ` Jan-Benedict Glaw @ 2006-07-31 19:42 ` Alan Cox 2006-07-31 19:34 ` Clay Barnes 2006-07-31 21:00 ` Gregory Maxwell 1 sibling, 2 replies; 639+ messages in thread From: Alan Cox @ 2006-07-31 19:42 UTC (permalink / raw) To: Clay Barnes Cc: Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Ar Llu, 2006-07-31 am 12:17 -0700, ysgrifennodd Clay Barnes: > Of course, if ext3 were proven to be more robust against failures, I bet > the reiser team would be very interested in all the forensic data you > can offer, since, from what I've seen, they are always trying to make > reiser as good as possible---in speed, flexability, *and* robustness. Its well accepted that reiserfs3 has some robustness problems in the face of physical media errors. The structure of the file system and the tree basis make it very hard to avoid such problems. XFS appears to have managed to achieve both robustness and better data structures. How reiser4 compares I've no idea. Alan ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 19:42 ` Alan Cox @ 2006-07-31 19:34 ` Clay Barnes 2006-07-31 21:00 ` Gregory Maxwell 1 sibling, 0 replies; 639+ messages in thread From: Clay Barnes @ 2006-07-31 19:34 UTC (permalink / raw) To: Alan Cox Cc: Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On 20:42 Mon 31 Jul , Alan Cox wrote: > Ar Llu, 2006-07-31 am 12:17 -0700, ysgrifennodd Clay Barnes: > > Of course, if ext3 were proven to be more robust against failures, I bet > > the reiser team would be very interested in all the forensic data you > > can offer, since, from what I've seen, they are always trying to make > > reiser as good as possible---in speed, flexability, *and* robustness. > > Its well accepted that reiserfs3 has some robustness problems in the > face of physical media errors. The structure of the file system and the > tree basis make it very hard to avoid such problems. XFS appears to have > managed to achieve both robustness and better data structures. Yes, that is true, and I think that's a big motivator for the reiser team to get reiser4 in a place where people can't say that. I suspect that they know that reiserfs's shortcomings in that respect are probably the biggest deterrent to using that fs, and they'll do everything they can to prevent such a problem in reiser4. That's pure conjecture based on the stuff I see on the list, so if I'm wrong, reiser people, please correct me. --Clay > > How reiser4 compares I've no idea. > > Alan ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 19:42 ` Alan Cox 2006-07-31 19:34 ` Clay Barnes @ 2006-07-31 21:00 ` Gregory Maxwell 2006-07-31 21:40 ` Alan Cox 2006-07-31 21:54 ` Jeff V. Merkey 1 sibling, 2 replies; 639+ messages in thread From: Gregory Maxwell @ 2006-07-31 21:00 UTC (permalink / raw) To: Alan Cox Cc: Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote: > Its well accepted that reiserfs3 has some robustness problems in the > face of physical media errors. The structure of the file system and the > tree basis make it very hard to avoid such problems. XFS appears to have > managed to achieve both robustness and better data structures. > > How reiser4 compares I've no idea. Citation? I ask because your clam differs from the only detailed research that I'm aware of on the subject[1]. In figure 2 of the iron filesystems paper that Ext3 is show to ignore a great number of data-loss inducing failure conditions that Reiser3 detects an panics under. Are you sure that you aren't commenting on cases where Reiser3 alerts the user to a critical data condition (via a panic) which leads to a trouble report while ext3 ignores the problem which suppresses the trouble report from the user? *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 21:00 ` Gregory Maxwell @ 2006-07-31 21:40 ` Alan Cox 2006-07-31 21:43 ` David Masover 2006-07-31 21:54 ` Jeff V. Merkey 1 sibling, 1 reply; 639+ messages in thread From: Alan Cox @ 2006-07-31 21:40 UTC (permalink / raw) To: Gregory Maxwell Cc: Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Ar Llu, 2006-07-31 am 17:00 -0400, ysgrifennodd Gregory Maxwell: > On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote: > > Its well accepted that reiserfs3 has some robustness problems in the > > face of physical media errors. The structure of the file system and the > > tree basis make it very hard to avoid such problems. XFS appears to have > > managed to achieve both robustness and better data structures. > > > > How reiser4 compares I've no idea. > > Citation? Two sources, the cases I've looked at myself when IDE maintainer and also comments Hans made when we met at UKUUG a few years ago. Generally speaking on an IDE failure that lost chunks of disk the ext2/ext3 users got most of their data back. The reiserfs ones sometimes got it back and sometimes got catastrophic failure. The ext3 fsck is extremely effective in the face of serious errors. Some of that is clever code that knows about things like rewriting inode blocks to force reallocation of failed metadata by the drive but the majority of it is simply because you *know* where inode X is on the disk rather than having to deal with data structures and walk them. That's a tradeoff with the reiser performance with small files and until recently the reiser large directory performance. Which is right is another question altogether. Its also something I think XFS demonstrates can be done better so doesn't invalidate the theory behind reiserfs just the rev 3 implementation. > Are you sure that you aren't commenting on cases where Reiser3 alerts > the user to a critical data condition (via a panic) which leads to a > trouble report while ext3 ignores the problem which suppresses the > trouble report from the user? man mount Ext3 is configurable, and has been for years via the errors= option. Alan ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 21:40 ` Alan Cox @ 2006-07-31 21:43 ` David Masover 0 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-07-31 21:43 UTC (permalink / raw) To: Alan Cox Cc: Gregory Maxwell, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Alan Cox wrote: > Ar Llu, 2006-07-31 am 17:00 -0400, ysgrifennodd Gregory Maxwell: >> Are you sure that you aren't commenting on cases where Reiser3 alerts >> the user to a critical data condition (via a panic) which leads to a >> trouble report while ext3 ignores the problem which suppresses the >> trouble report from the user? > > man mount > > Ext3 is configurable, and has been for years via the errors= option. Sure, but I think the suggestion is that the reason we generally see more ReiserFS complaints than ext3 complaints might be because of the default level of errors logged. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 21:00 ` Gregory Maxwell 2006-07-31 21:40 ` Alan Cox @ 2006-07-31 21:54 ` Jeff V. Merkey 2006-07-31 22:02 ` Jeff V. Merkey 2006-07-31 22:56 ` Nate Diller 1 sibling, 2 replies; 639+ messages in thread From: Jeff V. Merkey @ 2006-07-31 21:54 UTC (permalink / raw) To: Gregory Maxwell Cc: Alan Cox, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Gregory Maxwell wrote: > On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote: > >> Its well accepted that reiserfs3 has some robustness problems in the >> face of physical media errors. The structure of the file system and the >> tree basis make it very hard to avoid such problems. XFS appears to have >> managed to achieve both robustness and better data structures. >> >> How reiser4 compares I've no idea. > > > Citation? > > I ask because your clam differs from the only detailed research that > I'm aware of on the subject[1]. In figure 2 of the iron filesystems > paper that Ext3 is show to ignore a great number of data-loss inducing > failure conditions that Reiser3 detects an panics under. > > Are you sure that you aren't commenting on cases where Reiser3 alerts > the user to a critical data condition (via a panic) which leads to a > trouble report while ext3 ignores the problem which suppresses the > trouble report from the user? > > *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf Hi Gregory, Wikimedia Foundation and LKML? How's Wikimania going. :-) What he says is correct. I have seen some serious issues with reiserfs in terms of stability and data corruption. Resier is however FASTER, but the statement is has robustness issues is accurate. I was using reiserfs but we opted to make EXT3 the default for Solera appliances, even when using Suse 10 due to issues I have seen with data corruption and hard hangs on RAID 0 read/write sector errors. I have stopped using it for local drives and based everything on EXT3. Not to say it won't get there eventually, but file systems have to endure a lot of time in the field and deployment befor they are ready for prime time. The Wikimedia appliances use Wolf Mountain, and I've tested it for about 4 months with few problems, but I only use it for hosting the Cherokee Langauge Wikipedia. It's performance is several magnitudes better than either EXT3 or ReiserFS. Despite this, for vertical wiki servers, its ok to go out with, folks can specifiy whether they want appliances with EXT3, Reiser, or WMFS, but iit's a long way from being "cooked" completely, though it does scale to 1 exabyte FS images. Reiser does have issues still, and I hestitate to standardize on it until I stop seeing reports from the field about corruption and failover issues. Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 21:54 ` Jeff V. Merkey @ 2006-07-31 22:02 ` Jeff V. Merkey 2006-07-31 22:21 ` Jeff V. Merkey 2006-07-31 22:56 ` Nate Diller 1 sibling, 1 reply; 639+ messages in thread From: Jeff V. Merkey @ 2006-07-31 22:02 UTC (permalink / raw) To: Jeff V. Merkey Cc: Gregory Maxwell, Alan Cox, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Jeff V. Merkey wrote: > Gregory Maxwell wrote: > >> On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote: >> >>> Its well accepted that reiserfs3 has some robustness problems in the >>> face of physical media errors. The structure of the file system and the >>> tree basis make it very hard to avoid such problems. XFS appears to >>> have >>> managed to achieve both robustness and better data structures. >>> >>> How reiser4 compares I've no idea. >> >> >> >> Citation? >> >> I ask because your clam differs from the only detailed research that >> I'm aware of on the subject[1]. In figure 2 of the iron filesystems >> paper that Ext3 is show to ignore a great number of data-loss inducing >> failure conditions that Reiser3 detects an panics under. >> >> Are you sure that you aren't commenting on cases where Reiser3 alerts >> the user to a critical data condition (via a panic) which leads to a >> trouble report while ext3 ignores the problem which suppresses the >> trouble report from the user? >> >> *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf > > > Hi Gregory, Wikimedia Foundation and LKML? > How's Wikimania going. :-) > > What he says is correct. I have seen some serious issues with > reiserfs in terms of stability and > data corruption. Resier is however FASTER, but the statement is has > robustness issues is accurate. > I was using reiserfs but we opted to make EXT3 the default for Solera > appliances, even when using Suse 10 > due to issues I have seen with data corruption and hard hangs on RAID > 0 read/write sector errors. I have > stopped using it for local drives and based everything on EXT3. Not > to say it won't get there eventually, but > file systems have to endure a lot of time in the field and deployment > befor they are ready for prime time. > Correction, That's "MediWiki" appliances. Two many transposed acronyms... www.wolfmountaingroup.com :-) Jeff > The Wikimedia appliances use Wolf Mountain, and I've tested it for > about 4 months with few problems, but > I only use it for hosting the Cherokee Langauge Wikipedia. It's > performance is several magnitudes better > than either EXT3 or ReiserFS. Despite this, for vertical wiki > servers, its ok to go out with, folks can specifiy > whether they want appliances with EXT3, Reiser, or WMFS, but iit's a > long way from being "cooked" > completely, though it does scale to 1 exabyte FS images. > Reiser does have issues still, and I hestitate to standardize on it > until I stop seeing reports from the field about > corruption and failover issues. > > Jeff > > - > To unsubscribe from this list: send the line "unsubscribe > linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 22:02 ` Jeff V. Merkey @ 2006-07-31 22:21 ` Jeff V. Merkey 0 siblings, 0 replies; 639+ messages in thread From: Jeff V. Merkey @ 2006-07-31 22:21 UTC (permalink / raw) To: Jeff V. Merkey Cc: Gregory Maxwell, Alan Cox, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list > > Correction, > > That's "MediWiki" appliances. Two many transposed acronyms... > > www.wolfmountaingroup.com > > :-) > > Jeff > And WMFS does not belong to the WolfMountainGroup any longer. It has been acquired by another company, so you won't see any info about it on the website. It's has been rolled into another company. I can bundle it with appliances but it is no longer the property of Wolf Mountain Group. WMG is a MediaWiki/Wikipedia Appliance company that does Machine translations of Wikipedia into several Native American Languages. The Translator is Linux and Windows based, but WMFS has a new home now. Just to clarify. Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 21:54 ` Jeff V. Merkey 2006-07-31 22:02 ` Jeff V. Merkey @ 2006-07-31 22:56 ` Nate Diller 2006-07-31 23:52 ` Jeff V. Merkey 1 sibling, 1 reply; 639+ messages in thread From: Nate Diller @ 2006-07-31 22:56 UTC (permalink / raw) To: Jeff V. Merkey Cc: Gregory Maxwell, Alan Cox, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On 7/31/06, Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote: > Gregory Maxwell wrote: > > > On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote: > > > >> Its well accepted that reiserfs3 has some robustness problems in the > >> face of physical media errors. The structure of the file system and the > >> tree basis make it very hard to avoid such problems. XFS appears to have > >> managed to achieve both robustness and better data structures. > >> > >> How reiser4 compares I've no idea. > > > > > > Citation? > > > > I ask because your clam differs from the only detailed research that > > I'm aware of on the subject[1]. In figure 2 of the iron filesystems > > paper that Ext3 is show to ignore a great number of data-loss inducing > > failure conditions that Reiser3 detects an panics under. > > > > Are you sure that you aren't commenting on cases where Reiser3 alerts > > the user to a critical data condition (via a panic) which leads to a > > trouble report while ext3 ignores the problem which suppresses the > > trouble report from the user? > > > > *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf > > Hi Gregory, Wikimedia Foundation and LKML? > > How's Wikimania going. :-) > > What he says is correct. I have seen some serious issues with reiserfs > in terms of stability and > data corruption. Resier is however FASTER, but the statement is has > robustness issues is accurate. > I was using reiserfs but we opted to make EXT3 the default for Solera > appliances, even when using Suse 10 > due to issues I have seen with data corruption and hard hangs on RAID 0 > read/write sector errors. I have > stopped using it for local drives and based everything on EXT3. Not to > say it won't get there eventually, but > file systems have to endure a lot of time in the field and deployment > befor they are ready for prime time. > > The Wikimedia appliances use Wolf Mountain, and I've tested it for about > 4 months with few problems, but > I only use it for hosting the Cherokee Langauge Wikipedia. It's > performance is several magnitudes better > than either EXT3 or ReiserFS. Despite this, for vertical wiki servers, > its ok to go out with, folks can specifiy > whether they want appliances with EXT3, Reiser, or WMFS, but iit's a > long way from being "cooked" > completely, though it does scale to 1 exabyte FS images. i've seen you mention the Wolf Mountain FS in other emails, but google isn't telling me a lot about it. Do you have a whitepaper? are there any published benchmark results? what sort of workloads do you benchmark? NATE ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 22:56 ` Nate Diller @ 2006-07-31 23:52 ` Jeff V. Merkey 2006-07-31 23:43 ` Nate Diller 0 siblings, 1 reply; 639+ messages in thread From: Jeff V. Merkey @ 2006-07-31 23:52 UTC (permalink / raw) To: Nate Diller Cc: Gregory Maxwell, Alan Cox, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Nate Diller wrote: > On 7/31/06, Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote: > >> Gregory Maxwell wrote: >> >> > On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote: >> > >> >> Its well accepted that reiserfs3 has some robustness problems in the >> >> face of physical media errors. The structure of the file system >> and the >> >> tree basis make it very hard to avoid such problems. XFS appears >> to have >> >> managed to achieve both robustness and better data structures. >> >> >> >> How reiser4 compares I've no idea. >> > >> > >> > Citation? >> > >> > I ask because your clam differs from the only detailed research that >> > I'm aware of on the subject[1]. In figure 2 of the iron filesystems >> > paper that Ext3 is show to ignore a great number of data-loss inducing >> > failure conditions that Reiser3 detects an panics under. >> > >> > Are you sure that you aren't commenting on cases where Reiser3 alerts >> > the user to a critical data condition (via a panic) which leads to a >> > trouble report while ext3 ignores the problem which suppresses the >> > trouble report from the user? >> > >> > *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf >> >> Hi Gregory, Wikimedia Foundation and LKML? >> >> How's Wikimania going. :-) >> >> What he says is correct. I have seen some serious issues with reiserfs >> in terms of stability and >> data corruption. Resier is however FASTER, but the statement is has >> robustness issues is accurate. >> I was using reiserfs but we opted to make EXT3 the default for Solera >> appliances, even when using Suse 10 >> due to issues I have seen with data corruption and hard hangs on RAID 0 >> read/write sector errors. I have >> stopped using it for local drives and based everything on EXT3. Not to >> say it won't get there eventually, but >> file systems have to endure a lot of time in the field and deployment >> befor they are ready for prime time. >> >> The Wikimedia appliances use Wolf Mountain, and I've tested it for about >> 4 months with few problems, but >> I only use it for hosting the Cherokee Langauge Wikipedia. It's >> performance is several magnitudes better >> than either EXT3 or ReiserFS. Despite this, for vertical wiki servers, >> its ok to go out with, folks can specifiy >> whether they want appliances with EXT3, Reiser, or WMFS, but iit's a >> long way from being "cooked" >> completely, though it does scale to 1 exabyte FS images. > > > i've seen you mention the Wolf Mountain FS in other emails, but google > isn't telling me a lot about it. Do you have a whitepaper? are there > any published benchmark results? what sort of workloads do you > benchmark? > > NATE > Wikipedia is the app for now. I have not done any benchmarks on the FS side, just the capture side, and its been transferred to another entity. I have no idea what they are naming it to, but I expect you may hear about it soon. One of the incarnations of it is Solera's DSFS which can be reviewed here: www.soleranetworks.com I can sustain 850 MB/S throughput from user space with it -- about 5 x any other FS. On some hardware, I've broken the 1.25 GB/S (gigabyte/second) windows with it. Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 23:52 ` Jeff V. Merkey @ 2006-07-31 23:43 ` Nate Diller 2006-08-01 0:15 ` Jeffrey V. Merkey 0 siblings, 1 reply; 639+ messages in thread From: Nate Diller @ 2006-07-31 23:43 UTC (permalink / raw) To: Jeff V. Merkey Cc: Gregory Maxwell, Alan Cox, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On 7/31/06, Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote: > Nate Diller wrote: > > > On 7/31/06, Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote: > > > >> Gregory Maxwell wrote: > >> > >> > On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote: > >> > > >> >> Its well accepted that reiserfs3 has some robustness problems in the > >> >> face of physical media errors. The structure of the file system > >> and the > >> >> tree basis make it very hard to avoid such problems. XFS appears > >> to have > >> >> managed to achieve both robustness and better data structures. > >> >> > >> >> How reiser4 compares I've no idea. > >> > > >> > > >> > Citation? > >> > > >> > I ask because your clam differs from the only detailed research that > >> > I'm aware of on the subject[1]. In figure 2 of the iron filesystems > >> > paper that Ext3 is show to ignore a great number of data-loss inducing > >> > failure conditions that Reiser3 detects an panics under. > >> > > >> > Are you sure that you aren't commenting on cases where Reiser3 alerts > >> > the user to a critical data condition (via a panic) which leads to a > >> > trouble report while ext3 ignores the problem which suppresses the > >> > trouble report from the user? > >> > > >> > *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf > >> > >> Hi Gregory, Wikimedia Foundation and LKML? > >> > >> How's Wikimania going. :-) > >> > >> What he says is correct. I have seen some serious issues with reiserfs > >> in terms of stability and > >> data corruption. Resier is however FASTER, but the statement is has > >> robustness issues is accurate. > >> I was using reiserfs but we opted to make EXT3 the default for Solera > >> appliances, even when using Suse 10 > >> due to issues I have seen with data corruption and hard hangs on RAID 0 > >> read/write sector errors. I have > >> stopped using it for local drives and based everything on EXT3. Not to > >> say it won't get there eventually, but > >> file systems have to endure a lot of time in the field and deployment > >> befor they are ready for prime time. > >> > >> The Wikimedia appliances use Wolf Mountain, and I've tested it for about > >> 4 months with few problems, but > >> I only use it for hosting the Cherokee Langauge Wikipedia. It's > >> performance is several magnitudes better > >> than either EXT3 or ReiserFS. Despite this, for vertical wiki servers, > >> its ok to go out with, folks can specifiy > >> whether they want appliances with EXT3, Reiser, or WMFS, but iit's a > >> long way from being "cooked" > >> completely, though it does scale to 1 exabyte FS images. > > > > > > i've seen you mention the Wolf Mountain FS in other emails, but google > > isn't telling me a lot about it. Do you have a whitepaper? are there > > any published benchmark results? what sort of workloads do you > > benchmark? > > > > NATE > > > Wikipedia is the app for now. I have not done any benchmarks on the FS > side, just the capture side, and its been transferred to > another entity. I have no idea what they are naming it to, but I expect > you may hear about it soon. One of the incarnations > of it is Solera's DSFS which can be reviewed here: > > www.soleranetworks.com so this is a single stream, write only? ... > I can sustain 850 MB/S throughput from user space with it -- about 5 x > any other FS. On some hardware, I've broken > the 1.25 GB/S (gigabyte/second) windows with it. and you're saying it scales to much higher multi-spindle single-machine throughput. cool. i'd love to see a whitepaper, or failing that, have an off-list discussion of your approach and the various kernel limitations you ran up against in testing. i don't suppose they invited you to the Kernel Summit to talk about it, heh. NATE ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 23:43 ` Nate Diller @ 2006-08-01 0:15 ` Jeffrey V. Merkey 2006-08-01 6:28 ` Toby Thain 0 siblings, 1 reply; 639+ messages in thread From: Jeffrey V. Merkey @ 2006-08-01 0:15 UTC (permalink / raw) To: Nate Diller Cc: Gregory Maxwell, Alan Cox, Clay Barnes, Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Nate Diller wrote: > On 7/31/06, Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote: > >> Nate Diller wrote: >> >> > On 7/31/06, Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote: >> > >> >> Gregory Maxwell wrote: >> >> >> >> > On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote: >> >> > >> >> >> Its well accepted that reiserfs3 has some robustness problems >> in the >> >> >> face of physical media errors. The structure of the file system >> >> and the >> >> >> tree basis make it very hard to avoid such problems. XFS appears >> >> to have >> >> >> managed to achieve both robustness and better data structures. >> >> >> >> >> >> How reiser4 compares I've no idea. >> >> > >> >> > >> >> > Citation? >> >> > >> >> > I ask because your clam differs from the only detailed research >> that >> >> > I'm aware of on the subject[1]. In figure 2 of the iron filesystems >> >> > paper that Ext3 is show to ignore a great number of data-loss >> inducing >> >> > failure conditions that Reiser3 detects an panics under. >> >> > >> >> > Are you sure that you aren't commenting on cases where Reiser3 >> alerts >> >> > the user to a critical data condition (via a panic) which leads >> to a >> >> > trouble report while ext3 ignores the problem which suppresses the >> >> > trouble report from the user? >> >> > >> >> > *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf >> >> >> >> Hi Gregory, Wikimedia Foundation and LKML? >> >> >> >> How's Wikimania going. :-) >> >> >> >> What he says is correct. I have seen some serious issues with >> reiserfs >> >> in terms of stability and >> >> data corruption. Resier is however FASTER, but the statement is has >> >> robustness issues is accurate. >> >> I was using reiserfs but we opted to make EXT3 the default for Solera >> >> appliances, even when using Suse 10 >> >> due to issues I have seen with data corruption and hard hangs on >> RAID 0 >> >> read/write sector errors. I have >> >> stopped using it for local drives and based everything on EXT3. >> Not to >> >> say it won't get there eventually, but >> >> file systems have to endure a lot of time in the field and deployment >> >> befor they are ready for prime time. >> >> >> >> The Wikimedia appliances use Wolf Mountain, and I've tested it for >> about >> >> 4 months with few problems, but >> >> I only use it for hosting the Cherokee Langauge Wikipedia. It's >> >> performance is several magnitudes better >> >> than either EXT3 or ReiserFS. Despite this, for vertical wiki >> servers, >> >> its ok to go out with, folks can specifiy >> >> whether they want appliances with EXT3, Reiser, or WMFS, but iit's a >> >> long way from being "cooked" >> >> completely, though it does scale to 1 exabyte FS images. >> > >> > >> > i've seen you mention the Wolf Mountain FS in other emails, but google >> > isn't telling me a lot about it. Do you have a whitepaper? are there >> > any published benchmark results? what sort of workloads do you >> > benchmark? >> > >> > NATE >> > >> Wikipedia is the app for now. I have not done any benchmarks on the FS >> side, just the capture side, and its been transferred to >> another entity. I have no idea what they are naming it to, but I expect >> you may hear about it soon. One of the incarnations >> of it is Solera's DSFS which can be reviewed here: >> >> www.soleranetworks.com > > > so this is a single stream, write only? ... > >> I can sustain 850 MB/S throughput from user space with it -- about 5 x >> any other FS. On some hardware, I've broken >> the 1.25 GB/S (gigabyte/second) windows with it. > > > and you're saying it scales to much higher multi-spindle > single-machine throughput. cool. > > i'd love to see a whitepaper, or failing that, have an off-list > discussion of your approach and the various kernel limitations you ran > up against in testing. i don't suppose they invited you to the Kernel > Summit to talk about it, heh. > > NATE > The patents have been filed for over a year, and will publish in several weeks at uspto.gov -- that's the only acclaim I care for -- one that results in value for the industry and more patent protection for Linux and profits for folks. No, I have not been invited to the summit, probably because of the lawsuit I filed against some folks who were threatening my family -- Peter Anvin booted me off Kernel.org after allowing folks to pinch my code and copy my bash history files all over the internet, and several folks have stiffed me. I could care less. I keep creating cool technology, make tons of money off of it, and I have cultivated an excellent relationship with the Wikimedia Foundation, and I am now the principal contributor on the Cherokee Wikipedia. Wales even deleted the article folks had used to smear me and made folks rewrite it. Wales is a very nice man and good dude. I am content to contribute to Linux from a business viewpoint, and if the treatment I received from Anvin is par for kernel.org accounts, I don't care for one -- IP addresses are rather cheap on the internet. I was and have remained loyal to Linux through it all. I am appreciative of your interest. Check uspto.gov in next few weeks for published applications, it's all described there, distributed architecture and all. All my Wikilove. Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 0:15 ` Jeffrey V. Merkey @ 2006-08-01 6:28 ` Toby Thain 0 siblings, 0 replies; 639+ messages in thread From: Toby Thain @ 2006-08-01 6:28 UTC (permalink / raw) Cc: ReiserFS List On 1-Aug-06, at 4:15 AM, Jeffrey V. Merkey wrote: > ... I was and have remained loyal to Linux through it all. Except for that little fling with SCO, eh? Off topic, but no more so than your self-aggrandising. --T ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 18:43 ` Jan-Benedict Glaw 2006-07-31 19:17 ` Clay Barnes @ 2006-07-31 22:17 ` Matthias Andree 1 sibling, 0 replies; 639+ messages in thread From: Matthias Andree @ 2006-07-31 22:17 UTC (permalink / raw) To: Rudy Zijlstra, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Jan-Benedict Glaw schrieb am 2006-07-31: > > Massive hardware problems don't count. ext2/ext3 doesn't look much better in > > such cases. I had a machine with RAM gone bad (no ECC - I wonder what > > They do! Very much, actually. These happen In Real Life, so I have to > pay attention to them. Once you're in setups with > 10000 machines, > everything counts. At some certain point, you can even use HDD's > temperature sensors in old machines to diagnose dead fans. > > Everything that eases recovery for whatever reason is something you > have to pay attention to. The simplicity of ext{2,3} is something I > really fail to find proper words for. As well as the really good fsck. > Once seen a SIGSEGV'ing fsck, you really don't want to go there. The point is: If you've written data with broken hardware (RAM, bus, controllers - loads of them, CPU), what is on your disks is untrustworthy anyways, and fsck isn't going to repair your gzip file where every 64th bit has become a 1 or when the battery-backed write cache threw 60 MB down the drain... Of course, an fsck that crashes is unbearable, but that doesn't apply to "broken hardware" failures. You need backups with a few generations to avoid massively losing data. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 17:32 ` Jan-Benedict Glaw @ 2006-07-31 21:21 ` Łukasz Mierzwa 2006-07-31 18:11 ` Matthias Andree 2006-07-31 21:21 ` Łukasz Mierzwa 2 siblings, 0 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-07-31 21:21 UTC (permalink / raw) To: Jan-Benedict Glaw, LKML, reiserfs-list Dnia Mon, 31 Jul 2006 19:32:39 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de> napisał: > On Mon, 2006-07-31 18:44:33 +0200, Rudy Zijlstra <rudy@edsons.demon.nl> > wrote: >> On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote: >> > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich >> > <reiser4@blinkenlights.ch> wrote: >> > > A colleague of mine happened to create a ~300gb filesystem and >> started >> > > to migrate Mailboxes (Maildir-style format = many small files >> (1-3kb)) >> > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a >> > >> > So preparation work wasn't done. >> >> Of course you are right. Preparation work was not fully done. And using >> ext1 would also have been possible. I suspect you are still using ext1, >> cause with proper preparation it is perfectly usable. > > Oh, and before people start laughing at me, here are some personal or > friend's experiences with different filesystems: > > * reiser3: A HDD containing a reiser3 filesystem was tried to be > booted on a machine that fucked up DMA writes. Fortunately, it > crashed really soon (right after going for read-write.) After > rebooting the HDD on a sane PeeCee, it refused to boot. Starting > off some rescue system showed an _empty_ root filesystem. > > * A friend's XFS data partition (portable USB/FireWire HDD) once > crashed due to being hot-unplugged off the USB. The in-kernel XFS > driver refused to mount that thing again, and the tools also > refused to fix any errors. (Don't ask, no details at my hands...) > > * JFS just always worked for me. Though I've never ever had a broken > HDD where it (or it's tools) could have shown how well-done they > were, so from a crash-recovery point of view, it's untested. > > * Being a regular ext3 user, I had lots of broken HDDs containing > ext3 filesystems. For every single case, it has been easy fixing > the filesystem after cloning. Just _once_, fsck wasn't able to fix > something, so I did it manually with some disk editor. This worked > well because the on-disk data structures are actually as simple as > they are. Is this some kind of "who lost more files on what fs" competition? What's the prize? Topic subject sugests that it should cover something else. Please, let's get back on track. First of all, it's about reiser4 so can't we forget about other filesystem's unless it's got something to do with reiser4 merge? ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion @ 2006-07-31 21:21 ` Łukasz Mierzwa 0 siblings, 0 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-07-31 21:21 UTC (permalink / raw) To: Jan-Benedict Glaw, LKML, reiserfs-list Dnia Mon, 31 Jul 2006 19:32:39 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de> napisa³: > On Mon, 2006-07-31 18:44:33 +0200, Rudy Zijlstra <rudy@edsons.demon.nl> > wrote: >> On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote: >> > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich >> > <reiser4@blinkenlights.ch> wrote: >> > > A colleague of mine happened to create a ~300gb filesystem and >> started >> > > to migrate Mailboxes (Maildir-style format = many small files >> (1-3kb)) >> > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a >> > >> > So preparation work wasn't done. >> >> Of course you are right. Preparation work was not fully done. And using >> ext1 would also have been possible. I suspect you are still using ext1, >> cause with proper preparation it is perfectly usable. > > Oh, and before people start laughing at me, here are some personal or > friend's experiences with different filesystems: > > * reiser3: A HDD containing a reiser3 filesystem was tried to be > booted on a machine that fucked up DMA writes. Fortunately, it > crashed really soon (right after going for read-write.) After > rebooting the HDD on a sane PeeCee, it refused to boot. Starting > off some rescue system showed an _empty_ root filesystem. > > * A friend's XFS data partition (portable USB/FireWire HDD) once > crashed due to being hot-unplugged off the USB. The in-kernel XFS > driver refused to mount that thing again, and the tools also > refused to fix any errors. (Don't ask, no details at my hands...) > > * JFS just always worked for me. Though I've never ever had a broken > HDD where it (or it's tools) could have shown how well-done they > were, so from a crash-recovery point of view, it's untested. > > * Being a regular ext3 user, I had lots of broken HDDs containing > ext3 filesystems. For every single case, it has been easy fixing > the filesystem after cloning. Just _once_, fsck wasn't able to fix > something, so I did it manually with some disk editor. This worked > well because the on-disk data structures are actually as simple as > they are. Is this some kind of "who lost more files on what fs" competition? What's the prize? Topic subject sugests that it should cover something else. Please, let's get back on track. First of all, it's about reiser4 so can't we forget about other filesystem's unless it's got something to do with reiser4 merge? ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 16:22 ` Jan-Benedict Glaw 2006-07-31 16:44 ` David Masover 2006-07-31 16:44 ` Rudy Zijlstra @ 2006-07-31 16:47 ` Dan Oglesby 2006-07-31 17:16 ` Jan-Benedict Glaw 2 siblings, 1 reply; 639+ messages in thread From: Dan Oglesby @ 2006-07-31 16:47 UTC (permalink / raw) To: Jan-Benedict Glaw Cc: Adrian Ulrich, Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On Mon, 2006-07-31 at 18:22 +0200, Jan-Benedict Glaw wrote: > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich <reiser4@blinkenlights.ch> wrote: > > A colleague of mine happened to create a ~300gb filesystem and started > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a > > So preparation work wasn't done. > > MfG, JBG > I'd agree with that statement. The wrong filesystem was chosen at the beginning of the project. As someone who is currently planning to migrate ~100GB of stored mail to the Maildirs format, it was pretty clear early on that EXT3 would not cut it (from past and current experiences), and not just for the sake of calculating inodes. --Dan ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 16:47 ` Dan Oglesby @ 2006-07-31 17:16 ` Jan-Benedict Glaw 2006-07-31 17:34 ` Matthias Andree 2006-07-31 17:44 ` Dan Oglesby 0 siblings, 2 replies; 639+ messages in thread From: Jan-Benedict Glaw @ 2006-07-31 17:16 UTC (permalink / raw) To: Dan Oglesby Cc: Adrian Ulrich, Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 1454 bytes --] On Mon, 2006-07-31 11:47:00 -0500, Dan Oglesby <doglesby@teleformix.com> wrote: > On Mon, 2006-07-31 at 18:22 +0200, Jan-Benedict Glaw wrote: > > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich <reiser4@blinkenlights.ch> wrote: > > > A colleague of mine happened to create a ~300gb filesystem and started > > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) > > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a > > So preparation work wasn't done. > > As someone who is currently planning to migrate ~100GB of stored mail to > the Maildirs format, it was pretty clear early on that EXT3 would not > cut it (from past and current experiences), and not just for the sake of > calculating inodes. Uh? Where did you face a problem there? With maildir, you shouldn't face any problems IMO. Even users with zillions of mails should work properly with the dir_index stuff: tune2fs -O dir_index /dev/hdXX or alternatively (to start that for already existing directories): e2fsck -fD /dev/hdXX Of course, you'll always face a problem with lots of files in one directory at getdents() time (eg. opendir()/readdir()/closedir()), but this is a common limit for all filesystems. MfG, JBG -- Jan-Benedict Glaw jbglaw@lug-owl.de +49-172-7608481 Signature of: Alles wird gut! ...und heute wirds schon ein bißchen besser. the second : [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 17:16 ` Jan-Benedict Glaw @ 2006-07-31 17:34 ` Matthias Andree 2006-07-31 17:44 ` Dan Oglesby 1 sibling, 0 replies; 639+ messages in thread From: Matthias Andree @ 2006-07-31 17:34 UTC (permalink / raw) To: Dan Oglesby, Adrian Ulrich, Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Jan-Benedict Glaw schrieb am 2006-07-31: > Uh? Where did you face a problem there? > > With maildir, you shouldn't face any problems IMO. Even users with > zillions of mails should work properly with the dir_index stuff: > > tune2fs -O dir_index /dev/hdXX > > or alternatively (to start that for already existing directories): > > e2fsck -fD /dev/hdXX hat is not "alternatively", but "tune2fs first", then "e2fsck -fD" (which can't happen on a RW-mounted FS and you should only try this on your rootfs if you can reboot with magic sysrq or from a rescue system). -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 17:16 ` Jan-Benedict Glaw 2006-07-31 17:34 ` Matthias Andree @ 2006-07-31 17:44 ` Dan Oglesby 1 sibling, 0 replies; 639+ messages in thread From: Dan Oglesby @ 2006-07-31 17:44 UTC (permalink / raw) To: Jan-Benedict Glaw Cc: Adrian Ulrich, Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On Mon, 2006-07-31 at 19:16 +0200, Jan-Benedict Glaw wrote: > On Mon, 2006-07-31 11:47:00 -0500, Dan Oglesby <doglesby@teleformix.com> wrote: > > On Mon, 2006-07-31 at 18:22 +0200, Jan-Benedict Glaw wrote: > > > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich <reiser4@blinkenlights.ch> wrote: > > > > A colleague of mine happened to create a ~300gb filesystem and started > > > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) > > > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a > > > So preparation work wasn't done. > > > > As someone who is currently planning to migrate ~100GB of stored mail to > > the Maildirs format, it was pretty clear early on that EXT3 would not > > cut it (from past and current experiences), and not just for the sake of > > calculating inodes. > > Uh? Where did you face a problem there? > Past experiences dealing with systems that generate several thousand to tens of thousands of files a day, adding up to well in the millions over the course of normal production (this is not for a mail server, BTW). Once we got close to a million files, filesystem transactions started bogging the system, driving the load over 50. Simply switching to ReiserFS v3 allowed us to go well past the number of files EXT3 could handle reasonably and the max load stayed right around the number of CPUs the system contained (typically 8). There is a LOT going on with these systems, not just filesystem transactions. EXT3 does not do well in our environment at all. > With maildir, you shouldn't face any problems IMO. Even users with > zillions of mails should work properly with the dir_index stuff: > > tune2fs -O dir_index /dev/hdXX > > or alternatively (to start that for already existing directories): > > e2fsck -fD /dev/hdXX > > I've been tuning EXT3 to see what performance I can get for the mail server, and it's just not there compared to ReiserFS with minimal tuning. > Of course, you'll always face a problem with lots of files in one > directory at getdents() time (eg. opendir()/readdir()/closedir()), but > this is a common limit for all filesystems. > > MfG, JBG > Of course, but the issue is EXT3 does this a whole lot worse than ReiserFS v3 from my experiences. At any rate, that's about all I have to say about this issue. I'll be patiently waiting to see ReiserFS v4 included in the main kernel, so I have less hoops to jump through to implement the latest and greatest from Namesys. --Dan ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 15:59 ` Adrian Ulrich (?) (?) @ 2006-07-31 16:54 ` Matthias Andree 2006-07-31 17:56 ` Adrian Ulrich ` (2 more replies) -1 siblings, 3 replies; 639+ messages in thread From: Matthias Andree @ 2006-07-31 16:54 UTC (permalink / raw) To: Adrian Ulrich Cc: vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list (resending complete message to the list). Adrian Ulrich schrieb am 2006-07-31: > Hello Matthias, > > > This looks rather like an education issue rather than a technical limit. > > We aren't talking about the same issue: I was asking to do it > on-the-fly. Umounting the filesystem, running e2fsck and resize2fs > is something different ;-) There was stuff by Andreas Dilger, to support "online" resizing of mounted ext2 file systems. I never cared to look for this (does it support ext3, does it work with current kernels, merge status) since offline resizing was always sufficient for me. > A colleague of mine happened to create a ~300gb filesystem and started > to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) > to the new LUN. At about 70% the filesystem ran out of inodes; Well - easy to fix, newfs again with proper inode density (perhaps 1 per 2 kB) and redo the migration. Of course you're free to pay for a new file system if your fellow admin can't be bothered to remember newfs's -i option. > > Well, such "silly limitations"... looks like they are mostly hot air > > spewn by marketroids that need to justify people spending money on their > > new filesystem. > > Have you ever seen VxFS or WAFL in action? No I haven't. As long as they are commercial, it's not likely that I will. > Great to see that Sun ships a state-of-the-art Filesystem with > Solaris... I think linux should do the same... I think reallocating inodes for UFS and/or ext2/ext3 is possible, even online, but someone needs to write, debug and field-test the code to do that - possibly based on Andreas Dilger's earlier ext2 online resizing work. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 16:54 ` Matthias Andree @ 2006-07-31 17:56 ` Adrian Ulrich 2006-07-31 20:07 ` Matthias Andree 2006-07-31 19:41 ` Theodore Tso 2006-08-01 2:33 ` Hans Reiser 2 siblings, 1 reply; 639+ messages in thread From: Adrian Ulrich @ 2006-07-31 17:56 UTC (permalink / raw) To: Matthias Andree Cc: vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list > Well - easy to fix, newfs again with proper inode density (perhaps 1 per > 2 kB) and redo the migration. Ehr: Such a migration (on a very busy system) takes *some* time (weeks). Re-Doing (migrate users back / recreate the FS / start again) the whole thing isn't really an option.. > Of course you're free to pay for a new > file system if your fellow admin can't be bothered to remember newfs's > -i option. Let's face it: Shit happens and nobody is perfect. A filesystem should be flexible (modern..) and support Admin/User-needs. We wouldn't need ECC / Raid / UPS's in a perfect world. > > Have you ever seen VxFS or WAFL in action? > > No I haven't. As long as they are commercial, it's not likely that I > will. Why? NetApp WAFL-Blurb: http://www.netapp.com/library/tr/3002.pdf Maybe we should crop the Cc: list .. this is getting OT. -- Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 17:56 ` Adrian Ulrich @ 2006-07-31 20:07 ` Matthias Andree 2006-07-31 20:32 ` Adrian Ulrich 0 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-07-31 20:07 UTC (permalink / raw) To: Adrian Ulrich; +Cc: Matthias Andree, linux-kernel, reiserfs-list Adrian Ulrich schrieb am 2006-07-31: > Ehr: Such a migration (on a very busy system) takes *some* time (weeks). > Re-Doing (migrate users back / recreate the FS / start again) the whole > thing isn't really an option.. All the more important to think about FS requirements *before* newfs-ing if a quick "one day for rsync/star/dump+restore" isn't available. If you're hitting, for instance, the hash collision problem in reiser3, you're as dead as with a FS without inodes. > > > Have you ever seen VxFS or WAFL in action? > > > > No I haven't. As long as they are commercial, it's not likely that I > > will. > > Why? I'm trying to shift my focus away from computer administration and better file systems than old-style non-journalling, non-softupdates UFS are available today and more will follow. Cc: list weeded out. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 20:07 ` Matthias Andree @ 2006-07-31 20:32 ` Adrian Ulrich 0 siblings, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-07-31 20:32 UTC (permalink / raw) To: Matthias Andree; +Cc: matthias.andree, linux-kernel, reiserfs-list > All the more important to think about FS requirements *before* > newfs-ing if a quick "one day for rsync/star/dump+restore" isn't > available. If you're hitting, for instance, the hash collision problem > in reiser3, you're as dead as with a FS without inodes. Quoting myself: >> Let's face it: Shit happens and nobody is perfect. A filesystem should >> be flexible (modern..) and support Admin/User-needs. Of COURSE you should think BEFORE creating the filesystem. But if you somehow failed to do it 'the right thing' your filesystem shouldn't let you down. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion @ 2006-07-31 20:32 ` Adrian Ulrich 0 siblings, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-07-31 20:32 UTC (permalink / raw) Cc: matthias.andree, linux-kernel, reiserfs-list > All the more important to think about FS requirements *before* > newfs-ing if a quick "one day for rsync/star/dump+restore" isn't > available. If you're hitting, for instance, the hash collision problem > in reiser3, you're as dead as with a FS without inodes. Quoting myself: >> Let's face it: Shit happens and nobody is perfect. A filesystem should >> be flexible (modern..) and support Admin/User-needs. Of COURSE you should think BEFORE creating the filesystem. But if you somehow failed to do it 'the right thing' your filesystem shouldn't let you down. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 16:54 ` Matthias Andree 2006-07-31 17:56 ` Adrian Ulrich @ 2006-07-31 19:41 ` Theodore Tso 2006-07-31 22:53 ` Matthias Andree 2006-08-01 2:33 ` Hans Reiser 2 siblings, 1 reply; 639+ messages in thread From: Theodore Tso @ 2006-07-31 19:41 UTC (permalink / raw) To: Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list On Mon, Jul 31, 2006 at 06:54:06PM +0200, Matthias Andree wrote: > > > This looks rather like an education issue rather than a technical limit. > > > > We aren't talking about the same issue: I was asking to do it > > on-the-fly. Umounting the filesystem, running e2fsck and resize2fs > > is something different ;-) > > There was stuff by Andreas Dilger, to support "online" resizing of > mounted ext2 file systems. I never cared to look for this (does it > support ext3, does it work with current kernels, merge status) since > offline resizing was always sufficient for me. With the latest e2fsprogs and 2.6 kernels, the online resizing support has been merged in, and as long as the filesystem was created with space reserved for growing the filesystem (which is now the default, or if the filesystem has the off-line prepration step ext2prepare run on it), you can run resize2fs on a mounted filesystem and grow an ext2/3 filesystem on-line. And yes, you get more inodes as you add more disk blocks, using the original inode ratio that was established when the filesystem was created. - Ted ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 19:41 ` Theodore Tso @ 2006-07-31 22:53 ` Matthias Andree 0 siblings, 0 replies; 639+ messages in thread From: Matthias Andree @ 2006-07-31 22:53 UTC (permalink / raw) To: Theodore Tso, Adrian Ulrich, vonbrand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list Theodore Tso schrieb am 2006-07-31: > With the latest e2fsprogs and 2.6 kernels, the online resizing support > has been merged in, and as long as the filesystem was created with > space reserved for growing the filesystem (which is now the default, > or if the filesystem has the off-line prepration step ext2prepare run > on it), you can run resize2fs on a mounted filesystem and grow an > ext2/3 filesystem on-line. And yes, you get more inodes as you add > more disk blocks, using the original inode ratio that was established > when the filesystem was created. That's cool. The interesting part for some people would be, if I read past postings correctly, to change the inode ratio in an existing (perhaps even mounted) file system without losing data. (I'm not sure how many blocks have to be moved and/or changed for that purpose, because I know too little about the on-disk ext2 layout, but since block relocating is already in place for shrink support in the offline resizer, some of the work appears to be done already.) -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 16:54 ` Matthias Andree 2006-07-31 17:56 ` Adrian Ulrich 2006-07-31 19:41 ` Theodore Tso @ 2006-08-01 2:33 ` Hans Reiser 2 siblings, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-08-01 2:33 UTC (permalink / raw) To: Matthias Andree Cc: Adrian Ulrich, vonbrand, ipso, lkml, jeff, tytso, linux-kernel, reiserfs-list Matthias Andree wrote: > >>Have you ever seen VxFS or WAFL in action? >> >> > >No I haven't. As long as they are commercial, it's not likely that I >will. > > WAFL was well done. It has several innovations that I admire, including quota trees, non-support of fragments for performance reasons, and the basic WAFL notion applied to an NFS RAID special (though important) case. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 15:59 ` Adrian Ulrich ` (2 preceding siblings ...) (?) @ 2006-07-31 19:18 ` Horst H. von Brand 2006-07-31 20:31 ` Toby Thain 2006-07-31 20:57 ` Adrian Ulrich -1 siblings, 2 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-07-31 19:18 UTC (permalink / raw) To: Adrian Ulrich Cc: Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Adrian Ulrich <reiser4@blinkenlights.ch> wrote: [...] > ZFS uses 'dnodes'. The dnodes are allocated on demand from your > available space so running out of [di]nodes is impossible. > > Great to see that Sun ships a state-of-the-art Filesystem with > Solaris... I think linux should do the same... This would be worthwhile, if only to be able to futz around in Solaris-made filesystems. Are you volunteering? You'd probably need a friend in Solaris-land who passes you information on how things are done, and copies of filessytems to take apart, and so on. First question is if there are any restrictions (patent or otherwise) on doing this, just copying is out of the question due to (unfortunate) licence on Sun's part. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 19:18 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Horst H. von Brand @ 2006-07-31 20:31 ` Toby Thain 2006-07-31 20:57 ` Adrian Ulrich 1 sibling, 0 replies; 639+ messages in thread From: Toby Thain @ 2006-07-31 20:31 UTC (permalink / raw) To: Horst H.von Brand; +Cc: ReiserFS List On 31-Jul-06, at 11:18 PM, Horst H. von Brand wrote: > Adrian Ulrich <reiser4@blinkenlights.ch> wrote: > > [...] > >> ZFS uses 'dnodes'. The dnodes are allocated on demand from your >> available space so running out of [di]nodes is impossible. >> >> Great to see that Sun ships a state-of-the-art Filesystem with >> Solaris... I think linux should do the same... > > This would be worthwhile, if only to be able to futz around in > Solaris-made > filesystems. > > Are you volunteering? You'd probably need a friend in Solaris-land who > passes you information on how things are done, and copies of > filessytems to > take apart, and so on. > > First question is if there are any restrictions (patent or > otherwise) on > doing this, just copying is out of the question due to (unfortunate) > licence on Sun's part. This may eventually be good enough for "futzing around": http://zfs-on-fuse.blogspot.com/ > -- > Dr. Horst H. von Brand User #22616 counter.li.org > Departamento de Informatica Fono: +56 32 654431 > Universidad Tecnica Federico Santa Maria +56 32 654239 > Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 19:18 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Horst H. von Brand @ 2006-07-31 20:57 ` Adrian Ulrich 2006-07-31 20:57 ` Adrian Ulrich 1 sibling, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-07-31 20:57 UTC (permalink / raw) To: Horst H. von Brand Cc: matthias.andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list > > Great to see that Sun ships a state-of-the-art Filesystem with > > Solaris... I think linux should do the same... > > This would be worthwhile, if only to be able to futz around in Solaris-made > filesystems. s/I think linux should do the same/I think linux should include Reiser4/ ;-) > First question is if there are any restrictions (patent or otherwise) on > doing this, Quoting from http://it.sun.com/eventi/jc06/pdf/mi27_p5_poccia_virtualization.pdf > 47 ZFS patents added to CDDL patent commons But i'd rather like to see a Linux version of WAFL :-) ZFS didn't really impress me: The Volume-Manager is nice but the Filesystem.. well: It beats UFS .. sometimes ;-) See also: http://spam.workaround.ch/dull/postmark.txt A quick'n'dirty ZFS-vs-UFS-vs-Reiser3-vs-Reiser4-vs-Ext3 'benchmark' Regards, Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] @ 2006-07-31 20:57 ` Adrian Ulrich 0 siblings, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-07-31 20:57 UTC (permalink / raw) Cc: matthias.andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list > > Great to see that Sun ships a state-of-the-art Filesystem with > > Solaris... I think linux should do the same... > > This would be worthwhile, if only to be able to futz around in Solaris-made > filesystems. s/I think linux should do the same/I think linux should include Reiser4/ ;-) > First question is if there are any restrictions (patent or otherwise) on > doing this, Quoting from http://it.sun.com/eventi/jc06/pdf/mi27_p5_poccia_virtualization.pdf > 47 ZFS patents added to CDDL patent commons But i'd rather like to see a Linux version of WAFL :-) ZFS didn't really impress me: The Volume-Manager is nice but the Filesystem.. well: It beats UFS .. sometimes ;-) See also: http://spam.workaround.ch/dull/postmark.txt A quick'n'dirty ZFS-vs-UFS-vs-Reiser3-vs-Reiser4-vs-Ext3 'benchmark' Regards, Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 20:57 ` Adrian Ulrich (?) @ 2006-07-31 21:54 ` Matthias Andree 2006-07-31 23:21 ` Nate Diller -1 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-07-31 21:54 UTC (permalink / raw) To: Adrian Ulrich Cc: Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Adrian Ulrich wrote: > See also: http://spam.workaround.ch/dull/postmark.txt > > A quick'n'dirty ZFS-vs-UFS-vs-Reiser3-vs-Reiser4-vs-Ext3 'benchmark' Whatever Postmark does, this looks pretty besides the point. Are these actual transactions with the "D"urability guarantee? 3000/s doesn't look too much like you're doing synchronous I/O (else figures around 70/s perhaps 100/s would be more adequate), and cache exercise is rather irrelevant for databases that manage real (=valuable) data... -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 21:54 ` Matthias Andree @ 2006-07-31 23:21 ` Nate Diller 2006-07-31 23:27 ` David Lang 0 siblings, 1 reply; 639+ messages in thread From: Nate Diller @ 2006-07-31 23:21 UTC (permalink / raw) To: Matthias Andree Cc: Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On 7/31/06, Matthias Andree <matthias.andree@gmx.de> wrote: > Adrian Ulrich wrote: > > > See also: http://spam.workaround.ch/dull/postmark.txt > > > > A quick'n'dirty ZFS-vs-UFS-vs-Reiser3-vs-Reiser4-vs-Ext3 'benchmark' > > Whatever Postmark does, this looks pretty besides the point. why's that? postmark is one of the standard benchmarks... > Are these actual transactions with the "D"urability guarantee? > 3000/s doesn't look too much like you're doing synchronous I/O (else > figures around 70/s perhaps 100/s would be more adequate), and cache > exercise is rather irrelevant for databases that manage real (=valuable) > data... Data: 204.62 megabytes read (8.53 megabytes per second) 271.49 megabytes written (11.31 megabytes per second) looks pretty I/O bound to me, 11.31 MB/s isn't exactly your latest DDR RAM bandwidth. as far as the synchronous I/O question, Reiser4 in this case acts more like a log-based FS. That allows it to "overlap" synchronous operations that are being submitted by multiple threads. NATE ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 23:21 ` Nate Diller @ 2006-07-31 23:27 ` David Lang 2006-07-31 23:50 ` Nate Diller 0 siblings, 1 reply; 639+ messages in thread From: David Lang @ 2006-07-31 23:27 UTC (permalink / raw) To: Nate Diller Cc: Matthias Andree, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On Mon, 31 Jul 2006, Nate Diller wrote: > > On 7/31/06, Matthias Andree <matthias.andree@gmx.de> wrote: >> Adrian Ulrich wrote: >> >> > See also: http://spam.workaround.ch/dull/postmark.txt >> > >> > A quick'n'dirty ZFS-vs-UFS-vs-Reiser3-vs-Reiser4-vs-Ext3 'benchmark' >> >> Whatever Postmark does, this looks pretty besides the point. > > why's that? postmark is one of the standard benchmarks... > >> Are these actual transactions with the "D"urability guarantee? >> 3000/s doesn't look too much like you're doing synchronous I/O (else >> figures around 70/s perhaps 100/s would be more adequate), and cache >> exercise is rather irrelevant for databases that manage real (=valuable) >> data... > > Data: > 204.62 megabytes read (8.53 megabytes per second) > 271.49 megabytes written (11.31 megabytes per second) > > looks pretty I/O bound to me, 11.31 MB/s isn't exactly your latest DDR > RAM bandwidth. as far as the synchronous I/O question, Reiser4 in > this case acts more like a log-based FS. That allows it to "overlap" > synchronous operations that are being submitted by multiple threads. what you are missing is that apps that need to do lots of syncing (databases, mail servers) need to wait for the data to hit non-volitile media before the write is complete. this limits such apps to ~1 write per revolution of the platters (yes it's possible for a limited time to have multiple writes to different things happen to be on the same track, but the counter is the extra seek time needed between tracks) so any benchmark that shows more transactions then the media has revolutions is highly suspect (now if you have battery-backed cache, or the equivalent you can blow past these limits) on consumer (7200 rpm) drives this limit is 120/sec, on high-end drives (15Krpm scsi's this is 250/sec, and on the 10k rpm drives in the middle it's about 166/sec. David Lang ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 23:27 ` David Lang @ 2006-07-31 23:50 ` Nate Diller 2006-07-31 23:55 ` David Lang 0 siblings, 1 reply; 639+ messages in thread From: Nate Diller @ 2006-07-31 23:50 UTC (permalink / raw) To: David Lang Cc: Matthias Andree, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On 7/31/06, David Lang <dlang@digitalinsight.com> wrote: > On Mon, 31 Jul 2006, Nate Diller wrote: > > > > > On 7/31/06, Matthias Andree <matthias.andree@gmx.de> wrote: > >> Adrian Ulrich wrote: > >> > >> > See also: http://spam.workaround.ch/dull/postmark.txt > >> > > >> > A quick'n'dirty ZFS-vs-UFS-vs-Reiser3-vs-Reiser4-vs-Ext3 'benchmark' > >> > >> Whatever Postmark does, this looks pretty besides the point. > > > > why's that? postmark is one of the standard benchmarks... > > > >> Are these actual transactions with the "D"urability guarantee? > >> 3000/s doesn't look too much like you're doing synchronous I/O (else > >> figures around 70/s perhaps 100/s would be more adequate), and cache > >> exercise is rather irrelevant for databases that manage real (=valuable) > >> data... > > > > Data: > > 204.62 megabytes read (8.53 megabytes per second) > > 271.49 megabytes written (11.31 megabytes per second) > > > > looks pretty I/O bound to me, 11.31 MB/s isn't exactly your latest DDR > > RAM bandwidth. as far as the synchronous I/O question, Reiser4 in > > this case acts more like a log-based FS. That allows it to "overlap" > > synchronous operations that are being submitted by multiple threads. > > what you are missing is that apps that need to do lots of syncing (databases, > mail servers) need to wait for the data to hit non-volitile media before the > write is complete. this limits such apps to ~1 write per revolution of the > platters (yes it's possible for a limited time to have multiple writes to > different things happen to be on the same track, but the counter is the extra > seek time needed between tracks) this is true so long as there is only one thread submitting I/O and doing fsync(). for something like a mail server, it can run multi-threaded, and still get data integrity, if the changes are spread out across more than one file. > so any benchmark that shows more transactions then the media has revolutions is > highly suspect (now if you have battery-backed cache, or the equivalent you can > blow past these limits) not all workloads are completely serial, transactions themselves may have no inter-dependencies at all. so it depends on the benchmark, and what workload you're measuring. in cases like this, threading can have a big advantage. NATE ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 23:50 ` Nate Diller @ 2006-07-31 23:55 ` David Lang 2006-08-01 0:05 ` Nate Diller 0 siblings, 1 reply; 639+ messages in thread From: David Lang @ 2006-07-31 23:55 UTC (permalink / raw) To: Nate Diller Cc: Matthias Andree, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On Mon, 31 Jul 2006, Nate Diller wrote: > On 7/31/06, David Lang <dlang@digitalinsight.com> wrote: >> On Mon, 31 Jul 2006, Nate Diller wrote: >> >> > >> > On 7/31/06, Matthias Andree <matthias.andree@gmx.de> wrote: >> >> Adrian Ulrich wrote: >> >> >> >> > See also: http://spam.workaround.ch/dull/postmark.txt >> >> > >> >> > A quick'n'dirty ZFS-vs-UFS-vs-Reiser3-vs-Reiser4-vs-Ext3 'benchmark' >> >> >> >> Whatever Postmark does, this looks pretty besides the point. >> > >> > why's that? postmark is one of the standard benchmarks... >> > >> >> Are these actual transactions with the "D"urability guarantee? >> >> 3000/s doesn't look too much like you're doing synchronous I/O (else >> >> figures around 70/s perhaps 100/s would be more adequate), and cache >> >> exercise is rather irrelevant for databases that manage real (=valuable) >> >> data... >> > >> > Data: >> > 204.62 megabytes read (8.53 megabytes per second) >> > 271.49 megabytes written (11.31 megabytes per second) >> > >> > looks pretty I/O bound to me, 11.31 MB/s isn't exactly your latest DDR >> > RAM bandwidth. as far as the synchronous I/O question, Reiser4 in >> > this case acts more like a log-based FS. That allows it to "overlap" >> > synchronous operations that are being submitted by multiple threads. >> >> what you are missing is that apps that need to do lots of syncing >> (databases, >> mail servers) need to wait for the data to hit non-volitile media before >> the >> write is complete. this limits such apps to ~1 write per revolution of the >> platters (yes it's possible for a limited time to have multiple writes to >> different things happen to be on the same track, but the counter is the >> extra >> seek time needed between tracks) > > this is true so long as there is only one thread submitting I/O and > doing fsync(). for something like a mail server, it can run > multi-threaded, and still get data integrity, if the changes are > spread out across more than one file. only if those multiple files all happen to live (along with their metadata) on the same track. >> so any benchmark that shows more transactions then the media has >> revolutions is >> highly suspect (now if you have battery-backed cache, or the equivalent you >> can >> blow past these limits) > > not all workloads are completely serial, transactions themselves may > have no inter-dependencies at all. so it depends on the benchmark, > and what workload you're measuring. in cases like this, threading can > have a big advantage. in the real-world (and benchmarks that simulate it fairly) the data spans multiple tracks so your best case is considerably less then the max I listed becouse you frequently have to seek around a lot to do your writes to multiple places on disk. more threads running should mean that you are attempting to write to more places on disk, which will cause more seeks, dropping you further below the max. David Lang ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 23:55 ` David Lang @ 2006-08-01 0:05 ` Nate Diller 2006-08-01 1:02 ` Matthias Andree 0 siblings, 1 reply; 639+ messages in thread From: Nate Diller @ 2006-08-01 0:05 UTC (permalink / raw) To: David Lang Cc: Matthias Andree, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On 7/31/06, David Lang <dlang@digitalinsight.com> wrote: > On Mon, 31 Jul 2006, Nate Diller wrote: > > > On 7/31/06, David Lang <dlang@digitalinsight.com> wrote: > >> On Mon, 31 Jul 2006, Nate Diller wrote: > >> > >> > > >> > On 7/31/06, Matthias Andree <matthias.andree@gmx.de> wrote: > >> >> Adrian Ulrich wrote: > >> >> > >> >> > See also: http://spam.workaround.ch/dull/postmark.txt > >> >> > > >> >> > A quick'n'dirty ZFS-vs-UFS-vs-Reiser3-vs-Reiser4-vs-Ext3 'benchmark' > >> >> > >> >> Whatever Postmark does, this looks pretty besides the point. > >> > > >> > why's that? postmark is one of the standard benchmarks... > >> > > >> >> Are these actual transactions with the "D"urability guarantee? > >> >> 3000/s doesn't look too much like you're doing synchronous I/O (else > >> >> figures around 70/s perhaps 100/s would be more adequate), and cache > >> >> exercise is rather irrelevant for databases that manage real (=valuable) > >> >> data... > >> > > >> > Data: > >> > 204.62 megabytes read (8.53 megabytes per second) > >> > 271.49 megabytes written (11.31 megabytes per second) > >> > > >> > looks pretty I/O bound to me, 11.31 MB/s isn't exactly your latest DDR > >> > RAM bandwidth. as far as the synchronous I/O question, Reiser4 in > >> > this case acts more like a log-based FS. That allows it to "overlap" > >> > synchronous operations that are being submitted by multiple threads. > >> > >> what you are missing is that apps that need to do lots of syncing > >> (databases, > >> mail servers) need to wait for the data to hit non-volitile media before > >> the > >> write is complete. this limits such apps to ~1 write per revolution of the > >> platters (yes it's possible for a limited time to have multiple writes to > >> different things happen to be on the same track, but the counter is the > >> extra > >> seek time needed between tracks) > > > > this is true so long as there is only one thread submitting I/O and > > doing fsync(). for something like a mail server, it can run > > multi-threaded, and still get data integrity, if the changes are > > spread out across more than one file. > > only if those multiple files all happen to live (along with their metadata) on > the same track. this is only a limitation for filesystems which do in-place data and metadata updates. this is why i mentioned the similarities to log file systems (see rosenblum and ousterhout, 1991). they observed an order-of-magnitude increase in performance for such workloads on their system. > >> so any benchmark that shows more transactions then the media has > >> revolutions is > >> highly suspect (now if you have battery-backed cache, or the equivalent you > >> can > >> blow past these limits) > > > > not all workloads are completely serial, transactions themselves may > > have no inter-dependencies at all. so it depends on the benchmark, > > and what workload you're measuring. in cases like this, threading can > > have a big advantage. > > in the real-world (and benchmarks that simulate it fairly) the data spans > multiple tracks so your best case is considerably less then the max I listed > becouse you frequently have to seek around a lot to do your writes to multiple > places on disk. more threads running should mean that you are attempting to > write to more places on disk, which will cause more seeks, dropping you further > below the max. postmark is very much real world. reiser4 just doesn't always do in-place writes. NATE ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 0:05 ` Nate Diller @ 2006-08-01 1:02 ` Matthias Andree 2006-08-01 1:25 ` Nate Diller ` (2 more replies) 0 siblings, 3 replies; 639+ messages in thread From: Matthias Andree @ 2006-08-01 1:02 UTC (permalink / raw) To: Nate Diller Cc: David Lang, Matthias Andree, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On Mon, 31 Jul 2006, Nate Diller wrote: > this is only a limitation for filesystems which do in-place data and > metadata updates. this is why i mentioned the similarities to log > file systems (see rosenblum and ousterhout, 1991). they observed an > order-of-magnitude increase in performance for such workloads on their > system. It's well known that transactions that would thrash on UFS or ext2fs may have quieter access patterns with shorter strokes can benefit from logging, data journaling, whatever else turns seeks into serial writes. And then, the other question with wandering logs (to avoid double writes) and such, you start wondering how much fragmentation you get as the price to pay for avoiding seeks and double writes at the same time. TANSTAAFL, or how long the system can sustain such access patterns, particularly if it gets under memory pressure and must move. Even with lazy allocation and other optimizations, I question the validity of 3000/s or faster transaction frequencies. Even the 500 on ext3 are suspect, particularly with 7200/min (s)ATA crap. This sounds pretty much like the drive doing its best to shuffle blocks around in its 8 MB cache and lazily writing back. sdparm --clear=WCE /dev/sda # please. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 1:02 ` Matthias Andree @ 2006-08-01 1:25 ` Nate Diller 2006-08-01 1:31 ` David Masover 2006-08-01 7:51 ` Adrian Ulrich 2 siblings, 0 replies; 639+ messages in thread From: Nate Diller @ 2006-08-01 1:25 UTC (permalink / raw) To: Nate Diller, David Lang, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Cc: Matthias Andree On 7/31/06, Matthias Andree <matthias.andree@gmx.de> wrote: > On Mon, 31 Jul 2006, Nate Diller wrote: > > > this is only a limitation for filesystems which do in-place data and > > metadata updates. this is why i mentioned the similarities to log > > file systems (see rosenblum and ousterhout, 1991). they observed an > > order-of-magnitude increase in performance for such workloads on their > > system. > > It's well known that transactions that would thrash on UFS or ext2fs may > have quieter access patterns with shorter strokes can benefit from > logging, data journaling, whatever else turns seeks into serial writes. > And then, the other question with wandering logs (to avoid double > writes) and such, you start wondering how much fragmentation you get as > the price to pay for avoiding seeks and double writes at the same time. > TANSTAAFL, or how long the system can sustain such access patterns, > particularly if it gets under memory pressure and must move. Even with > lazy allocation and other optimizations, I question the validity of > 3000/s or faster transaction frequencies. Even the 500 on ext3 are > suspect, particularly with 7200/min (s)ATA crap. This sounds pretty much > like the drive doing its best to shuffle blocks around in its 8 MB cache > and lazily writing back. it's not my benchmark, and you are right to be interested in more information. I would be curious about such things as write barrier support, average/min/max transaction latency, and number of individual threads, as well as hardware specs. i also suspect that the numbers would be altered a bit by testing with different I/O schedulers. unfortunately, namesys has considered mongo a replacement for postmark, so i cannot point to any more rigorous postmark tests ATM. however, the results seem consistent with what i would expect for the various file systems, with a significant number of threads. after all, even ext3 has the benefit of a disk scheduler, especially if barriers are disabled NATE ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 1:02 ` Matthias Andree 2006-08-01 1:25 ` Nate Diller @ 2006-08-01 1:31 ` David Masover 2006-08-01 3:00 ` Theodore Tso 2006-08-01 4:21 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed " David Lang 2006-08-01 7:51 ` Adrian Ulrich 2 siblings, 2 replies; 639+ messages in thread From: David Masover @ 2006-08-01 1:31 UTC (permalink / raw) To: Nate Diller, David Lang, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Matthias Andree wrote: > On Mon, 31 Jul 2006, Nate Diller wrote: > >> this is only a limitation for filesystems which do in-place data and >> metadata updates. this is why i mentioned the similarities to log >> file systems (see rosenblum and ousterhout, 1991). they observed an >> order-of-magnitude increase in performance for such workloads on their >> system. > > It's well known that transactions that would thrash on UFS or ext2fs may > have quieter access patterns with shorter strokes can benefit from > logging, data journaling, whatever else turns seeks into serial writes. > And then, the other question with wandering logs (to avoid double > writes) and such, you start wondering how much fragmentation you get as > the price to pay for avoiding seeks and double writes at the same time. So you use a repacker. Nice thing about a repacker is, everyone has downtime. Better to plan to be a little sluggish when you'll have 1/10th or 1/50th of the users than be MUCH slower all the time. You're right, though, to ask the question: > TANSTAAFL, or how long the system can sustain such access patterns, > particularly if it gets under memory pressure and must move. Anyone care to run some very long benchmarks? > Even with > lazy allocation and other optimizations, I question the validity of > 3000/s or faster transaction frequencies. Even the 500 on ext3 are > suspect, particularly with 7200/min (s)ATA crap. This sounds pretty much > like the drive doing its best to shuffle blocks around in its 8 MB cache > and lazily writing back. Oh, I'm curious -- do hard drives ever carry enough battery/capacitance to cover their caches? It doesn't seem like it would be that hard/expensive, and if it is done that way, then I think it's valid to leave them on. You could just say that other filesystems aren't taking as much advantage of newer drive features as Reiser :P Anyway, remember that the primary tool of science is not logic. Logic is the primary tool of philosophy. The primary tool of science is observation. Sorry, the only machines I could really run this on are about to be in remote only mode for a couple weeks. I'm hesitant to hit them too hard. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 1:31 ` David Masover @ 2006-08-01 3:00 ` Theodore Tso 2006-08-01 3:38 ` David Masover ` (2 more replies) 2006-08-01 4:21 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed " David Lang 1 sibling, 3 replies; 639+ messages in thread From: Theodore Tso @ 2006-08-01 3:00 UTC (permalink / raw) To: David Masover Cc: Nate Diller, David Lang, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list On Mon, Jul 31, 2006 at 08:31:32PM -0500, David Masover wrote: > So you use a repacker. Nice thing about a repacker is, everyone has > downtime. Better to plan to be a little sluggish when you'll have > 1/10th or 1/50th of the users than be MUCH slower all the time. Actually, that's a problem with log-structured filesystems in general. There are quite a few real-life workloads where you *don't* have downtime. The thing is, in a global economy, you move from the London/European stock exchanges, to the New York/US exchanges, to the Asian exchanges, with little to no downtime available. In addition, people have been getting more sophisticated with workload consolidation tricks so that you use your "downtime" for other applications (either to service other parts of the world, or to do daily summaries, 3-d frame rendering at animation companies, etc.) So the assumption that there will always be time to run the repacker is a dangerous one. The problem is that many benchmarks (such as taring and untaring the kernel sources in reiser4 sort order) are overly simplistic, in that they don't really reflect how people use the filesystem in real life. (How many times can you guarantee that files will be written in the precise hash/tree order so that the filesystem gets the best possible time?) A more subtle version of this problem happens for filesystems where their performance degrades dramatically over-time without a repacker. If the benchmark doesn't take into account the need for repacker, or if the repacker is disabled or fails to run during the benchmark, the filesystem are in effect "cheating" on the benchmark because there is critical work which is necessary for the long-term health of the filesystem which is getting deferred until after the benchmark has finished measuring the performance of the system under test. This sort of marketing benchmarks ("lies, d*mn lies, and benchmarks") may be useful for trying to scam mainline acceptance of the filesystem code, or to make pretty graphs that make log-structured filesystems look good on Usenix papers, but despite the fact that huge numbers of papers were written about the lfs filesystem two decades ago, it never was used in real-life by any of the commercial Unix systems. This wasn't an accident, and it wasn't due to a secret conspiracy of BSD fast filesystem hackers keeping people from using lfs. No, the BSD lfs died on its own merits.... - Ted ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 3:00 ` Theodore Tso @ 2006-08-01 3:38 ` David Masover 2006-08-01 3:47 ` Timothy Webster 2006-08-01 4:38 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed " Mike Benoit 2 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-08-01 3:38 UTC (permalink / raw) To: Theodore Tso, David Masover, Nate Diller, David Lang, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list Theodore Tso wrote: > On Mon, Jul 31, 2006 at 08:31:32PM -0500, David Masover wrote: >> So you use a repacker. Nice thing about a repacker is, everyone has >> downtime. Better to plan to be a little sluggish when you'll have >> 1/10th or 1/50th of the users than be MUCH slower all the time. > > Actually, that's a problem with log-structured filesystems in general. > There are quite a few real-life workloads where you *don't* have > downtime. The thing is, in a global economy, you move from the > London/European stock exchanges, to the New York/US exchanges, to the > Asian exchanges, with little to no downtime available. Such systems must have redundancy, however. And if you have 2-3 servers hot in case one of them goes down, I can see switching between which is more active, and which is repacking. This repacker is online, hence a filesystem being repacked would have to be less active, not necessarily down. So repack the backup server, then make the backup server the active one and repack the main server. If the main server goes down while the backup is repacking, kill the repack process. I actually have a problem imagining a system where you don't have enough spare capacity (disk, CPU, spare servers) to run a repacker every now and then, but which also must have 100% uptime. What happens when a disk goes bad? Or when power to half the country goes out? Or... You get the idea. > In addition, > people have been getting more sophisticated with workload > consolidation tricks so that you use your "downtime" for other > applications (either to service other parts of the world, or to do > daily summaries, 3-d frame rendering at animation companies, etc.) So > the assumption that there will always be time to run the repacker is a > dangerous one. 3D frame rendering in particular doesn't require much disk use, does it? Daily summaries, I guess, depends on what kind of summaries they are. And anyway, those applications are making the same dangerous assumption. And anyway, I suspect the repacker will work best once a week or so, but no one knows yet, as they haven't written it yet. > The problem is that many benchmarks (such as taring and untaring the > kernel sources in reiser4 sort order) are overly simplistic, in that > they don't really reflect how people use the filesystem in real life. That's true. I'd also like to see lots more benchmarks. > If the benchmark doesn't take into account the need for > repacker, or if the repacker is disabled or fails to run during the > benchmark, the filesystem are in effect "cheating" on the benchmark > because there is critical work which is necessary for the long-term > health of the filesystem which is getting deferred until after the > benchmark has finished measuring the performance of the system under > test. In this case, the only fair test would be to run the benchmark 24/7 for a week, and run the repacker on a weekend. Or however you're planning to do it. It wouldn't be fair to run a 10-minute or 1-hour benchmark and then immediately run the repacker. But I'd also like to see more here, especially about fragmentation. If the repacker will cost money, the system should be reasonably good at avoiding fragmentation. I'm wondering if I should run a benchmark on my systems -- they're over a year old, and while they aren't under particularly heavy load, they should be getting somewhat fragmented by now. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 3:00 ` Theodore Tso 2006-08-01 3:38 ` David Masover @ 2006-08-01 3:47 ` Timothy Webster 2006-08-01 4:24 ` David Masover 2006-08-01 4:38 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed " Mike Benoit 2 siblings, 1 reply; 639+ messages in thread From: Timothy Webster @ 2006-08-01 3:47 UTC (permalink / raw) To: Theodore Tso, David Masover Cc: Nate Diller, David Lang, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list Different users have different needs. I agree, there are many users who can not afford any downtime. I worked at the NYSE and they reboot all their computers once a week. We had a policy at NYSE. If you suspect a computer has hardware problems, take it off line. It is better to be short a few computers then have that computer bring everything down. And fix that computer off line. Until last year I worked at world wide webmail provider. And trust me they could not avoid downtime. But the fact is everyone has down time, because hardware breaks and software is broken. However on the other hand if I am using a filesystem for recording TV programs or to play home computer games. Downtime is not the problem I really care about. I am rebooting lots anyway. The problem I see is managing disk errors. NOT repackers, unless ofcourse I need to run it all the time just to keep the filesystem in a usable state. The question is why not include lots of new filesystems. reiser4, ZFS They both have their own markets. And perhaps a really good clustering filesystem for markets that require NO downtime. -Tim --- Theodore Tso <tytso@mit.edu> wrote: > On Mon, Jul 31, 2006 at 08:31:32PM -0500, David > Masover wrote: > > So you use a repacker. Nice thing about a > repacker is, everyone has > > downtime. Better to plan to be a little sluggish > when you'll have > > 1/10th or 1/50th of the users than be MUCH slower > all the time. > > Actually, that's a problem with log-structured > filesystems in general. > There are quite a few real-life workloads where you > *don't* have > downtime. The thing is, in a global economy, you __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 3:47 ` Timothy Webster @ 2006-08-01 4:24 ` David Masover 2006-08-01 4:41 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed " David Lang 0 siblings, 1 reply; 639+ messages in thread From: David Masover @ 2006-08-01 4:24 UTC (permalink / raw) To: tdwebste2 Cc: Theodore Tso, Nate Diller, David Lang, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list Timothy Webster wrote: > Different users have different needs. I'm having trouble thinking of users who need an FS that doesn't need a repacker. The disk error problem, though, you're right -- most users will have to get bitten by this, hard, at least once, or they'll never get the importance of it. But it'd be nice if it's not too hard, and we can actually recover most of their files. Still, I can see most people who are aware of this problem using RAID, backups, and not caring if their filesystem tolerates bad hardware. > The problem I see is managing disk errors. I see this kind of the same way. If your disk has errors, you should be getting a new disk. If you can't do that, you can run a mirrored RAID -- even on SATA, you should be able to hotswap it. Even for a home/desktop user, disks are cheap, and getting cheaper all the time. All you have to do is run the mean time between failure numbers by them, and ask them if their backup is enough. > And perhaps a > really good clustering filesystem for markets that > require NO downtime. Thing is, a cluster is about the only FS I can imagine that could reasonably require (and MAYBE provide) absolutely no downtime. Everything else, the more you say it requires no downtime, the more I say it requires redundancy. Am I missing any more obvious examples where you can't have enough redundancy, but you can't have downtime either? ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 4:24 ` David Masover @ 2006-08-01 4:41 ` David Lang 2006-08-01 4:57 ` David Masover 2006-08-01 6:48 ` Theodore Tso 0 siblings, 2 replies; 639+ messages in thread From: David Lang @ 2006-08-01 4:41 UTC (permalink / raw) To: David Masover Cc: tdwebste2, Theodore Tso, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list On Mon, 31 Jul 2006, David Masover wrote: >> And perhaps a >> really good clustering filesystem for markets that >> require NO downtime. > > Thing is, a cluster is about the only FS I can imagine that could reasonably > require (and MAYBE provide) absolutely no downtime. Everything else, the more > you say it requires no downtime, the more I say it requires redundancy. > > Am I missing any more obvious examples where you can't have enough > redundancy, but you can't have downtime either? just becouse you have redundancy doesn't mean that your data is idle enough for you to run a repacker with your spare cycles. to run a repacker you need a time when the chunk of the filesystem that you are repacking is not being accessed or written to. it doesn't matter if that data lives on one disk or 9 disks all mirroring the same data, you can't just break off 1 of the copies and repack that becouse by the time you finish it won't match the live drives anymore. database servers have a repacker (vaccum), and they are under tremendous preasure from their users to avoid having to use it becouse of the performance hit that it generates. (the theory in the past is exactly what was presented in this thread, make things run faster most of the time and accept the performance hit when you repack). the trend seems to be for a repacker thread that runs continuously, causing a small impact all the time (that can be calculated into the capacity planning) instead of a large impact once in a while. the other thing they are seeing as new people start useing them is that the newbys don't realize they need to do somthing as archaic as running a repacker periodicly, as a result they let things devolve down to where performance is really bad without understanding why. David Lang ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 4:41 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed " David Lang @ 2006-08-01 4:57 ` David Masover 2006-08-01 6:48 ` Theodore Tso 1 sibling, 0 replies; 639+ messages in thread From: David Masover @ 2006-08-01 4:57 UTC (permalink / raw) To: David Lang Cc: tdwebste2, Theodore Tso, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list David Lang wrote: > On Mon, 31 Jul 2006, David Masover wrote: > >>> And perhaps a >>> really good clustering filesystem for markets that >>> require NO downtime. >> >> Thing is, a cluster is about the only FS I can imagine that could >> reasonably require (and MAYBE provide) absolutely no downtime. >> Everything else, the more you say it requires no downtime, the more I >> say it requires redundancy. >> >> Am I missing any more obvious examples where you can't have enough >> redundancy, but you can't have downtime either? > > just becouse you have redundancy doesn't mean that your data is idle > enough for you to run a repacker with your spare cycles. Then you don't have redundancy, at least not for reliability. In that case, you have redundancy for speed. > to run a > repacker you need a time when the chunk of the filesystem that you are > repacking is not being accessed or written to. Reasonably, yes. But it will be an online repacker, so it will be somewhat tolerant of this. > it doesn't matter if that > data lives on one disk or 9 disks all mirroring the same data, you can't > just break off 1 of the copies and repack that becouse by the time you > finish it won't match the live drives anymore. Aha. That really depends how you're doing the mirroring. If you're doing it at the block level, then no, it won't work. But if you're doing it at the filesystem level (a cluster-based FS, or something that layers on top of an FS), or (most likely) the database/application level, then when you come back up, the new data is just pulled in from the logs as if it had been written to the FS. The only example I can think of that I've actually used and seen working is MySQL tables, but that already covers a huge number of websites. > database servers have a repacker (vaccum), and they are under tremendous > preasure from their users to avoid having to use it becouse of the > performance hit that it generates. (the theory in the past is exactly > what was presented in this thread, make things run faster most of the > time and accept the performance hit when you repack). the trend seems to > be for a repacker thread that runs continuously, causing a small impact > all the time (that can be calculated into the capacity planning) instead > of a large impact once in a while. Hmm, if that could be done right, it wouldn't be so bad -- if you get twice the performance but have to repack for 2 hrs at the end of the week, repacker is better, right? So if you could spread the 2 hours out over the week, in theory, you'd still be pretty close to twice the performance. But that is fairly difficult to do, and may be more difficult to do well than to implement, say, a Reiser4 plugin that operates about on the level of rsync, but on every file modification. > the other thing they are seeing as new people start useing them is that > the newbys don't realize they need to do somthing as archaic as running > a repacker periodicly, as a result they let things devolve down to where > performance is really bad without understanding why. Yikes. But then, that may be a failure of distro maintainers for not throwing it in cron for them. I had a similar problem with MySQL. I turned on binary logging so I could do database replication, but I didn't realize I had to actually delete the logs. I now have a daily cron job that wipes out everything except the last day's logs. It could probably be modified pretty easily to run hourly, if I needed to. Moral of the story? Maybe there's something to this "continuous repacker" idea, but don't ruin a good thing for the rest of us because of newbies. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 4:41 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed " David Lang 2006-08-01 4:57 ` David Masover @ 2006-08-01 6:48 ` Theodore Tso 2006-08-01 2:40 ` Hans Reiser ` (2 more replies) 1 sibling, 3 replies; 639+ messages in thread From: Theodore Tso @ 2006-08-01 6:48 UTC (permalink / raw) To: David Lang Cc: David Masover, tdwebste2, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list On Mon, Jul 31, 2006 at 09:41:02PM -0700, David Lang wrote: > just becouse you have redundancy doesn't mean that your data is idle enough > for you to run a repacker with your spare cycles. to run a repacker you > need a time when the chunk of the filesystem that you are repacking is not > being accessed or written to. it doesn't matter if that data lives on one > disk or 9 disks all mirroring the same data, you can't just break off 1 of > the copies and repack that becouse by the time you finish it won't match > the live drives anymore. > > database servers have a repacker (vaccum), and they are under tremendous > preasure from their users to avoid having to use it becouse of the > performance hit that it generates. (the theory in the past is exactly what > was presented in this thread, make things run faster most of the time and > accept the performance hit when you repack). the trend seems to be for a > repacker thread that runs continuously, causing a small impact all the time > (that can be calculated into the capacity planning) instead of a large > impact once in a while. Ah, but as soon as the repacker thread runs continuously, then you lose all or most of the claimed advantage of "wandering logs". Specifically, the claim of the "wandering log" is that you don't have to write your data twice --- once to the log, and once to the final location on disk (whereas with ext3 you end up having to do double writes). But if the repacker is running continuously, you end up doing double writes anyway, as the repacker moves things from a location that is convenient for the log, to a location which is efficient for reading. Worse yet, if the repacker is moving disk blocks or objects which are no longer in cache, it may end up having to read objects in before writing them to a final location on disk. So instead of a write-write overhead, you end up with a write-read-write overhead. But of course, people tend to disable the repacker when doing benchmarks because they're trying to play the "my filesystem/database has bigger performance numbers than yours" game.... - Ted ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 6:48 ` Theodore Tso @ 2006-08-01 2:40 ` Hans Reiser 2006-08-01 10:59 ` Jan Engelhardt 2006-08-01 7:24 ` Avi Kivity 2006-08-01 17:03 ` David Masover 2 siblings, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-08-01 2:40 UTC (permalink / raw) To: Theodore Tso Cc: David Lang, David Masover, tdwebste2, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, lkml, jeff, linux-kernel, reiserfs-list, Alexander Zarochentcev Theodore Tso wrote: >On Mon, Jul 31, 2006 at 09:41:02PM -0700, David Lang wrote: > > >>just becouse you have redundancy doesn't mean that your data is idle enough >>for you to run a repacker with your spare cycles. to run a repacker you >>need a time when the chunk of the filesystem that you are repacking is not >>being accessed or written to. it doesn't matter if that data lives on one >>disk or 9 disks all mirroring the same data, you can't just break off 1 of >>the copies and repack that becouse by the time you finish it won't match >>the live drives anymore. >> >>database servers have a repacker (vaccum), and they are under tremendous >>preasure from their users to avoid having to use it becouse of the >>performance hit that it generates. (the theory in the past is exactly what >>was presented in this thread, make things run faster most of the time and >>accept the performance hit when you repack). the trend seems to be for a >>repacker thread that runs continuously, causing a small impact all the time >>(that can be calculated into the capacity planning) instead of a large >>impact once in a while. >> >> > >Ah, but as soon as the repacker thread runs continuously, then you >lose all or most of the claimed advantage of "wandering logs". > > Wandering logs is a term specific to reiser4, and I think you are making a more general remark. You are missing the implications of the oft-cited statistic that 80% of files never or rarely move. You are also missing the implications of the repacker being able to do larger IOs than occur for a random tiny IO workload which is impacting a filesystem that is performing allocations on the fly. >Specifically, the claim of the "wandering log" is that you don't have >to write your data twice --- once to the log, and once to the final >location on disk (whereas with ext3 you end up having to do double >writes). But if the repacker is running continuously, you end up >doing double writes anyway, as the repacker moves things from a >location that is convenient for the log, to a location which is >efficient for reading. Worse yet, if the repacker is moving disk >blocks or objects which are no longer in cache, it may end up having >to read objects in before writing them to a final location on disk. >So instead of a write-write overhead, you end up with a >write-read-write overhead. > >But of course, people tend to disable the repacker when doing >benchmarks because they're trying to play the "my filesystem/database >has bigger performance numbers than yours" game.... > > When the repacker is done, we will just for you run one of our benchmarks the morning after the repacker is run (and reference this email);-).... that was what you wanted us to do to address your concern, yes?;-) > - Ted > > > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 2:40 ` Hans Reiser @ 2006-08-01 10:59 ` Jan Engelhardt 2006-08-01 10:56 ` Hans Reiser 0 siblings, 1 reply; 639+ messages in thread From: Jan Engelhardt @ 2006-08-01 10:59 UTC (permalink / raw) To: Hans Reiser Cc: Theodore Tso, David Lang, David Masover, tdwebste2, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, lkml, jeff, linux-kernel, reiserfs-list, Alexander Zarochentcev > >Wandering logs is a term specific to reiser4, and I think you are making >a more general remark. So, what is UDF's "wandering" log then? Jan Engelhardt -- ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 10:59 ` Jan Engelhardt @ 2006-08-01 10:56 ` Hans Reiser 0 siblings, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-08-01 10:56 UTC (permalink / raw) To: Jan Engelhardt Cc: Theodore Tso, David Lang, David Masover, tdwebste2, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, lkml, jeff, linux-kernel, reiserfs-list, Alexander Zarochentcev Jan Engelhardt wrote: >>Wandering logs is a term specific to reiser4, and I think you are making >>a more general remark. >> >> > >So, what is UDF's "wandering" log then? > > > >Jan Engelhardt > > I have no idea, when did they introduce it? ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 6:48 ` Theodore Tso 2006-08-01 2:40 ` Hans Reiser @ 2006-08-01 7:24 ` Avi Kivity 2006-08-01 9:25 ` Matthias Andree 2006-08-01 17:03 ` David Masover 2 siblings, 1 reply; 639+ messages in thread From: Avi Kivity @ 2006-08-01 7:24 UTC (permalink / raw) To: Theodore Tso Cc: David Lang, David Masover, tdwebste2, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list Theodore Tso wrote: > > Ah, but as soon as the repacker thread runs continuously, then you > lose all or most of the claimed advantage of "wandering logs". > Specifically, the claim of the "wandering log" is that you don't have > to write your data twice --- once to the log, and once to the final > location on disk (whereas with ext3 you end up having to do double > writes). But if the repacker is running continuously, you end up > doing double writes anyway, as the repacker moves things from a > location that is convenient for the log, to a location which is > efficient for reading. Worse yet, if the repacker is moving disk > blocks or objects which are no longer in cache, it may end up having > to read objects in before writing them to a final location on disk. > So instead of a write-write overhead, you end up with a > write-read-write overhead. > There's no reason to repack *all* of the data. Many workloads write and delete whole files, so file data should be contiguous. The repacker would only need to move metadata and small files. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 7:24 ` Avi Kivity @ 2006-08-01 9:25 ` Matthias Andree 2006-08-01 9:38 ` Avi Kivity 0 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-08-01 9:25 UTC (permalink / raw) To: Avi Kivity Cc: Theodore Tso, David Lang, David Masover, tdwebste2, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list On Tue, 01 Aug 2006, Avi Kivity wrote: > There's no reason to repack *all* of the data. Many workloads write and > delete whole files, so file data should be contiguous. The repacker > would only need to move metadata and small files. Move small files? What for? Even if it is "only" moving metadata, it is not different from what ext3 or xfs are doing today (rewriting metadata from the intent log or block journal to the final location). The UFS+softupdates from the BSD world looks pretty good at avoiding unnecessary writes (at the expense of a long-running but nice background fsck after a crash, which is however easy on the I/O as of recent FreeBSD versions). Which was their main point against logging/journaling BTW, but they are porting XFS as well to save those that need instant complete recovery. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 9:25 ` Matthias Andree @ 2006-08-01 9:38 ` Avi Kivity 0 siblings, 0 replies; 639+ messages in thread From: Avi Kivity @ 2006-08-01 9:38 UTC (permalink / raw) To: Matthias Andree Cc: Theodore Tso, David Lang, David Masover, tdwebste2, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list Matthias Andree wrote: > > On Tue, 01 Aug 2006, Avi Kivity wrote: > > > There's no reason to repack *all* of the data. Many workloads write > and > > delete whole files, so file data should be contiguous. The repacker > > would only need to move metadata and small files. > > Move small files? What for? > WAFL-style filesystems like contiguous space, so if small files are scattered in otherwise free space, the repacker should free them. > Even if it is "only" moving metadata, it is not different from what ext3 > or xfs are doing today (rewriting metadata from the intent log or block > journal to the final location). > There is no need to repack all metadata; only that which helps in creating free space. For example: if you untar a source tree you'd get mixed metadata and small file data packed together, but there's no need to repack that data. -- error compiling committee.c: too many arguments to function ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 6:48 ` Theodore Tso 2006-08-01 2:40 ` Hans Reiser 2006-08-01 7:24 ` Avi Kivity @ 2006-08-01 17:03 ` David Masover 2 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-08-01 17:03 UTC (permalink / raw) To: Theodore Tso, David Lang, David Masover, tdwebste2, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, linux-kernel, reiserfs-list Theodore Tso wrote: > Ah, but as soon as the repacker thread runs continuously, then you > lose all or most of the claimed advantage of "wandering logs". [...] > So instead of a write-write overhead, you end up with a > write-read-write overhead. This would tend to suggest that the repacker should not run constantly, but also that while it's running, performance could be almost as good as ext3. > But of course, people tend to disable the repacker when doing > benchmarks because they're trying to play the "my filesystem/database > has bigger performance numbers than yours" game.... So you run your own benchmarks, I'll run mine... Benchmarks for everyone! I'd especially like to see what performance is like with the repacker not running, and during the repack. If performance during a repack is comparable to ext3, I think we win, although we have to amend that statement to "My filesystem/database has the same or bigger perfomance numbers than yours." ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 3:00 ` Theodore Tso 2006-08-01 3:38 ` David Masover 2006-08-01 3:47 ` Timothy Webster @ 2006-08-01 4:38 ` Mike Benoit 2 siblings, 0 replies; 639+ messages in thread From: Mike Benoit @ 2006-08-01 4:38 UTC (permalink / raw) To: Theodore Tso Cc: David Masover, Nate Diller, David Lang, Adrian Ulrich, Horst H. von Brand, reiser, lkml, jeff, linux-kernel, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 3177 bytes --] On Mon, 2006-07-31 at 23:00 -0400, Theodore Tso wrote: > The problem is that many benchmarks (such as taring and untaring the > kernel sources in reiser4 sort order) are overly simplistic, in that > they don't really reflect how people use the filesystem in real life. > (How many times can you guarantee that files will be written in the > precise hash/tree order so that the filesystem gets the best possible > time?) A more subtle version of this problem happens for filesystems > where their performance degrades dramatically over-time without a > repacker. If the benchmark doesn't take into account the need for > repacker, or if the repacker is disabled or fails to run during the > benchmark, the filesystem are in effect "cheating" on the benchmark > because there is critical work which is necessary for the long-term > health of the filesystem which is getting deferred until after the > benchmark has finished measuring the performance of the system under > test. If the file system that requires a repacker can do X operations in 1/2 the time all week long, even if the repacker takes several hours once a week to run, you're still ahead of the game. The load averages on the vast majority of servers have significant peaks and valleys throughout the day, and throughout the week, it wouldn't be hard to find a time where a online repacker is virtually unnoticeable to users. Delaying certain work until the server is less busy might be considered "cheating" on benchmarks, but in the real world most people would consider it good use of resources. Just like RAID rebuilds, where you can set maximum IO speeds I could see a repacker working in a similar fashion. Obviously there are some servers where this is unacceptable and in such cases, don't use Reiser4, but I would guess they are few and far between. No file system is perfect for 100% of the work loads thrown at it. PostgreSQL and its vacuum process comes to mind as something similar to a repacker. PostgreSQL puts off doing some work to later, and it has proven itself over and over again, especially when it comes to scalability. PostgreSQL has recently (v8.0 I believe) moved to a system where it can automatically detect the need to vacuum specific tables, so tables that need it are vacuumed more often, and tables that don't are rarely touched. I don't see any reason why a repacker couldn't work in a similar fashion, once it detects a file to be fragmented over some value, it gets scheduled for repacking when there is idle disk IO available. The bottom line is once you have a online repacker you instantly open up all sorts of doors. Its well known that disk drives have different sustained read/write performance depending on if the data is on the inside or outside tracks. Perhaps the repacker could also move files around on the disk to get further gains, for instance larger or most commonly used files could be moved to where the disk has the highest sustained read/write performance, and smaller less used files could be moved to the slowest sustained read/write performance areas. -- Mike Benoit <ipso@snappymail.ca> [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 1:31 ` David Masover 2006-08-01 3:00 ` Theodore Tso @ 2006-08-01 4:21 ` David Lang 2006-08-01 4:32 ` David Masover 1 sibling, 1 reply; 639+ messages in thread From: David Lang @ 2006-08-01 4:21 UTC (permalink / raw) To: David Masover Cc: Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On Mon, 31 Jul 2006, David Masover wrote: > Oh, I'm curious -- do hard drives ever carry enough battery/capacitance to > cover their caches? It doesn't seem like it would be that hard/expensive, > and if it is done that way, then I think it's valid to leave them on. You > could just say that other filesystems aren't taking as much advantage of > newer drive features as Reiser :P there are no drives that have the ability to flush their cache after they loose power. now, that being said, /. had a story within the last couple of days about hard drive manufacturers adding flash to their hard drives. they may be aiming to add some non-volitile cache capability to their drives, although I didn't think that flash writes were that fast (needed if you dump the cache to flash when you loose power), or that easy on power (given that you would first loose power), and flash has limited write cycles (needed if you always use the cache). I've heard to many fancy-sounding drive technologies that never hit the market, I'll wait until thye are actually available before I start counting on them for anything (let alone design/run a filesystem that requires them :-) external battery backed cache is readily available, either on high-end raid controllers or as seperate ram drives (and in raid array boxes), but nothing on individual drives. David Lang ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 4:21 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed " David Lang @ 2006-08-01 4:32 ` David Masover 2006-08-01 4:53 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressedby " David Lang 2006-08-01 23:50 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by " Ian Stirling 0 siblings, 2 replies; 639+ messages in thread From: David Masover @ 2006-08-01 4:32 UTC (permalink / raw) To: David Lang Cc: Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list David Lang wrote: > On Mon, 31 Jul 2006, David Masover wrote: > >> Oh, I'm curious -- do hard drives ever carry enough >> battery/capacitance to cover their caches? It doesn't seem like it >> would be that hard/expensive, and if it is done that way, then I think >> it's valid to leave them on. You could just say that other >> filesystems aren't taking as much advantage of newer drive features as >> Reiser :P > > there are no drives that have the ability to flush their cache after > they loose power. Aha, so back to the usual argument: UPS! It takes a fraction of a second to flush that cache. > now, that being said, /. had a story within the last couple of days > about hard drive manufacturers adding flash to their hard drives. they > may be aiming to add some non-volitile cache capability to their drives, > although I didn't think that flash writes were that fast (needed if you > dump the cache to flash when you loose power), or that easy on power > (given that you would first loose power), and flash has limited write > cycles (needed if you always use the cache). But, the point of flash was not to replace the RAM cache, but to be another level. That is, you have your Flash which may be as fast as the disk, maybe faster, maybe less, and you have maybe a gig worth of it. Even the bloatiest of OSes aren't really all that big -- my OS X came installed, with all kinds of apps I'll never use, in less than 10 gigs. And I think this story was awhile ago (a dupe? Not surprising), and the point of the Flash is that as long as your read/write cache doesn't run out, and you're still in that 1 gig of Flash, you're a bit safer than the RAM cache, and you can also leave the disk off, as in, spinned down. Parked. Very useful for a laptop -- I used to do this in Linux by using Reiser4, setting the disk to spin down, and letting lazy writes do their thing, but I didn't have enough RAM, and there's always the possibility of losing data. But leaving the disk off is nice, because in the event of sudden motion, it's safer that way. Besides, most hardware gets designed for That Other OS, which doesn't support any kind of Laptop Mode, so it's nice to be able to enforce this at a hardware level, in a safe way. > I've heard to many fancy-sounding drive technologies that never hit the > market, I'll wait until thye are actually available before I start > counting on them for anything (let alone design/run a filesystem that > requires them :-) Or even remember their names. > external battery backed cache is readily available, either on high-end > raid controllers or as seperate ram drives (and in raid array boxes), > but nothing on individual drives. Ah. Curses. UPS, then. If you have enough time, you could even do a Software Suspend first -- that way, when power comes back on, you boot back up, and if it's done quickly enough, connections won't even be dropped... ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressedby kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 4:32 ` David Masover @ 2006-08-01 4:53 ` David Lang 2006-08-01 5:59 ` David Masover 2006-08-01 23:50 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by " Ian Stirling 1 sibling, 1 reply; 639+ messages in thread From: David Lang @ 2006-08-01 4:53 UTC (permalink / raw) To: David Masover Cc: Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On Mon, 31 Jul 2006, David Masover wrote: > David Lang wrote: >> On Mon, 31 Jul 2006, David Masover wrote: >> >>> Oh, I'm curious -- do hard drives ever carry enough battery/capacitance to >>> cover their caches? It doesn't seem like it would be that hard/expensive, >>> and if it is done that way, then I think it's valid to leave them on. You >>> could just say that other filesystems aren't taking as much advantage of >>> newer drive features as Reiser :P >> >> there are no drives that have the ability to flush their cache after they >> loose power. > > Aha, so back to the usual argument: UPS! It takes a fraction of a second to > flush that cache. which does absolutly no good if someone trips over the power cord, the fuse blows in the power supply, someone yanks the drive out of the hot-swap bay, etc. >> now, that being said, /. had a story within the last couple of days about >> hard drive manufacturers adding flash to their hard drives. they may be >> aiming to add some non-volitile cache capability to their drives, although >> I didn't think that flash writes were that fast (needed if you dump the >> cache to flash when you loose power), or that easy on power (given that you >> would first loose power), and flash has limited write cycles (needed if you >> always use the cache). > > But, the point of flash was not to replace the RAM cache, but to be another > level. That is, you have your Flash which may be as fast as the disk, maybe > faster, maybe less, and you have maybe a gig worth of it. Even the bloatiest > of OSes aren't really all that big -- my OS X came installed, with all kinds > of apps I'll never use, in less than 10 gigs. > > And I think this story was awhile ago (a dupe? Not surprising), and the > point of the Flash is that as long as your read/write cache doesn't run out, > and you're still in that 1 gig of Flash, you're a bit safer than the RAM > cache, and you can also leave the disk off, as in, spinned down. Parked. as I understand it flash reads are fast (ram speeds), but writes are pretty slow (comparable or worse to spinning media) writing to a ram cache, but having a flash drive behind it doesn't gain you any protection. and I don't think you need it for reads >> external battery backed cache is readily available, either on high-end raid >> controllers or as seperate ram drives (and in raid array boxes), but >> nothing on individual drives. > > Ah. Curses. > > UPS, then. If you have enough time, you could even do a Software Suspend > first -- that way, when power comes back on, you boot back up, and if it's > done quickly enough, connections won't even be dropped... remember, it can take 90W of power to run your CPU, 100+ to run your video card, plus everything else. even a few seconds of power for this is a very significant amount of energy storage. however, I did get a pointer recently at a company makeing super-high capcity caps, up to 2600F (F, not uF!) in a 138mmx tall 57mm dia cyliner, however it only handles 2.7v (they have modules that handle higher voltages available) http://www.maxwell.com/ultracapacitors/index.html however I don't see these as being standard equipment in systems or on drives anytime soon David Lang ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressedby kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 4:53 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressedby " David Lang @ 2006-08-01 5:59 ` David Masover 0 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-08-01 5:59 UTC (permalink / raw) To: David Lang Cc: Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list David Lang wrote: > On Mon, 31 Jul 2006, David Masover wrote: >> Aha, so back to the usual argument: UPS! It takes a fraction of a >> second to flush that cache. > > which does absolutly no good if someone trips over the power cord, the > fuse blows in the power supply, someone yanks the drive out of the > hot-swap bay, etc. Power supply fuse... Yeah, it happens. Drives die, too. This seems fairly uncommon. And dear God, please tell me anyone smart enough to set up a UPS would also be smart enough to make tripping over the power cord rare or impossible. My box has a cable that runs down behind a desk, between the desk and the wall. Power strip is on the floor, where a UPS will be when I get around to buying one. If someone kicks any cable, it would be where the UPS hits the wall -- but that's also behind the same desk. > as I understand it flash reads are fast (ram speeds), but writes are > pretty slow (comparable or worse to spinning media) > > writing to a ram cache, but having a flash drive behind it doesn't gain > you any protection. and I don't think you need it for reads Does gain you protection if you're not using the RAM cache, if you're that paranoid. I don't know if it's cheaper than RAM, but more read cache is always better. And losing power seems a lot less likely than crashing, especially on a Windows laptop, so it does make sense as a product. And a laptop, having a battery, will give you a good bit of warning before it dies. My Powerbook syncs and goes into Sleep mode when it runs low on power (~1%/5mins) >>> external battery backed cache is readily available, either on >>> high-end raid controllers or as seperate ram drives (and in raid >>> array boxes), but nothing on individual drives. >> >> Ah. Curses. >> >> UPS, then. If you have enough time, you could even do a Software >> Suspend first -- that way, when power comes back on, you boot back up, >> and if it's done quickly enough, connections won't even be dropped... > > remember, it can take 90W of power to run your CPU, 100+ to run your > video card, plus everything else. even a few seconds of power for this > is a very significant amount of energy storage. Suspend2 can take about 10-20 seconds. It should be possible to work out the maximum amount of time it can take. Anyway, according to a quick Google search, my CPU is more like 70W. Video card isn't required on a server, but you may be right on mine. I haven't looked at UPSes lately, though. I need about 3 seconds for a sync, maybe 10 for a suspend, so to be safe I can say for sure I'd be down in about 30 seconds. So, another Google search, and while you can get a cheap UPS for anywhere from $10 to $100, the sweet spot seems to be a little over $200. $229, and it's 865W, supposedly for 3.7 minutes. Here's a review: "This is a great product. It powers an AMD 64 3200+ with beefy (6800GT) graphics card, 21" CRT monitor, secondary 19" CRT, a linux server, a 15" CRT, Cisco 2800XL switch, Linksys WRTG54GS, cable modem, speakers, and many other things. The software says I will get 9 minutes runtime with all of that hooked up, realistically it's about 4 minutes." This was the lowest time reported. Most of the other reviews say at least 15 minutes, sometimes 30 minutes, with fairly high-end computers listed (and monitors, sometimes two computers/monitors), but nowhere near as much stuff as this guy has. I checked most of these for Linux support, and UPSes in general seem well supported. So yes, the box will shut off automatically. On a network, it shouldn't be too hard to get one box to shut off all the rest. It's a lot of money, even at the low end, but when you're already spending a pile of money on a new computer, keep power in mind. And really, even 11 minutes would be fine, but 40 minutes of power is quite a lot compared to less than a minute of time taken to shut down normally -- not even suspend, but a normal shut down. I'd be tempted to try to ride it out for the first 20 minutes, see if power comes back up... > however, I did get a pointer recently at a company makeing super-high > capcity caps, up to 2600F (F, not uF!) in a 138mmx tall 57mm dia > cyliner, however it only handles 2.7v (they have modules that handle > higher voltages available) > http://www.maxwell.com/ultracapacitors/index.html > > however I don't see these as being standard equipment in systems or on > drives anytime soon This seems to be a whole different approach -- more along the lines of in the drive, which would be cool... ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 4:32 ` David Masover 2006-08-01 4:53 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressedby " David Lang @ 2006-08-01 23:50 ` Ian Stirling 2006-08-02 2:29 ` Kyle Moffett 2006-08-02 3:52 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] David Masover 1 sibling, 2 replies; 639+ messages in thread From: Ian Stirling @ 2006-08-01 23:50 UTC (permalink / raw) To: David Masover Cc: David Lang, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list David Masover wrote: > David Lang wrote: > >> On Mon, 31 Jul 2006, David Masover wrote: >> >>> Oh, I'm curious -- do hard drives ever carry enough >>> battery/capacitance to cover their caches? It doesn't seem like it >>> would be that hard/expensive, and if it is done that way, then I >>> think it's valid to leave them on. You could just say that other >>> filesystems aren't taking as much advantage of newer drive features >>> as Reiser :P >> >> >> there are no drives that have the ability to flush their cache after >> they loose power. > > > Aha, so back to the usual argument: UPS! It takes a fraction of a > second to flush that cache. You probably don't actually want to flush the cache - but to write to a journal. 16M of cache - split into 32000 writes to single sectors spread over the disk could well take several minutes to write. Slapping it onto a journal would take well under .2 seconds. That's a non-trivial amount of storage though - 3J or so, 40mF@12V - a moderately large/expensive capacitor. And if you've got to spin the drive up, you've just added another order of magnitude. You can see why a flash backup of the write cache may be nicer. You can do it if the disk isn't spinning. It uses moderately less energy - and at a much lower rate, which means the power supply can be _much_ cheaper. I'd guess it's the difference between under $2 and $10. And if you can use it as a lazy write cache for laptops - things just got better battery life wise too. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 23:50 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by " Ian Stirling @ 2006-08-02 2:29 ` Kyle Moffett 2006-08-02 14:28 ` Solaris ZFS on Linux Krzysztof Halasa 2006-08-02 3:52 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] David Masover 1 sibling, 1 reply; 639+ messages in thread From: Kyle Moffett @ 2006-08-02 2:29 UTC (permalink / raw) To: Ian Stirling Cc: David Masover, David Lang, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, lkml, Jeff Garzik, Theodore Ts'o, LKML Kernel, reiserfs-list On Aug 01, 2006, at 19:50:49, Ian Stirling wrote: > You probably don't actually want to flush the cache - but to write > to a journal. 16M of cache - split into 32000 writes to single > sectors spread over the disk could well take several minutes to > write. Slapping it onto a journal would take well under .2 seconds. > That's a non-trivial amount of storage though - 3J or so, 40mF@12V > - a moderately large/expensive capacitor. IMHO the best alternative for a situation like that is a storage controller with a battery-backed cache and a hunk of flash NVRAM for when the power shuts off (just in case you run out of battery), as well as a separate 1GB battery-backed PCI ramdisk for an external journal device (likewise equipped with flash NVRAM). It doesn't take much power at all to write a gig of stuff to a small flash chip (Think about your digital camera which runs off a couple AA's), so with a fair-sized on-board battery pack you could easily transfer its data to NVRAM and still have power left to back up data in RAM for 12 hours or so. That way bootup is fast (no reading 1GB of data from NVRAM) but there's no risk of data loss. Cheers, Kyle Moffett ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux 2006-08-02 2:29 ` Kyle Moffett @ 2006-08-02 14:28 ` Krzysztof Halasa 2006-08-02 18:12 ` Ian Stirling 2006-08-03 2:20 ` Wil Reichert 0 siblings, 2 replies; 639+ messages in thread From: Krzysztof Halasa @ 2006-08-02 14:28 UTC (permalink / raw) To: Kyle Moffett Cc: Ian Stirling, David Masover, David Lang, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, lkml, Jeff Garzik, Theodore Ts'o, LKML Kernel, reiserfs-list Kyle Moffett <mrmacman_g4@mac.com> writes: > IMHO the best alternative for a situation like that is a storage > controller with a battery-backed cache and a hunk of flash NVRAM for > when the power shuts off (just in case you run out of battery), as > well as a separate 1GB battery-backed PCI ramdisk for an external > journal device (likewise equipped with flash NVRAM). It doesn't take > much power at all to write a gig of stuff to a small flash chip > (Think about your digital camera which runs off a couple AA's), so > with a fair-sized on-board battery pack you could easily transfer its > data to NVRAM and still have power left to back up data in RAM for 12 > hours or so. That way bootup is fast (no reading 1GB of data from > NVRAM) but there's no risk of data loss. Not sure - reading flash is fast, but writing is quite slow. A digital camera can consume a set of 2 or 4 2500 mAh AA cells for a fraction of 1 GB (of course, only a part of power goes to flash). -- Krzysztof Halasa ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux 2006-08-02 14:28 ` Solaris ZFS on Linux Krzysztof Halasa @ 2006-08-02 18:12 ` Ian Stirling 2006-08-03 2:20 ` Wil Reichert 1 sibling, 0 replies; 639+ messages in thread From: Ian Stirling @ 2006-08-02 18:12 UTC (permalink / raw) To: Krzysztof Halasa Cc: Kyle Moffett, David Masover, David Lang, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, lkml, Jeff Garzik, Theodore Ts'o, LKML Kernel, reiserfs-list Krzysztof Halasa wrote: > Kyle Moffett <mrmacman_g4@mac.com> writes: > > >>IMHO the best alternative for a situation like that is a storage >>controller with a battery-backed cache and a hunk of flash NVRAM for >>when the power shuts off (just in case you run out of battery), as >>well as a separate 1GB battery-backed PCI ramdisk for an external >>journal device (likewise equipped with flash NVRAM). It doesn't take > Not sure - reading flash is fast, but writing is quite slow. > A digital camera can consume a set of 2 or 4 2500 mAh AA cells > for a fraction of 1 GB (of course, only a part of power goes > to flash). Yeah - that's why I said in the original message that it's not especially lower in energy - the energy is used at a lower rate, so is much cheaper to supply. http://www.samsung.com/products/semiconductor/NORFlash/256Mbit/K8A5615EBA/K8A5615EBA.htm 's datasheet says to program the 32Mbyte chip takes about 30mw*120s, or 3.5J or so. For a gigabyte, that's 100J - a fairly substantial amount of energy. However - it's at a low rate, so it's not _too_ expensive to supply. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux 2006-08-02 14:28 ` Solaris ZFS on Linux Krzysztof Halasa 2006-08-02 18:12 ` Ian Stirling @ 2006-08-03 2:20 ` Wil Reichert 2006-08-03 9:32 ` Helge Hafting 1 sibling, 1 reply; 639+ messages in thread From: Wil Reichert @ 2006-08-03 2:20 UTC (permalink / raw) To: Krzysztof Halasa Cc: Kyle Moffett, Ian Stirling, David Masover, David Lang, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, lkml, Jeff Garzik, Theodore Ts'o, LKML Kernel, reiserfs-list On 8/2/06, Krzysztof Halasa <khc@pm.waw.pl> wrote: > Kyle Moffett <mrmacman_g4@mac.com> writes: > > > IMHO the best alternative for a situation like that is a storage > > controller with a battery-backed cache and a hunk of flash NVRAM for > > when the power shuts off (just in case you run out of battery), as > > well as a separate 1GB battery-backed PCI ramdisk for an external > > journal device (likewise equipped with flash NVRAM). It doesn't take > > much power at all to write a gig of stuff to a small flash chip > > (Think about your digital camera which runs off a couple AA's), so > > with a fair-sized on-board battery pack you could easily transfer its > > data to NVRAM and still have power left to back up data in RAM for 12 > > hours or so. That way bootup is fast (no reading 1GB of data from > > NVRAM) but there's no risk of data loss. > > Not sure - reading flash is fast, but writing is quite slow. > A digital camera can consume a set of 2 or 4 2500 mAh AA cells > for a fraction of 1 GB (of course, only a part of power goes > to flash). Seeks are fast, throughput is terrible, power is minimal: http://techreport.com/reviews/2006q3/supertalent-flashide/index.x?pg=1 Wil ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux 2006-08-03 2:20 ` Wil Reichert @ 2006-08-03 9:32 ` Helge Hafting 0 siblings, 0 replies; 639+ messages in thread From: Helge Hafting @ 2006-08-03 9:32 UTC (permalink / raw) To: Wil Reichert Cc: Krzysztof Halasa, Kyle Moffett, Ian Stirling, David Masover, David Lang, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, lkml, Jeff Garzik, Theodore Ts'o, LKML Kernel, reiserfs-list On Wed, Aug 02, 2006 at 07:20:25PM -0700, Wil Reichert wrote: > On 8/2/06, Krzysztof Halasa <khc@pm.waw.pl> wrote: > >Kyle Moffett <mrmacman_g4@mac.com> writes: > > > >> IMHO the best alternative for a situation like that is a storage > >> controller with a battery-backed cache and a hunk of flash NVRAM for > >> when the power shuts off (just in case you run out of battery), as > >> well as a separate 1GB battery-backed PCI ramdisk for an external > >> journal device (likewise equipped with flash NVRAM). It doesn't take > >> much power at all to write a gig of stuff to a small flash chip > >> (Think about your digital camera which runs off a couple AA's), so > >> with a fair-sized on-board battery pack you could easily transfer its > >> data to NVRAM and still have power left to back up data in RAM for 12 > >> hours or so. That way bootup is fast (no reading 1GB of data from > >> NVRAM) but there's no risk of data loss. > > > >Not sure - reading flash is fast, but writing is quite slow. > >A digital camera can consume a set of 2 or 4 2500 mAh AA cells > >for a fraction of 1 GB (of course, only a part of power goes > >to flash). > > Seeks are fast, throughput is terrible, power is minimal: > > http://techreport.com/reviews/2006q3/supertalent-flashide/index.x?pg=1 > That particular flash drive had terrible througput. But there are other alternatives. I use a kingston 4GB compactflash card as a disk, and it reads 22MB/s, according to specs and tests with hdparm. And it writes 16MB/s. Much better than the sorry thing in that test, about the same read speed as their worst platter-based harddisk. And of course it still have the nice seek times of non-rotating media. :-) Helge Hafting ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 23:50 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by " Ian Stirling 2006-08-02 2:29 ` Kyle Moffett @ 2006-08-02 3:52 ` David Masover 1 sibling, 0 replies; 639+ messages in thread From: David Masover @ 2006-08-02 3:52 UTC (permalink / raw) To: Ian Stirling Cc: David Lang, Nate Diller, Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Ian Stirling wrote: > David Masover wrote: >> David Lang wrote: >> >>> On Mon, 31 Jul 2006, David Masover wrote: >>> >>>> Oh, I'm curious -- do hard drives ever carry enough >>>> battery/capacitance to cover their caches? It doesn't seem like it >>>> would be that hard/expensive, and if it is done that way, then I >>>> think it's valid to leave them on. You could just say that other >>>> filesystems aren't taking as much advantage of newer drive features >>>> as Reiser :P >>> >>> >>> there are no drives that have the ability to flush their cache after >>> they loose power. >> >> >> Aha, so back to the usual argument: UPS! It takes a fraction of a >> second to flush that cache. > > You probably don't actually want to flush the cache - but to write > to a journal. > 16M of cache - split into 32000 writes to single sectors spread over > the disk could well take several minutes to write. Slapping it onto > a journal would take well under .2 seconds. > That's a non-trivial amount of storage though - 3J or so, 40mF@12V - > a moderately large/expensive capacitor. Before we get ahead of ourselves, remember: ~$200 buys you a huge amount of battery storage. We're talking several minutes for several boxes, at the very least -- more like 10 minutes. But yes, a journal or a software suspend. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 1:02 ` Matthias Andree @ 2006-08-01 7:51 ` Adrian Ulrich 2006-08-01 1:31 ` David Masover 2006-08-01 7:51 ` Adrian Ulrich 2 siblings, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-08-01 7:51 UTC (permalink / raw) To: Matthias Andree Cc: nate.diller, dlang, matthias.andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list > suspect, particularly with 7200/min (s)ATA crap. Quoting myself (again): >> A quick'n'dirty ZFS-vs-UFS-vs-Reiser3-vs-Reiser4-vs-Ext3 'benchmark' Yeah, the test ran on a single SATA-Harddisk (quick'n'dirty). I'm so sorry but i don't have access to a $$$ Raid-System at home. Anyway: The test shows us that Reiser4 performed very good on my (common 0-8-15) hardware. > sdparm --clear=WCE /dev/sda # please. How about using /dev/emcpower* for the next benchmark? I mighty be able to re-run it in a few weeks if people are interested and if i receive constructive suggestions (= Postmark parameters, mkfs options, etc..) Regards, Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] @ 2006-08-01 7:51 ` Adrian Ulrich 0 siblings, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-08-01 7:51 UTC (permalink / raw) Cc: nate.diller, dlang, matthias.andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list > suspect, particularly with 7200/min (s)ATA crap. Quoting myself (again): >> A quick'n'dirty ZFS-vs-UFS-vs-Reiser3-vs-Reiser4-vs-Ext3 'benchmark' Yeah, the test ran on a single SATA-Harddisk (quick'n'dirty). I'm so sorry but i don't have access to a $$$ Raid-System at home. Anyway: The test shows us that Reiser4 performed very good on my (common 0-8-15) hardware. > sdparm --clear=WCE /dev/sda # please. How about using /dev/emcpower* for the next benchmark? I mighty be able to re-run it in a few weeks if people are interested and if i receive constructive suggestions (= Postmark parameters, mkfs options, etc..) Regards, Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 7:51 ` Adrian Ulrich (?) @ 2006-08-01 9:09 ` Matthias Andree 2006-08-01 9:57 ` Avi Kivity 2006-08-01 10:57 ` Jan Engelhardt -1 siblings, 2 replies; 639+ messages in thread From: Matthias Andree @ 2006-08-01 9:09 UTC (permalink / raw) To: Adrian Ulrich Cc: Matthias Andree, nate.diller, dlang, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Adrian Ulrich schrieb am 2006-08-01: > > suspect, particularly with 7200/min (s)ATA crap. > > Quoting myself (again): > >> A quick'n'dirty ZFS-vs-UFS-vs-Reiser3-vs-Reiser4-vs-Ext3 'benchmark' > > Yeah, the test ran on a single SATA-Harddisk (quick'n'dirty). > I'm so sorry but i don't have access to a $$$ Raid-System at home. I'm not asking for you to perform testing on a $$$$ RAID system with SCSI or SAS, but I consider the obtained data (I am focussing on transactions per unit of time) highly suspicious, and suspect write caches might have contributed their share - I haven't seen a drive that shipped with write cache disabled in the past years. > > sdparm --clear=WCE /dev/sda # please. > > How about using /dev/emcpower* for the next benchmark? No, it is valid to run the test on commodity hardware, but if you (or the benchmark rather) is claiming "transactions", I tend to think "ACID", and I highly doubt any 200 GB SATA drive manages 3000 synchronous writes per second without causing either serious fragmentation or background block moving. This is a figure I'd expect for synchronous random access to RAM disks that have no seek and rotational latencies (and research for hybrid disks w/ flash or other nonvolatile fast random access media to cache actual rotating magnetic plattern access is going on elsewhere). I didn't mean to say your particular drive were crap, but 200GB SATA drives are low end, like it or not -- still, I have one in my home computer because these Samsung SP2004C are so nicely quiet. > I mighty be able to re-run it in a few weeks if people are interested > and if i receive constructive suggestions (= Postmark parameters, > mkfs options, etc..) I don't know Postmark, I did suggest to turn the write cache off. If your systems uses hdparm -W0 /dev/sda instead, go ahead. But you're right to collect and evaluate suggestions first if you don't want to run a new benchmark every day :) -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 9:09 ` Matthias Andree @ 2006-08-01 9:57 ` Avi Kivity 2006-08-01 10:57 ` Jan Engelhardt 1 sibling, 0 replies; 639+ messages in thread From: Avi Kivity @ 2006-08-01 9:57 UTC (permalink / raw) To: Matthias Andree Cc: Adrian Ulrich, nate.diller, dlang, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Matthias Andree wrote: > > No, it is valid to run the test on commodity hardware, but if you (or > the benchmark rather) is claiming "transactions", I tend to think > "ACID", and I highly doubt any 200 GB SATA drive manages 3000 > synchronous writes per second without causing either serious > fragmentation or background block moving. > You are assuming 1 transaction = 1 sync write. That's not true. Databases and log filesystems can get much more out of a disk write. -- error compiling committee.c: too many arguments to function ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 9:09 ` Matthias Andree 2006-08-01 9:57 ` Avi Kivity @ 2006-08-01 10:57 ` Jan Engelhardt 2006-08-01 13:15 ` Matthias Andree 1 sibling, 1 reply; 639+ messages in thread From: Jan Engelhardt @ 2006-08-01 10:57 UTC (permalink / raw) To: Matthias Andree Cc: Adrian Ulrich, nate.diller, dlang, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list > >I didn't mean to say your particular drive were crap, but 200GB SATA >drives are low end, like it or not -- And you think an 18 GB SCSI disk just does it better because it's SCSI? Esp. in long sequential reads. Jan Engelhardt -- ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 10:57 ` Jan Engelhardt @ 2006-08-01 13:15 ` Matthias Andree 2006-08-01 13:40 ` Jan Engelhardt 0 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-08-01 13:15 UTC (permalink / raw) To: Jan Engelhardt Cc: Matthias Andree, Adrian Ulrich, nate.diller, dlang, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Jan Engelhardt schrieb am 2006-08-01: > >I didn't mean to say your particular drive were crap, but 200GB SATA > >drives are low end, like it or not -- > > And you think an 18 GB SCSI disk just does it better because it's SCSI? 18 GB SCSI disks are 1999 gear, so who cares? Seagate didn't sell 200 GB SATA drives at that time. > Esp. in long sequential reads. You think SCSI drives aren't on par? Right, they're ahead. 98 MB/s for the fastest SCSI drives vs. 88 MB/s for Raptor 150 GB SATA and 74 MB/s for the fastest other ATA drives. (Figures obtained from StorageReview.com's Performance Database.) -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-08-01 13:15 ` Matthias Andree @ 2006-08-01 13:40 ` Jan Engelhardt 0 siblings, 0 replies; 639+ messages in thread From: Jan Engelhardt @ 2006-08-01 13:40 UTC (permalink / raw) To: Matthias Andree Cc: Adrian Ulrich, nate.diller, dlang, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list >> >I didn't mean to say your particular drive were crap, but 200GB SATA >> >drives are low end, like it or not -- >> >> And you think an 18 GB SCSI disk just does it better because it's SCSI? > >18 GB SCSI disks are 1999 gear, so who cares? >Seagate didn't sell 200 GB SATA drives at that time. > >> Esp. in long sequential reads. > >You think SCSI drives aren't on par? Right, they're ahead. >98 MB/s for the fastest SCSI drives vs. 88 MB/s for Raptor 150 GB SATA >and 74 MB/s for the fastest other ATA drives. Uhuh. And how do they measure that? Did they actually ran sth like... dd_rescue /dev/hda /dev/null Jan Engelhardt -- ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 20:57 ` Adrian Ulrich (?) (?) @ 2006-07-31 22:08 ` Horst H. von Brand 2006-07-31 23:25 ` Nate Diller 2006-08-01 7:47 ` Adrian Ulrich -1 siblings, 2 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-07-31 22:08 UTC (permalink / raw) To: Adrian Ulrich Cc: Horst H. von Brand, matthias.andree, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Adrian Ulrich <reiser4@blinkenlights.ch> wrote: > > > Great to see that Sun ships a state-of-the-art Filesystem with > > > Solaris... I think linux should do the same... > > > > This would be worthwhile, if only to be able to futz around in Solaris-made > > filesystems. > s/I think linux should do the same/I think linux should include Reiser4/ > ;-) So ZFS isn't "state-of-the-art"? [...] > But i'd rather like to see a Linux version of WAFL :-) WAFL is for high-turnover filesystems on RAID-5 (and assumes flash memory staging areas). Not your run-of-the-mill desktop... > ZFS didn't really impress me: > The Volume-Manager is nice but the Filesystem.. well: It beats UFS > .. sometimes ;-) OK, ext3 + LVM it is then. > See also: http://spam.workaround.ch/dull/postmark.txt Interesting. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 22:08 ` Horst H. von Brand @ 2006-07-31 23:25 ` Nate Diller 2006-08-01 7:47 ` Adrian Ulrich 1 sibling, 0 replies; 639+ messages in thread From: Nate Diller @ 2006-07-31 23:25 UTC (permalink / raw) To: Horst H. von Brand Cc: Adrian Ulrich, matthias.andree, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On 7/31/06, Horst H. von Brand <vonbrand@inf.utfsm.cl> wrote: > Adrian Ulrich <reiser4@blinkenlights.ch> wrote: > > > > Great to see that Sun ships a state-of-the-art Filesystem with > > > > Solaris... I think linux should do the same... > > > > > > This would be worthwhile, if only to be able to futz around in Solaris-made > > > filesystems. > > > s/I think linux should do the same/I think linux should include Reiser4/ > > ;-) > > So ZFS isn't "state-of-the-art"? maybe reiser4 and ZFS are. certainly, they optimize for different behavior though > [...] > > > But i'd rather like to see a Linux version of WAFL :-) > > WAFL is for high-turnover filesystems on RAID-5 (and assumes flash memory > staging areas). Not your run-of-the-mill desktop... yeah, good thing nobody tries to use linux for high-turnover servers with RAID. pish-posh. > > ZFS didn't really impress me: > > The Volume-Manager is nice but the Filesystem.. well: It beats UFS > > .. sometimes ;-) > > OK, ext3 + LVM it is then. > > > See also: http://spam.workaround.ch/dull/postmark.txt > > Interesting. yeah, i hadn't seen postmark numbers on linux before. maybe mongo isn't so biased after all NATE ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 22:08 ` Horst H. von Brand @ 2006-08-01 7:47 ` Adrian Ulrich 2006-08-01 7:47 ` Adrian Ulrich 1 sibling, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-08-01 7:47 UTC (permalink / raw) To: Horst H. von Brand Cc: vonbrand, matthias.andree, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list > So ZFS isn't "state-of-the-art"? Of course it's state-of-the-art (on Solaris ;-) ) > WAFL is for high-turnover filesystems on RAID-5 (and assumes flash memory > staging areas). s/RAID-5/RAID-4/ > Not your run-of-the-mill desktop... The WAFL-Thing was just a joke ;-) Regards, Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] @ 2006-08-01 7:47 ` Adrian Ulrich 0 siblings, 0 replies; 639+ messages in thread From: Adrian Ulrich @ 2006-08-01 7:47 UTC (permalink / raw) Cc: vonbrand, matthias.andree, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list > So ZFS isn't "state-of-the-art"? Of course it's state-of-the-art (on Solaris ;-) ) > WAFL is for high-turnover filesystems on RAID-5 (and assumes flash memory > staging areas). s/RAID-5/RAID-4/ > Not your run-of-the-mill desktop... The WAFL-Thing was just a joke ;-) Regards, Adrian ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 15:59 ` Adrian Ulrich ` (3 preceding siblings ...) (?) @ 2006-08-01 10:40 ` Helge Hafting 2006-08-01 10:59 ` venom -1 siblings, 1 reply; 639+ messages in thread From: Helge Hafting @ 2006-08-01 10:40 UTC (permalink / raw) To: Adrian Ulrich Cc: Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list On Mon, Jul 31, 2006 at 05:59:58PM +0200, Adrian Ulrich wrote: > Hello Matthias, > > > This looks rather like an education issue rather than a technical limit. > > We aren't talking about the same issue: I was asking to do it > on-the-fly. Umounting the filesystem, running e2fsck and resize2fs > is something different ;-) > > > Which is untrue at least for Solaris, which allows resizing a life file > > system. FreeBSD and Linux require an unmount. > > Correct: You can add more inodes to a Solaris UFS on-the-fly if you are > lucky enough to have some free space available. > > A colleague of mine happened to create a ~300gb filesystem and started > to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) > to the new LUN. At about 70% the filesystem ran out of inodes; Not a > big deal with VxFS because such a problem is fixable within seconds. > What would have happened if he had used UFS? mkfs -G wouldn't work > because he had no additional Diskspace left... *ouch*.. > This case is solvable by planning. When you know that the new fs must be created with all inodes from the start, simply count how many you need before migration. (And add a decent safety margin.) That's what I do with my home machine ask disks wear out every third year or so. The tools for ext2/3 tells how many inodes are in use, and the new fs can be made accordingly. The approach works for bigger machines too of course. Helge Hafting ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-08-01 10:40 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Helge Hafting @ 2006-08-01 10:59 ` venom 0 siblings, 0 replies; 639+ messages in thread From: venom @ 2006-08-01 10:59 UTC (permalink / raw) To: Helge Hafting Cc: Adrian Ulrich, Matthias Andree, vonbrand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list planning sometimes is not possible, exspecially in certain highly stressed environment. Just think. I had 3 years ago a database that was 2 TB, we were supposing it could grow in three years of 6 TB, but now it is 40 TB because the market situation is changed, and with this the number of the users and their needs. Please you have to suppose that when you have to deal with filesystems use for some kind of services, it is impossible to predict the grown rate, and this is true also about the numeber of used i-nodes. On Tue, 1 Aug 2006, Helge Hafting wrote: > On Mon, Jul 31, 2006 at 05:59:58PM +0200, Adrian Ulrich wrote: >> Hello Matthias, >> >>> This looks rather like an education issue rather than a technical limit. >> >> We aren't talking about the same issue: I was asking to do it >> on-the-fly. Umounting the filesystem, running e2fsck and resize2fs >> is something different ;-) >> >>> Which is untrue at least for Solaris, which allows resizing a life file >>> system. FreeBSD and Linux require an unmount. >> >> Correct: You can add more inodes to a Solaris UFS on-the-fly if you are >> lucky enough to have some free space available. >> >> A colleague of mine happened to create a ~300gb filesystem and started >> to migrate Mailboxes (Maildir-style format = many small files (1-3kb)) >> to the new LUN. At about 70% the filesystem ran out of inodes; Not a >> big deal with VxFS because such a problem is fixable within seconds. >> What would have happened if he had used UFS? mkfs -G wouldn't work >> because he had no additional Diskspace left... *ouch*.. >> > This case is solvable by planning. When you know that the new fs > must be created with all inodes from the start, simply count > how many you need before migration. (And add a decent safety margin.) > That's what I do with my home machine ask disks wear out every third > year or so. The tools for ext2/3 tells how many inodes are in use, > and the new fs can be made accordingly. The approach works for bigger > machines too of course. > > Helge Hafting > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 14:47 ` Matthias Andree 2006-07-31 15:59 ` Adrian Ulrich @ 2006-07-31 16:52 ` David Masover 2006-08-01 6:22 ` Jan Engelhardt 2 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-07-31 16:52 UTC (permalink / raw) To: Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list Matthias Andree wrote: > Adrian Ulrich schrieb am 2006-07-31: >> Why are a lot of Solaris-people using (buying) VxFS? Maybe because UFS >> also has such silly limitations? (..and performs awkward with trillions >> of files..?..) > > Well, such "silly limitations"... looks like they are mostly hot air > spewn by marketroids that need to justify people spending money on their > new filesystem. I think the limitations are silly, and I'm not paid to say this. Besides, we're talking about a filesystem that will be free (and libre), so I don't see the point of marketroids, certainly not in this context. But let's not stoop to name-calling. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-31 14:47 ` Matthias Andree 2006-07-31 15:59 ` Adrian Ulrich 2006-07-31 16:52 ` David Masover @ 2006-08-01 6:22 ` Jan Engelhardt 2 siblings, 0 replies; 639+ messages in thread From: Jan Engelhardt @ 2006-08-01 6:22 UTC (permalink / raw) To: Matthias Andree Cc: Adrian Ulrich, Horst H. von Brand, ipso, reiser, lkml, jeff, tytso, linux-kernel, reiserfs-list > >> A filesystem with a fixed number of inodes (= not readjustable while >> mounted) is ehr.. somewhat unuseable for a lot of people with >> big and *flexible* storage needs (Talking about NetApp/EMC owners) > >Which is untrue at least for Solaris, which allows resizing a life file >system. FreeBSD and >Linux require an unmount. Only for shrinking. Jan Engelhardt -- ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 10:25 ` Matthias Andree 2006-07-24 11:34 ` Christian Iversen 2006-07-24 16:57 ` Mike Benoit @ 2006-07-24 20:28 ` Lex Lyamin 2 siblings, 0 replies; 639+ messages in thread From: Lex Lyamin @ 2006-07-24 20:28 UTC (permalink / raw) To: Hans Reiser, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List Cc: Matthias Andree [-- Attachment #1: Type: text/plain, Size: 501 bytes --] You see, I don't care a iota about "plugins" or other implementation > details. > > The bottom line is reiserfs 3.6 imposes practial limits that ext3fs > doesn't impose and that's reason enough for an administrator not to > install reiserfs 3.6. Sorry. > > I dont care a iota about ext3fs, reiser3 does its job for me. Ext3 got quite a number of its own limits, but your iotas and ext3 limit are not even close to ssubject of this conversation. thank you. -- liberation loophole will make it clear [-- Attachment #2: Type: text/html, Size: 741 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 8:13 ` Hans Reiser 2006-07-24 10:25 ` Matthias Andree @ 2006-07-26 13:17 ` Pavel Machek 2006-07-27 15:39 ` Grzegorz Kulewski 2006-07-27 17:56 ` Jeff Garzik 1 sibling, 2 replies; 639+ messages in thread From: Pavel Machek @ 2006-07-26 13:17 UTC (permalink / raw) To: Hans Reiser Cc: Matthias Andree, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List Hi! > >of the story for me. There's nothing wrong about focusing on newer code, > >but the old code needs to be cared for, too, to fix remaining issues > >such as the "can only have N files with the same hash value". > > > Requires a disk format change, in a filesystem without plugins, to fix it. Well, too bad, if reiser3 is so broken it needs on-disk-format-change, then I guess doing that change is the right thing to do... Pavel -- Thanks for all the (sleeping) penguins. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 13:17 ` Pavel Machek @ 2006-07-27 15:39 ` Grzegorz Kulewski 2006-07-27 17:28 ` Matthias Andree 2006-07-27 17:56 ` Jeff Garzik 1 sibling, 1 reply; 639+ messages in thread From: Grzegorz Kulewski @ 2006-07-27 15:39 UTC (permalink / raw) To: Pavel Machek Cc: Hans Reiser, Matthias Andree, lkml, Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Wed, 26 Jul 2006, Pavel Machek wrote: > Hi! > >>> of the story for me. There's nothing wrong about focusing on newer code, >>> but the old code needs to be cared for, too, to fix remaining issues >>> such as the "can only have N files with the same hash value". >>> >> Requires a disk format change, in a filesystem without plugins, to fix it. > > Well, too bad, if reiser3 is so broken it needs on-disk-format-change, > then I guess doing that change is the right thing to do... Sorry for my stupid question, but could you tell me why starting to make incompatible changes to reiserfs3 now (when reiserfs3 "technology" is rather old) and making reiserfs3 unstable (again), possibly for several months or even years is better than fixing big issues with reiser4 (if there are any really big left) merging it and trying to stabilize it? For end user both ways will result in mkfs so... Thanks, Grzegorz Kulewski ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 15:39 ` Grzegorz Kulewski @ 2006-07-27 17:28 ` Matthias Andree 2006-07-28 2:40 ` Hans Reiser 0 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2006-07-27 17:28 UTC (permalink / raw) To: Grzegorz Kulewski; +Cc: LKML, ReiserFS List On Thu, 27 Jul 2006, Grzegorz Kulewski wrote: > Sorry for my stupid question, but could you tell me why starting to make > incompatible changes to reiserfs3 now (when reiserfs3 "technology" is > rather old) and making reiserfs3 unstable (again), possibly for several > months or even years is better than fixing big issues with reiser4 (if > there are any really big left) merging it and trying to stabilize it? > > For end user both ways will result in mkfs so... ext2fs and ext3fs, without "plugins", added dir_index as a compatible upgrade, with an e2fsck option (that implies optional) to build indices for directories without them. ext3fs is a compatible upgrade from ext2fs, it's as simple as unmount, tune2fs -j, mount. reiserfs 3.6 could deal with 3.5 file systems, and "mount -o conv" with a 3.6 driver would convert a 3.5 file system to 3.6 level (ISTR it had to do with large file support and perhaps NFS exportability, but don't quote me on that). I wonder what makes the hash overflow issue so complicated (other than differing business plans, that is) that upgrading in place isn't possible. Changes introduce instability, but namesys were proud of their regression testing - so how sustainable is their internal test suite? Instead, we're told reiser4 would fix this (quite likely) and we should wait until it's ready (OK, we shouldn't be using experimental stuff for production but rather for /tmp, but the file system will take many months to mature after integration) and it will be "mkfs" time - so reiser4 better be mature before we go that way if there's no way back short of "amrecover", "restore" or "tar -x". Smashing out most of the Cc:s in order not to bore people. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 17:28 ` Matthias Andree @ 2006-07-28 2:40 ` Hans Reiser 0 siblings, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-28 2:40 UTC (permalink / raw) To: Matthias Andree; +Cc: Grzegorz Kulewski, LKML, ReiserFS List Matthias Andree wrote: >I wonder what makes the hash overflow issue so complicated (other than >differing business plans, that is) that upgrading in place isn't >possible. Changes introduce instability, but namesys were proud of their >regression testing - so how sustainable is their internal test suite? > > > Never met a test suite the equal of a few million users..... People have this image of Namesys as some large corporation that has large resources. We just barely are going to be able to ship reiser4, at the cost of a LOT of financial pain. We can't afford to go in two directions at once. We can add bugfixes to V3, but adding features, I have to tell you that we ain't got the staff for both that and shipping V4. Our whole corporation has a budget about what most corporations spend on two programmers. We have 5 developers, including me, and making little bits of money is a constant distraction from the main work. Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 13:17 ` Pavel Machek 2006-07-27 15:39 ` Grzegorz Kulewski @ 2006-07-27 17:56 ` Jeff Garzik 2006-07-27 19:06 ` David Masover 2006-07-28 2:26 ` Hans Reiser 1 sibling, 2 replies; 639+ messages in thread From: Jeff Garzik @ 2006-07-27 17:56 UTC (permalink / raw) To: Pavel Machek Cc: Hans Reiser, Matthias Andree, lkml, Theodore Tso, LKML, ReiserFS List Pavel Machek wrote: > Hi! > >>> of the story for me. There's nothing wrong about focusing on newer code, >>> but the old code needs to be cared for, too, to fix remaining issues >>> such as the "can only have N files with the same hash value". >>> >> Requires a disk format change, in a filesystem without plugins, to fix it. > > Well, too bad, if reiser3 is so broken it needs on-disk-format-change, > then I guess doing that change is the right thing to do... Actually, there is reiser4 brokenness lurking in Hans' statement, too: A filesystem WITH plugins must still handle the standard Linux compatibility stuff that other filesystems handle. Plugins --do not-- mean that you can just change the filesystem format willy-nilly, with zero impact. Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 17:56 ` Jeff Garzik @ 2006-07-27 19:06 ` David Masover 2006-07-28 2:26 ` Hans Reiser 1 sibling, 0 replies; 639+ messages in thread From: David Masover @ 2006-07-27 19:06 UTC (permalink / raw) To: Jeff Garzik Cc: Pavel Machek, Hans Reiser, Matthias Andree, lkml, Theodore Tso, LKML, ReiserFS List Jeff Garzik wrote: > Pavel Machek wrote: >> Hi! >> >>>> of the story for me. There's nothing wrong about focusing on newer >>>> code, >>>> but the old code needs to be cared for, too, to fix remaining issues >>>> such as the "can only have N files with the same hash value". >>> Requires a disk format change, in a filesystem without plugins, to >>> fix it. > A filesystem WITH plugins must still handle the standard Linux > compatibility stuff that other filesystems handle. > > Plugins --do not-- mean that you can just change the filesystem format > willy-nilly, with zero impact. They --do-- mean that you can change much of the filesystem behavior without requiring massive on-disk changes or massive interface changes. After all, this is how many FUSE plugins work -- standard FS interface, usually uses another standard FS as storage, but does crazy things like compression, encryption, and other transformations in between. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 17:56 ` Jeff Garzik 2006-07-27 19:06 ` David Masover @ 2006-07-28 2:26 ` Hans Reiser 2006-07-28 11:15 ` metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) Jeff Garzik 1 sibling, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-07-28 2:26 UTC (permalink / raw) To: Jeff Garzik Cc: Pavel Machek, Matthias Andree, lkml, Theodore Tso, LKML, ReiserFS List Jeff Garzik wrote: > > Actually, there is reiser4 brokenness lurking in Hans' statement, too: Where! Someone tell me!;-) > > A filesystem WITH plugins must still handle the standard Linux > compatibility stuff that other filesystems handle. Hmm, you mean we should first implement regular unix file plugins before implementing enhanced functionality ones? Are you aware that reiser4 plugins are per file, and thus if a user selects a plugin that is not the default, and which has user visible semantic differences, it means they said they want non-standard behavior? > > Plugins --do not-- mean that you can just change the filesystem format > willy-nilly, with zero impact. Yes they do..... > > Jeff > > > > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 2:26 ` Hans Reiser @ 2006-07-28 11:15 ` Jeff Garzik 2006-07-28 14:02 ` Horst H. von Brand 0 siblings, 1 reply; 639+ messages in thread From: Jeff Garzik @ 2006-07-28 11:15 UTC (permalink / raw) To: Hans Reiser, Andrew Morton Cc: Theodore Tso, LKML, ReiserFS List, Linus Torvalds Hans Reiser wrote: > Jeff Garzik wrote: >> Plugins --do not-- mean that you can just change the filesystem format >> willy-nilly, with zero impact. > Yes they do..... Take off your marketing cap for a second :) Plugins do not eliminate the going-back-to-older-kernels issue, as discussed at length in the recent ext4 thread. Plugins complicate, rather than simplify, the issue of determining the compatibility between the kernel's metadata handling code and on-disk metadata. Quite simply, there is always a compatibility impact when the filesystem format changes. Enterprise customers will inevitably run several -generations- of your software, and they are very aware of migration costs when dealing with hardware+software life cycles in the 5-10 year range. Power users and kernel developers bounce between kernel versions all the time. This is the same problem I described in the ext4 thread, then summarized as "what ext3 am I talking to, today?" You can easily rewrite that as "what reiser4 set of plugins am I talking to, today?" Plugins == a ton of new software to be versioned and tracked. Rather than just track "reiser4", you must now track the collection of plugin versions as well. OBVIOUS increased complexity, and OBVIOUS additional admin learning curve for the serious IT shop. Now prepare for the bombshell revelation... THAT SAID, I do think fs {algorithm|metadata} plugins are a very interesting idea. The plugin infrastructure is the logical result of trying to support multiple incompatible {inode, dir, file data, ...} object types in the same Linux filesystem. Looking at in-tree Linux filesystems, one can see common design patterns in filesystems that hand-code support for multiple metadata format versions -- Al Viro's fs/sysv hacks being a simple and elegant example. It is then simple to follow that train of logic: why not make it easy to replace the directory algorithm [and associated metadata]? or the file data space management algorithms? or even the inode handling? why not allow customers to replace a stock algorithm with an exotic, site-specific one? In essence, reiser4 is not really a filesystem, but a second VFS, with a few stock metadata modules. Furthermore, it completely changes the notion of what a Linux filesystem is. Currently, each Linux filesystem is a tightly constrained set of metadata support. reiser4 changes "tightly constrained" to "infinity". While that freedom is certainly liberating, it also has obvious support costs due to new admin paradigms and customer configuration possibilities. I don't want to be the distro support person trying to fix a crash in "reiser4", where the customer has secretly replaced the standard inode data structure with a plugin written by an intern, and secretly replaced the directory algorithm with a closed source plugin from PickYourVendor. Trying picking through that mess with a filesystem debugger. The reiser4 plugin system is far better implemented as the VFS level, as an optional library, like fs/libfs.c. Such a library would contain factored-out infrastructure common to all filesystems that support multiple versions of the same object type, thereby shrinking the "real" filesystem code. Then add a tiny bit of code that allows addition and removal of object types within each filesystem. At that point, you have reduced a Linux filesystem to binding between a common name ("reiser4", "ext3") and a set of cooperating modules. A very interesting train of thought. But not one that belongs in the kernel in its current form. If you hide a second VFS inside fs/reiser4, I GUARANTEE that you will get the locking and multi-threading wrong. I guarantee your code will be under-reviewed, and NAK'd. You lose all the testing that would be gained by others travelling your code paths, if the code was implemented generically in fs/libplugin.c rather than fs/reiser4/* To move forward towards the goal of merging reiser4 into the kernel, you must recognize two distinctly SEPARATE ISSUES, and deal with them separately: 1) merging the latest whiz-bang collection of filesystem algorithms, under the filesystem name "reiser4" 2) adding the plugin concept to the Linux VFS I don't know if #2 will pass consensus, but I certainly think we should at least experiment in that direction. Your guys definitely have a lot of good ideas, and I'm glad Linux is seeing a fresh injection of new ideas. I just don't think its a good idea to merge a filesystem called "foo" that can be completely redefined in the field. Or least, not without thinking a lot more on the impact. Its a VFS _and_ a filesystem, and should be evaluated as such. Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 11:15 ` metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) Jeff Garzik @ 2006-07-28 14:02 ` Horst H. von Brand 2006-07-28 15:48 ` David Masover 0 siblings, 1 reply; 639+ messages in thread From: Horst H. von Brand @ 2006-07-28 14:02 UTC (permalink / raw) To: Jeff Garzik Cc: Hans Reiser, Andrew Morton, Theodore Tso, LKML, ReiserFS List, Linus Torvalds Jeff Garzik <jeff@garzik.org> wrote: [...] > It is then simple to follow that train of logic: why not make it easy > to replace the directory algorithm [and associated metadata]? or the > file data space management algorithms? or even the inode handling? > why not allow customers to replace a stock algorithm with an exotic, > site-specific one? IMVHO, such experiments should/must be done in userspace. And AFAICS, they can today. Go wild, but leave the kernel alone. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 14:02 ` Horst H. von Brand @ 2006-07-28 15:48 ` David Masover 2006-07-28 16:33 ` Linus Torvalds 0 siblings, 1 reply; 639+ messages in thread From: David Masover @ 2006-07-28 15:48 UTC (permalink / raw) To: Horst H. von Brand Cc: Jeff Garzik, Hans Reiser, Andrew Morton, Theodore Tso, LKML, ReiserFS List, Linus Torvalds Horst H. von Brand wrote: > Jeff Garzik <jeff@garzik.org> wrote: > > [...] > >> It is then simple to follow that train of logic: why not make it easy >> to replace the directory algorithm [and associated metadata]? or the >> file data space management algorithms? or even the inode handling? >> why not allow customers to replace a stock algorithm with an exotic, >> site-specific one? > > IMVHO, such experiments should/must be done in userspace. And AFAICS, they > can today. inode handling? Really? But what's wrong with people doing such experiments outside the kernel? AFAICS, "exotic, site-specific one" is not something that would be considered for inclusion. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 15:48 ` David Masover @ 2006-07-28 16:33 ` Linus Torvalds 2006-07-28 11:57 ` Hans Reiser 2006-08-01 15:32 ` Łukasz Mierzwa 0 siblings, 2 replies; 639+ messages in thread From: Linus Torvalds @ 2006-07-28 16:33 UTC (permalink / raw) To: David Masover Cc: Horst H. von Brand, Jeff Garzik, Hans Reiser, Andrew Morton, Theodore Tso, LKML, ReiserFS List On Fri, 28 Jul 2006, David Masover wrote: > > But what's wrong with people doing such experiments outside the kernel? > AFAICS, "exotic, site-specific one" is not something that would be considered > for inclusion. Here's a few ground rules at least from my viewpoint: - as long you call them "plugins" and treat them as such, I (and I suspect a lot of other people) are totally uninterested, and in fact, a lot of people will suspect that the primary aim is to either subvert the kernel copyright rules, or at best to create a mess of incompatible semantics with no sane overlying rules for locking etc. - the kernel does not, and _will_ not, support "hooks" that aren't actually used (and make sense) by normal kernel uses, for largely the same reasons. Interfaces need to be architected, make sense, and have real and valid GPL'd uses. In other words, if a filesystem wants to do something fancy, it needs to do so WITH THE VFS LAYER, not as some plugin architecture of its own. We already have exactly the plugin interface we need, and it literally _is_ the VFS interfaces - you can plug in your own filesystems with "register_filesystem()", which in turn indirectly allows you to plug in your per-file and per-directory operations for things like lookup etc. If that isn't enough, then the filesystem shouldn't make its own internal plug-in architecture that bypasses the VFS layer and exposes functionality that isn't necessarily sane. For example, reiser4 used to have (perhaps still does) these cool files that can be both directories and links, and I don't mind that at all, but I _do_ mind the fact that when Al Viro (long long ago) pointed out serious locking issues with them, those seemed to be totally brushed away. So as far as I'm concerned, the problem with reiser4 is that it hasn't tried to work with the VFS people. Now, I realize that the main VFS people aren't always easy to work with (Al and Christoph, take a bow), but that doesn't really change the basic facts. Al in particular is _always_ right. I don't think I've ever had the cojones to argue with Al.. Linus ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 16:33 ` Linus Torvalds @ 2006-07-28 11:57 ` Hans Reiser 2006-07-28 19:44 ` David Masover 2006-08-01 15:32 ` Łukasz Mierzwa 1 sibling, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-07-28 11:57 UTC (permalink / raw) To: Linus Torvalds Cc: David Masover, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List Linus Torvalds wrote: > > >In other words, if a filesystem wants to do something fancy, it needs to >do so WITH THE VFS LAYER, not as some plugin architecture of its own. > Where does VFS store the plugin ids that specify per file variations? /etc/fstab? Also, is (current) VFS the interface for specifying where the hash directory plugin goes (specifies what order directory entries are within the directory)? What about the node layout plugin? The disk format plugin? Etc.? Our approach is different, but it has reasons. We eliminated the layer of indirection that Hellwig objected to, in which VFS called the plugin which called its method. (Let us try to avoid arguments over whether if you extend VFS it is still called VFS or is called reiser4's plugin layer, agreed?) Regarding copyright, these plugins are compiled in. I have resisted dynamically loaded plugins for now, for reasons I will not go into here. You can either portray reiser4 as duplicating VFS, or you can portray it as taking it to the next level, in which files (objects with classes and methods) vary rather than solely filesystems. I would prefer the latter.;-) If you agree with taking it to the next level, then it is only to be expected that there are things that aren't well suited as they are, like parsing /etc/fstab when you have a trillion files. It is not very feasible to do it for all of the filesystems all at once given finite resources, it needs a prototype. > We >already have exactly the plugin interface we need, and it literally _is_ >the VFS interfaces - you can plug in your own filesystems with >"register_filesystem()", which in turn indirectly allows you to plug in >your per-file and per-directory operations for things like lookup etc. > >If that isn't enough, then the filesystem shouldn't make its own internal >plug-in architecture that bypasses the VFS layer and exposes functionality >that isn't necessarily sane. For example, reiser4 used to have (perhaps >still does) these cool files that can be both directories and links, and I >don't mind that at all, but I _do_ mind the fact that when Al Viro (long >long ago) pointed out serious locking issues with them, those seemed to be >totally brushed away. > > We disabled them, and we won't enable them until them until the bug is fixed. It is fixable, but not within this year's programmer resources to fix it. I thank him for pointing out the bug, and it is not trivial to fix it. >I don't think I've ever had the cojones to argue with Al.. > > Linux needs all kinds of people, not just the kind that can audit locking and copy Plan 9 well (which was very valuable to do), but now that Linux is large in the market it also needs those who can take it where Plan 9 has not already been. Why should we remain technology trailers instead of moving into the role of leaders? We have finite resources. We can give you a working filesystem with roughly twice the IO performance of the next fastest you have that does not disturb other filesystems,. (4x once the compression plugin is fully debugged). It also fixes various V3 bugs without disturbing that code with deep fixes. We cannot take every advantage reiser4 has and port it to every other filesystem in the form of genericized code as a prerequisite for going in, we just don't have the finances. Without plugins our per file compression plugins and encryption plugins cannot work. We can however let other filesystems use our code, and cooperate as they extend it and genericize it for their needs. Imposing code on other development teams is not how one best leads in open source, one sets an example and sees if others copy it. That is what I propose to do with our plugins. If no one copies, then we have harmed no one. Reasonable? Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 11:57 ` Hans Reiser @ 2006-07-28 19:44 ` David Masover 2006-07-28 13:34 ` Hans Reiser 2006-07-28 20:14 ` Hua Zhong 0 siblings, 2 replies; 639+ messages in thread From: David Masover @ 2006-07-28 19:44 UTC (permalink / raw) To: Hans Reiser Cc: Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List Hans Reiser wrote: > Linus Torvalds wrote: > >> >> In other words, if a filesystem wants to do something fancy, it needs to >> do so WITH THE VFS LAYER, not as some plugin architecture of its own. >> > (Let us try to avoid arguments over whether if you extend VFS it is > still called VFS or is called reiser4's plugin layer, agreed?) Ok, assuming you actually extend the VFS. The point is that if we want plugins, we don't have to implement them in ext3, but we have to put the plugin interface somewhere standard that is obviously not part of one filesystem (the VFS is the place) so that ext3 can implement a plugin system without having to read or touch a line of reiser4 code, and without compiling reiser4 into the kernel. It may ultimately not be any different, technically. This seems more like an organizational and political thing. But that doesn't make it less important or valid. > Regarding copyright, these plugins are compiled in. I have resisted > dynamically loaded plugins for now, for reasons I will not go into here. Good point, there's no GPL issue here. Plugins will either not be distributed (used internally) or distributed as GPL. > If you agree with taking it to the next level, then it is only to be > expected that there are things that aren't well suited as they are, like > parsing /etc/fstab when you have a trillion files. It is not very > feasible to do it for all of the filesystems all at once given finite > resources, it needs a prototype. Doesn't have to be in fstab, I hope, but think of it this way: ext3 uses JBD for its journaling. As I understand it, any other filesystem can also use JBD, and ext3 is mostly ext2 + JDB. So, make the plugin interface generic enough that it compliments the VFS, doesn't duplicate it, and doesn't exist as part of Reiser4 (and requires Reiser4 to be present). This may be just a bunch of renaming or a lot of work, I don't know, but I suspect it would make a lot of people a lot happier. > We have finite resources. We can give you a working filesystem with > roughly twice the IO performance of the next fastest you have that does > not disturb other filesystems,. (4x once the compression plugin is > fully debugged). It also fixes various V3 bugs without disturbing that > code with deep fixes. We cannot take every advantage reiser4 has and > port it to every other filesystem in the form of genericized code as a > prerequisite for going in, we just don't have the finances. This is a very compelling argument to me, but that's preaching to the choir, I've been running Reiser4 since before it was released, and before it looked like it was going to be stable anytime soon. It may be bold of me to speak for the LKML, but I think the general consensus is: The speed of a nonworking program is irrelevant -- no one cares how fast it is if it breaks things, either now or in the future. Currently, the concern is that it breaks things in the future, like adding plugin support to other filesystems. And no one else cares what your finances are. Not out of compassion, but out of practicality. For instance, it would be a huge financial benefit to me if the kernel displayed, in big bold letters while booting, that DAVID MASOVER WROTE THIS! (I'm sure Linus knows what I'm talking about.) It would also be untrue in my case, and pointless for everyone else in the kernel, so I have to find another way to make money. This is because one way Linux stays ahead of the competition (technologically) is by having quality be a much greater motivation than money. > Without > plugins our per file compression plugins and encryption plugins cannot > work. We can however let other filesystems use our code, and cooperate > as they extend it and genericize it for their needs. Imposing code on > other development teams is not how one best leads in open source, one > sets an example and sees if others copy it. That is what I propose to > do with our plugins. If no one copies, then we have harmed no one. > Reasonable? Someone still has to maintain the FS. Anyway, like I said, this is a very compelling argument for me, but code speaks louder than words. Maybe, if you insist it's not in the VFS, maybe use some insanely simple FS like RomFS to demonstrate another FS using plugins? Do that, and put it in the VFS. Maybe implement something like cramfs as a romfs plugin (another demo). Maybe even per-file -- implement zisofs as isofs + compression plugin. I think that would effectively kill any argument that plugins are bad because they are only in Reiser4. Beyond that is all marketing, I guess. The word "plugin" is not helping here, too many people remember Plugins like Macromedia Flash... ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 19:44 ` David Masover @ 2006-07-28 13:34 ` Hans Reiser 2006-07-28 20:58 ` Łukasz Mierzwa ` (5 more replies) 2006-07-28 20:14 ` Hua Zhong 1 sibling, 6 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-28 13:34 UTC (permalink / raw) To: David Masover Cc: Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List Let me put it from my perspective and stop pretending to be unbiased, so others can see where I am coming from. No one was interested in our plugins. We put the design on a website, spoke at conferences, no one but users were interested. No one would have conceived of having plugins if not for us. Our plugins affect no one else. Our self-contained code should not be delayed because other people delayed getting interested in our ideas and now they don't want us to have an advantage from leading. If they want to some distant day implement generic plugins, for which they have written not one line of code to date, fine, we'll use it when it exists, but right now those who haven't coded should get out of the way of people with working code. It is not fair or just to do otherwise. It also prevents users from getting advances they could be getting today, for no reason. Our code will not be harder to change once it is in the kernel, it will be easier, because there will be more staff funded to work on it. As for this "we are all too grand to be bothered with money to feed our families" business, building a system in which those who contribute can find a way to be rewarded is what managers do. Free software programmers may be willing to live on less than others, but they cannot live on nothing, and code that does not ever ship means living on nothing. If reiser4 is delayed enough, for reasons that have nothing to do with its needs, and without it having encumbered anyone else, it won't be ahead of the other filesystems when it ships. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 13:34 ` Hans Reiser @ 2006-07-28 20:58 ` Łukasz Mierzwa 2006-07-28 21:32 ` Mike Benoit 2006-07-29 3:21 ` Hans Reiser 2006-07-28 21:53 ` David Masover ` (4 subsequent siblings) 5 siblings, 2 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-07-28 20:58 UTC (permalink / raw) To: reiserfs-list Dnia Fri, 28 Jul 2006 15:34:36 +0200, Hans Reiser <reiser@namesys.com> napisa³: > Let me put it from my perspective and stop pretending to be unbiased, so > others can see where I am coming from. No one was interested in our > plugins. We put the design on a website, spoke at conferences, no one > but users were interested. No one would have conceived of having > plugins if not for us. Our plugins affect no one else. Our > self-contained code should not be delayed because other people delayed > getting interested in our ideas and now they don't want us to have an > advantage from leading. If they want to some distant day implement > generic plugins, for which they have written not one line of code to > date, fine, we'll use it when it exists, but right now those who haven't > coded should get out of the way of people with working code. It is not > fair or just to do otherwise. It also prevents users from getting > advances they could be getting today, for no reason. Our code will not > be harder to change once it is in the kernel, it will be easier, because > there will be more staff funded to work on it. > > As for this "we are all too grand to be bothered with money to feed our > families" business, building a system in which those who contribute can > find a way to be rewarded is what managers do. Free software > programmers may be willing to live on less than others, but they cannot > live on nothing, and code that does not ever ship means living on > nothing. > > If reiser4 is delayed enough, for reasons that have nothing to do with > its needs, and without it having encumbered anyone else, it won't be > ahead of the other filesystems when it ships. > It just hited me that 90% of mails (those I've read and remember) in which You guys are talking why r4 should or should not be merged did not contain a patch or not even a line of code as a reference, most of complains feels so abstract and ahead of time, there is nothing wrong with planing ahead but does it still makes sense when nobody else actually said that he would want to use it in future? Can't it be pusshed up to vfs later if it proves itself and there is demand for it? My question is what does it break now? It's been in mm for some time now, what troubles does it couse? Did anyone complained that it breaks mm and should be dropped? I'm just a end user and have no idea of linux internals, but all the fuzz about r4 seems so political and it's not just me who things so. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 20:58 ` Łukasz Mierzwa @ 2006-07-28 21:32 ` Mike Benoit 2006-07-29 3:21 ` Hans Reiser 1 sibling, 0 replies; 639+ messages in thread From: Mike Benoit @ 2006-07-28 21:32 UTC (permalink / raw) To: Łukasz Mierzwa; +Cc: reiserfs-list [-- Attachment #1: Type: text/plain, Size: 1975 bytes --] On Fri, 2006-07-28 at 22:58 +0200, Łukasz Mierzwa wrote: > It just hited me that 90% of mails (those I've read and remember) in which > You guys are talking why r4 should or should not be merged did not contain > a patch or not even a line of code as a reference, most of complains feels > so abstract and ahead of time, there is nothing wrong with planing ahead > but does it still makes sense when nobody else actually said that he would > want to use it in future? Can't it be pusshed up to vfs later if it proves > itself and there is demand for it? > My question is what does it break now? It's been in mm for some time now, > what troubles does it couse? Did anyone complained that it breaks mm and > should be dropped? > I'm just a end user and have no idea of linux internals, but all the fuzz > about r4 seems so political and it's not just me who things so. Agreed. As an outsider looking in trying to understand this all, it sounds like Hans can't win either way. People complain if the plugin code is contained entirely within Reiser4 and people complain if he tries to add functionality to the VFS that only Reiser4 will likely use for the foreseeable future. Or maybe ever use, we don't know yet? Ignoring pseudo files which Hans has pulled for now anyways, what difference is their between Reiser4 plugins and what EXT3 does with mount options like dir_index? 3rd party plugins can't be dynamically loaded, and some of them need mount options to enable them even after they are compiled in. So people can't "mistakenly" load some plugin that corrupts their data, or changes the on-disk format making it impossible to downgrade kernel versions. Is it the fact that Reiser4 plugins can be enabled/disable on a per file basis? If you don't use the word "plugin", how is adding dir_index code to EXT3 any different then adding encryption/compression code to Reiser4? -- Mike Benoit <ipso@snappymail.ca> [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 20:58 ` Łukasz Mierzwa 2006-07-28 21:32 ` Mike Benoit @ 2006-07-29 3:21 ` Hans Reiser 1 sibling, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-29 3:21 UTC (permalink / raw) To: Łukasz Mierzwa; +Cc: reiserfs-list Mike and Lukasz, please post your email to not just reiserfs-list, where only the reiserfs team will read it, but also to lkml if you could, please? Thanks for your support, user opinions count for a lot on lkml. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 13:34 ` Hans Reiser 2006-07-28 20:58 ` Łukasz Mierzwa @ 2006-07-28 21:53 ` David Masover 2006-07-29 1:36 ` Hans Reiser 2006-07-28 23:08 ` Jeff Garzik ` (3 subsequent siblings) 5 siblings, 1 reply; 639+ messages in thread From: David Masover @ 2006-07-28 21:53 UTC (permalink / raw) To: Hans Reiser Cc: Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List Hans Reiser wrote: > plugins if not for us. Our plugins affect no one else. Our > self-contained code should not be delayed because other people delayed And at the moment, I can still use Reiser4. If I ever make a distro, I will include Reiser4 support, probably as the default FS. That will help with getting into the kernel. So, why is it that it's urgent to get into the kernel? It will have to be bootstrapped one way or another -- either get it into the kernel so distros are more likely to include it, or get it into distros so the kernel is more likely to include it. But this is exactly the kind of thing that has happened before. With XFS, with Nvidia even -- clean it up, do it the way the kernel people want you to, because they're the ones who will have to maintain it for 20 years, and make sure it doesn't stop working or break anything else. > advantage from leading. If they want to some distant day implement > generic plugins, for which they have written not one line of code to > date, fine, we'll use it when it exists, but right now those who haven't > coded should get out of the way of people with working code. It is not > fair or just to do otherwise. It also prevents users from getting > advances they could be getting today, for no reason. It prevents users from doing nothing. > Our code will not > be harder to change once it is in the kernel, it will be easier, because > there will be more staff funded to work on it. If indeed it can be changed easily at all. I think the burden is on you to prove that you can change it to be more generic, rather than saying "Well, we could do it later, if people want us to..." > As for this "we are all too grand to be bothered with money to feed our > families" business, building a system in which those who contribute can > find a way to be rewarded is what managers do. Free software > programmers may be willing to live on less than others, but they cannot > live on nothing, and code that does not ever ship means living on nothing. Let me put this in perspective the best way I know how, with an inane analogy: Suppose there's a band. A good band, full of impossible superstars, led by a benevolent dictator -- for the sake of argument, let's call him Elvis. (the King -- dictator...) The band's doing really well, and Elvis & crew are getting paid fairly well just to share their music. (Ok, maybe Elvis didn't write anything, but bear with me...) Now, along comes a young Jimi Hendrix. He wants to be in the band, and Elvis says "Sure, just come up with a song we like and we'll play it, and you can even play it with us!" Sounds like a pretty good deal, so Jimi goes and tells all his friends, a couple of girls... Now, Jimi finishes his song, Elvis listens to it, and if you know anything about the music Elvis did and the music Hendrix did, you can imagine what happens next. Elvis says "This song just isn't us. But if you change it here, and here, and maybe here, we'll play it." Jimi is devastated. He'd been counting on playing it with them that night, and if he doesn't, he won't have any groupies, all his friends will laugh at him, and his life will kind of suck. But, does anyone really think Elvis has any business singing Voodoo Child? Or Purple Haze? Is it really fair to ask Elvis to completely change his act and embarrass himself to help Jimi out? The answer is, Jimi shouldn't have staked so much on something that was never a guarantee. And what's more, the real-life Jimi Hendrix never played with Elvis, but had a very successful band of his own. And if Elvis was still alive, seeing Jimi play might make him change his mind, maybe -- but at least with his own band, Jimi's success isn't pinned on playing with Elvis. This analogy is flawed in many ways, aside from just being plain chronologically impossible, but while I'm sure Linus feels bad for you, I don't think it's his obligation to compromise his kernel to help you out with your financial situation. So it would help a lot if you wouldn't keep bringing it up in what should be a technical discussion. So, if you can't make it work with VFS, then I guess you can't, and you're stuck either creating another interface which is not tied to any one filesystem and isn't tied to the VFS either, or coming up with a better (more specific) idea of how to make the Reiser4 plugin system acceptable to kernel maintainers without having to eat Ramen for a few years. Understand that I'm putting on my devil's advocate hat right now. I'd love to see Reiser4 merged tomorrow, or a week from now, exactly as it's written today, but I just don't see it happening. I'd also love to get more technical, but I just don't know the Reiser4 internals well enough to understand the feasibility (or not) of any of my vague ideas. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 21:53 ` David Masover @ 2006-07-29 1:36 ` Hans Reiser 2006-07-29 9:12 ` Arjan van de Ven ` (3 more replies) 0 siblings, 4 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-29 1:36 UTC (permalink / raw) To: David Masover Cc: Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List David Masover wrote: > > If indeed it can be changed easily at all. I think the burden is on > you to prove that you can change it to be more generic, rather than > saying "Well, we could do it later, if people want us to..." None of the filesystems other than reiser4 have any interest in using plugins, and this whole argument over how it should be in VFS is nonsensical because nobody but us has any interest in using the functionality. The burden is on the generic code authors to prove that they will ever ever do anything at all besides complain. Frankly, I don't think they will. I think they will never produce one line of code. Please cite one ext3 developer who is signed up to implement ext3 using plugins if they are supported by VFS. > >> . It also prevents users from getting >> advances they could be getting today, for no reason. > > > It prevents users from doing nothing. Most users not only cannot patch a kernel, they don't know what a patch is. It most certainly does. Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 1:36 ` Hans Reiser @ 2006-07-29 9:12 ` Arjan van de Ven 2006-07-29 18:11 ` David Masover 2006-07-29 12:28 ` Nikita Danilov ` (2 subsequent siblings) 3 siblings, 1 reply; 639+ messages in thread From: Arjan van de Ven @ 2006-07-29 9:12 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List > Most users not only cannot patch a kernel, they don't know what a patch > is. It most certainly does. obviously you can provide complete kernels, including precompiled ones. Most distros have a yum or apt or similar tool to suck down packages, it's trivial for users to add a site to that, so you could provide packages if you want and make it easy for them. -- if you want to mail me at work (you don't), use arjan (at) linux.intel.com ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 9:12 ` Arjan van de Ven @ 2006-07-29 18:11 ` David Masover 2006-07-29 20:02 ` Sarath Menon ` (2 more replies) 0 siblings, 3 replies; 639+ messages in thread From: David Masover @ 2006-07-29 18:11 UTC (permalink / raw) To: Arjan van de Ven Cc: Hans Reiser, Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 2512 bytes --] Arjan van de Ven wrote: >> Most users not only cannot patch a kernel, they don't know what a patch >> is. It most certainly does. > > > obviously you can provide complete kernels, including precompiled ones. > Most distros have a yum or apt or similar tool to suck down packages, > it's trivial for users to add a site to that, so you could provide > packages if you want and make it easy for them. What's more, many distros patch their kernels extensively. They listen to their users, too. So if there are a lot of users wanting this to be in the kernel, let them complain -- loudly -- to their distro to patch for Reiser4. It could be made even easier than that -- if Reiser4 is really so self-contained, it could be a whole separate set of modules, distributed on its own. Most gamers have to be content with doing something similar with the nvidia drivers -- for different reasons (licensing) but with the same results. I know Gentoo handles this automatically (emerge nvidia-kernel). Hmm, maybe it makes it a pain to have it as a root filesystem, so that really needs distro support. And yet, we have a whole system designed specifically for being able to load modules and tweak settings before the root FS is available. It's called initrd, or more recently, initramfs. I use an old-style initrd on this box, because my root FS is on an nvidia RAID, so I have to run a program called "dmraid" before I mount my root FS -- it's really trivial for me to have Reiser4 as a module, and I do, despite it being a root FS. I suspect that, all technical, political, and "mine is bigger" arguments aside, being available as a root FS of a distro, especially a default FS, would go a long way towards inclusion in the kernel. So all you have to do is find a reasonably popular and friendly distro, with people who are (for the moment) easier to deal with than kernel people. Most people, if they even know what a filesystem or a kernel is, still won't bother compiling their own kernel, you're right. But that means they are more likely to be using a distro-patched kernel than a stock, vanilla one. Is this enough to be "in the jukebox", Hans? Of course, it's odd that I mention Gentoo, the Gentoo people (as a rule) hate ReiserFS, but there are far more distros than there are popular kernel forks. I'm sure someone will be interested. That's assuming that making further changes (putting stuff in the VFS) is out of the question (for now). [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 890 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 18:11 ` David Masover @ 2006-07-29 20:02 ` Sarath Menon 2006-07-29 20:30 ` David Masover 2006-07-30 0:45 ` Maciej Sołtysiak 2006-08-07 22:00 ` Pavel Machek 2 siblings, 1 reply; 639+ messages in thread From: Sarath Menon @ 2006-07-29 20:02 UTC (permalink / raw) To: reiserfs-list; +Cc: David Masover On Saturday 29 July 2006 23:41, David Masover wrote: > I know Gentoo handles this automatically (emerge nvidia-kernel). I hate to say this again, but its not automatically. It requires more knowledge of ther user, and is more than asking users to patch the kernel. (I am in the support industry and I know of solaris admins, who have used that os for more than ten years ask me what the command patch does and what its impact is to production systems :-) ) >I use an old-style initrd on this box, because my root FS is > on an nvidia RAID, so I have to run a program called "dmraid" before I > mount my root FS -- it's really trivial for me to have Reiser4 as a > module, and I do, despite it being a root FS. As a fact I have it as a root FS (and that too reiser4, and as a module). Surprisingly, this is a wiki article too in http://gentoo-wiki.com, by someone who's more patient enought to write one :-) > I suspect that, all technical, political, and "mine is bigger" arguments > aside, being available as a root FS of a distro, especially a default > FS, would go a long way towards inclusion in the kernel. So all you > have to do is find a reasonably popular and friendly distro, with people > who are (for the moment) easier to deal with than kernel people. Its actually a matter of a hastle for the end user. That's where I would agree with Hans' comments quite earlier. > Most people, if they even know what a filesystem or a kernel is, still > won't bother compiling their own kernel, you're right. But that means > they are more likely to be using a distro-patched kernel than a stock, > vanilla one. Well, that's different, and that's the main problem in the linux empowerment that we see around ourselves. It finally revolves around the user, and as harsh as it may seem, it ultimately is the user who decides which fs is better (Give or take, they don't know the difference between a kernel or user-space. or for that matter far more basic things.) > Of course, it's odd that I mention Gentoo, the Gentoo people (as a rule) > hate ReiserFS, but there are far more distros than there are popular > kernel forks. I'm sure someone will be interested. I do, and that's partly due to the speed of /usr/portage on reiser4, and the easiness of blowing everything and starting from scratch : -) Cheers, Sarath -- The gentlemen looked one another over with microscopic carelessness. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 20:02 ` Sarath Menon @ 2006-07-29 20:30 ` David Masover 2006-07-30 11:55 ` Christian Trefzer 0 siblings, 1 reply; 639+ messages in thread From: David Masover @ 2006-07-29 20:30 UTC (permalink / raw) To: Sarath Menon; +Cc: reiserfs-list Sarath Menon wrote: > On Saturday 29 July 2006 23:41, David Masover wrote: >> I know Gentoo handles this automatically (emerge nvidia-kernel). > > I hate to say this again, but its not automatically. It requires more My point is, there's a fairly large group of users who would be willing to do that, as they're willing to do that to get their video drivers working. Also, assuming a distro did choose to support it, the only reason nvidia-kernel isn't just distributed with a pre-built kernel (on pre-built OSes, anyway) is licensing. This isn't a problem for Reiser4, which is GPL'd. >> I suspect that, all technical, political, and "mine is bigger" arguments >> aside, being available as a root FS of a distro, especially a default >> FS, would go a long way towards inclusion in the kernel. So all you >> have to do is find a reasonably popular and friendly distro, with people >> who are (for the moment) easier to deal with than kernel people. > > Its actually a matter of a hastle for the end user. That's where I would agree > with Hans' comments quite earlier. Putting it in the kernel doesn't make it any more or less of a hassle for the end-user than getting distro support. I remember downloading a different set of Debian floppies which supported XFS, before XFS was mainstream. In that sense, it's somewhat done already -- there is a Gentoo livecd that is kept patched for Reiser4. The problem with Gentoo, of course, is that if you're going to use Gentoo, you're going to be compiling your own kernel. So when it comes down to getting vanilla-sources or gentoo-sources, it wouldn't take much -- just a reiser4-sources, or a separate reiser4-module package. >> Most people, if they even know what a filesystem or a kernel is, still >> won't bother compiling their own kernel, you're right. But that means >> they are more likely to be using a distro-patched kernel than a stock, >> vanilla one. > > Well, that's different, and that's the main problem in the linux empowerment > that we see around ourselves. It finally revolves around the user, and as > harsh as it may seem, it ultimately is the user who decides which fs is > better (Give or take, they don't know the difference between a kernel or > user-space. or for that matter far more basic things.) If I remember right, SuSe had ReiserFS as the default at one point. If even one moderately popular Linux had Reiser4 as the default FS, it would get a LOT more exposure than it would simply being included (as EXPERIMENTAL, at that) in the vanilla kernel. >> Of course, it's odd that I mention Gentoo, the Gentoo people (as a rule) >> hate ReiserFS, but there are far more distros than there are popular >> kernel forks. I'm sure someone will be interested. > > I do, and that's partly due to the speed of /usr/portage on reiser4, and the > easiness of blowing everything and starting from scratch : -) Yes, I love /var/lib/portage/world also. Is /usr/portage still faster on Reiser4? I know it was when I switched, but that was years ago... ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 20:30 ` David Masover @ 2006-07-30 11:55 ` Christian Trefzer 2006-07-30 21:10 ` Wil Reichert 0 siblings, 1 reply; 639+ messages in thread From: Christian Trefzer @ 2006-07-30 11:55 UTC (permalink / raw) To: David Masover; +Cc: Sarath Menon, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 265 bytes --] On Sat, Jul 29, 2006 at 03:30:15PM -0500, David Masover wrote: > > Is /usr/portage still faster on Reiser4? I know it was when I switched, > but that was years ago... It is for sure. v4 is even better with fantastillions of small files than v3. uziel [-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-30 11:55 ` Christian Trefzer @ 2006-07-30 21:10 ` Wil Reichert 2006-07-30 21:39 ` Christian Trefzer 0 siblings, 1 reply; 639+ messages in thread From: Wil Reichert @ 2006-07-30 21:10 UTC (permalink / raw) To: Christian Trefzer; +Cc: David Masover, Sarath Menon, reiserfs-list Hmm, looks like I have a partition to re-format now. Wil On 7/30/06, Christian Trefzer <ctrefzer@gmx.de> wrote: > On Sat, Jul 29, 2006 at 03:30:15PM -0500, David Masover wrote: > > > > Is /usr/portage still faster on Reiser4? I know it was when I switched, > > but that was years ago... > > It is for sure. v4 is even better with fantastillions of small files > than v3. > > uziel > > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-30 21:10 ` Wil Reichert @ 2006-07-30 21:39 ` Christian Trefzer 2006-07-30 21:42 ` Christian Trefzer 0 siblings, 1 reply; 639+ messages in thread From: Christian Trefzer @ 2006-07-30 21:39 UTC (permalink / raw) To: Wil Reichert; +Cc: David Masover, Sarath Menon, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 289 bytes --] Hi Wil, On Sun, Jul 30, 2006 at 02:10:04PM -0700, Wil Reichert wrote: > Hmm, looks like I have a partition to re-format now. In order to avoid having to pull the whole tree via rsync again, you might want to grab my script from the list and adapt it to your needs. Kind regards, Chris [-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-30 21:39 ` Christian Trefzer @ 2006-07-30 21:42 ` Christian Trefzer 2006-07-30 22:38 ` David Masover 2006-07-31 15:21 ` Wil Reichert 0 siblings, 2 replies; 639+ messages in thread From: Christian Trefzer @ 2006-07-30 21:42 UTC (permalink / raw) To: Wil Reichert; +Cc: David Masover, Sarath Menon, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 355 bytes --] On Sun, Jul 30, 2006 at 11:39:42PM +0200, Christian Trefzer wrote: > > In order to avoid having to pull the whole tree via rsync again, you > might want to grab my script from the list and adapt it to your needs. Of course, you can tar it up manually instead. Silly me, but after approx. 9h of studying, little wonder ; ) Kind regards, Chris [-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-30 21:42 ` Christian Trefzer @ 2006-07-30 22:38 ` David Masover 2006-07-31 15:21 ` Wil Reichert 1 sibling, 0 replies; 639+ messages in thread From: David Masover @ 2006-07-30 22:38 UTC (permalink / raw) To: Christian Trefzer; +Cc: Wil Reichert, Sarath Menon, reiserfs-list Christian Trefzer wrote: > On Sun, Jul 30, 2006 at 11:39:42PM +0200, Christian Trefzer wrote: >> In order to avoid having to pull the whole tree via rsync again, you >> might want to grab my script from the list and adapt it to your needs. > > Of course, you can tar it up manually instead. Silly me, but after > approx. 9h of studying, little wonder ; ) In fact, the official install guide tells you to download a snapshot tarball first, then start syncing. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-30 21:42 ` Christian Trefzer 2006-07-30 22:38 ` David Masover @ 2006-07-31 15:21 ` Wil Reichert 2006-07-31 15:57 ` David Masover 1 sibling, 1 reply; 639+ messages in thread From: Wil Reichert @ 2006-07-31 15:21 UTC (permalink / raw) To: Christian Trefzer; +Cc: David Masover, Sarath Menon, reiserfs-list =) That was sorta the plan. Any idea how the fragmentation resulting from re-syncing the tree affects performance over time? Wil On 7/30/06, Christian Trefzer <ctrefzer@gmx.de> wrote: > On Sun, Jul 30, 2006 at 11:39:42PM +0200, Christian Trefzer wrote: > > > > In order to avoid having to pull the whole tree via rsync again, you > > might want to grab my script from the list and adapt it to your needs. > > Of course, you can tar it up manually instead. Silly me, but after > approx. 9h of studying, little wonder ; ) > > Kind regards, > Chris > > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-31 15:21 ` Wil Reichert @ 2006-07-31 15:57 ` David Masover 2006-07-31 16:05 ` Łukasz Mierzwa 2006-08-01 8:50 ` Christian Trefzer 0 siblings, 2 replies; 639+ messages in thread From: David Masover @ 2006-07-31 15:57 UTC (permalink / raw) To: Wil Reichert; +Cc: Christian Trefzer, Sarath Menon, reiserfs-list Wil Reichert wrote: > =) > > That was sorta the plan. > > Any idea how the fragmentation resulting from re-syncing the tree > affects performance over time? Try to post replies at the bottom, or below the context. Yes, it does affect it a lot. I have no idea how much, and I've never benchmarked it, but purely subjectively, my portage has gotten slower over time. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-31 15:57 ` David Masover @ 2006-07-31 16:05 ` Łukasz Mierzwa 2006-07-31 22:05 ` Sarath Menon 2006-08-01 12:27 ` Christian Trefzer 2006-08-01 8:50 ` Christian Trefzer 1 sibling, 2 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-07-31 16:05 UTC (permalink / raw) To: reiserfs-list Dnia Mon, 31 Jul 2006 17:57:35 +0200, David Masover <ninja@slaphack.com> napisa³: > Wil Reichert wrote: >> =) >> That was sorta the plan. >> Any idea how the fragmentation resulting from re-syncing the tree >> affects performance over time? > > Try to post replies at the bottom, or below the context. > > Yes, it does affect it a lot. I have no idea how much, and I've never > benchmarked it, but purely subjectively, my portage has gotten slower > over time. I gues that extens are much harder to reuse then normal inodes so when You have something as big as portage tree filled with nano files wich are being modified all the time then You just can't keep performance all the time. You can always tar, rm -fr /usr/portage, untar and You will probably speed things up a lot. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-31 16:05 ` Łukasz Mierzwa @ 2006-07-31 22:05 ` Sarath Menon 2006-08-01 12:27 ` Christian Trefzer 1 sibling, 0 replies; 639+ messages in thread From: Sarath Menon @ 2006-07-31 22:05 UTC (permalink / raw) To: reiserfs-list; +Cc: Łukasz Mierzwa On Monday 31 July 2006 21:35, £ukasz Mierzwa wrote: > I gues that extens are much harder to reuse then normal inodes so when You > have something as big as portage tree filled with nano files wich are > being modified all the time then You just can't keep performance all the > time. You can always tar, rm -fr /usr/portage, untar and You will probably > speed things up a lot. I agree here. On my home system this gave me a visible improvement in the sync process. I blew up my portage tree and I rebuilt it, and it gave me around ~10% improvement - this was around 6 months ago. I am not all that greedy, so i didn't do this do this than once :-) Sarath ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-31 16:05 ` Łukasz Mierzwa 2006-07-31 22:05 ` Sarath Menon @ 2006-08-01 12:27 ` Christian Trefzer 1 sibling, 0 replies; 639+ messages in thread From: Christian Trefzer @ 2006-08-01 12:27 UTC (permalink / raw) To: Łukasz Mierzwa; +Cc: reiserfs-list [-- Attachment #1: Type: text/plain, Size: 2055 bytes --] On Mon, Jul 31, 2006 at 06:05:01PM +0200, Łukasz Mierzwa wrote: > I gues that extens are much harder to reuse then normal inodes so when You > have something as big as portage tree filled with nano files wich are > being modified all the time then You just can't keep performance all the > time. You can always tar, rm -fr /usr/portage, untar and You will probably > speed things up a lot. I submitted a script to this list which takes care of everything required to recreate your fs. It even converts between different filesystems, for migration purposes or comparitive tests, and currently supports ext2|3, reiser3|4 and xfs. The thing is undergoing some surgery atm. to reduce forced disk flushes. I already replaced the call to sync() after every operation by one fsync() call on the archive file before the formatting takes place. What is still missing is functionality to retrieve things like fs label and UUID from the existing fs and reuse them during mkfs. Testing is also pending, so you might not want to hold your breath waiting for the funky version, the idea of which is to leave everything as it was found, except for better disk layout and possibly changed fs type. It is a completely different approach from convertfs, which tries to do the conversion in-place by moving the fs's contents into a new fs created within a sparse file on the same device and relocating the sparse file's blocks afterwards. My guess is that a failure of any kind in the latter process will destroy your data (this was the case last time I checked), while I do (at least try) everything to ensure that the tarball is written to platters before mkfs occurs. The new version will be posted to wiki.namesys.com asap, no timeframe attached though as Thursday yields an exam, so maybe on Friday, but who knows. The version already posted to the list works well, I used it at least a hundred times, even on stuff like /home and /usr (the latter works only from a live cd or custom initramfs). Kind regards, Chris [-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-31 15:57 ` David Masover 2006-07-31 16:05 ` Łukasz Mierzwa @ 2006-08-01 8:50 ` Christian Trefzer 2006-08-01 16:59 ` David Masover 1 sibling, 1 reply; 639+ messages in thread From: Christian Trefzer @ 2006-08-01 8:50 UTC (permalink / raw) To: David Masover; +Cc: Wil Reichert, Sarath Menon, reiserfs-list [-- Attachment #1: Type: text/plain, Size: 800 bytes --] On Mon, Jul 31, 2006 at 10:57:35AM -0500, David Masover wrote: > > Wil Reichert wrote: > > >Any idea how the fragmentation resulting from re-syncing the tree > >affects performance over time? > > Yes, it does affect it a lot. I have no idea how much, and I've never > benchmarked it, but purely subjectively, my portage has gotten slower > over time. Delayed allocation still performs a lot better here than the v3 "immediate" allocation. In addition, tree balancing operations are performed on flush as well, so what you get on disk is basically an almost-optimal tree. Of course, this will change a bit over time, but with v4 it takes a lot longer for that to happen than with v3 afaict. There _has_ been some worthwile development in the meantime : ) Kind regards, Chris [-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-08-01 8:50 ` Christian Trefzer @ 2006-08-01 16:59 ` David Masover 0 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-08-01 16:59 UTC (permalink / raw) To: Christian Trefzer; +Cc: Wil Reichert, Sarath Menon, reiserfs-list Christian Trefzer wrote: > On Mon, Jul 31, 2006 at 10:57:35AM -0500, David Masover wrote: >> Wil Reichert wrote: >> >>> Any idea how the fragmentation resulting from re-syncing the tree >>> affects performance over time? >> Yes, it does affect it a lot. I have no idea how much, and I've never >> benchmarked it, but purely subjectively, my portage has gotten slower >> over time. > > Delayed allocation still performs a lot better here than the v3 > "immediate" allocation. In addition, tree balancing operations are > performed on flush as well, so what you get on disk is basically an > almost-optimal tree. Of course, this will change a bit over time, but > with v4 it takes a lot longer for that to happen than with v3 afaict. > There _has_ been some worthwile development in the meantime : ) Hmm. The thing is, I don't remember v3 slowing down much at all, whereas v4 slowed down pretty dramatically after the first few weeks. It does seem pretty stable now, though, and it doesn't seem to be getting any slower. I've had this particular FS since... hmm... Is there an FS tool to check mkfs time? I think it's a year now, but I'd like to be sure. If not, I'll just find the oldest file, but the clock on this machine isn't reliable (have to set it with NTP every boot)... ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 18:11 ` David Masover 2006-07-29 20:02 ` Sarath Menon @ 2006-07-30 0:45 ` Maciej Sołtysiak 2006-07-30 1:08 ` Hans Reiser 2006-08-07 22:00 ` Pavel Machek 2 siblings, 1 reply; 639+ messages in thread From: Maciej Sołtysiak @ 2006-07-30 0:45 UTC (permalink / raw) To: David Masover; +Cc: ReiserFS List Hello David, Saturday, July 29, 2006, 8:11:11 PM, you wrote: > What's more, many distros patch their kernels extensively. They listen > to their users, too. So if there are a lot of users wanting this to be > in the kernel, let them complain -- loudly -- to their distro to patch > for Reiser4. Hmm, what about linspire / freespire ? Linsire is a proud reiser4 debugging sponsor as the website (http://www.namesys.com) says. Wouldn't they want to include reiser4 in their distro first? I removed the busy people from CC to save their time reading this. -- Best regards, Maciej ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-30 0:45 ` Maciej Sołtysiak @ 2006-07-30 1:08 ` Hans Reiser 2006-07-30 10:30 ` Łukasz Mierzwa 0 siblings, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-07-30 1:08 UTC (permalink / raw) To: Maciej Sołtysiak; +Cc: David Masover, ReiserFS List Maciej SoÅ‚tysiak wrote: > >Hmm, what about linspire / freespire ? Linsire is a proud reiser4 debugging >sponsor as the website (http://www.namesys.com) says. > >Wouldn't they want to include reiser4 in their distro first? > > > Not if the mainstream kernel is not going to add it..... Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-30 1:08 ` Hans Reiser @ 2006-07-30 10:30 ` Łukasz Mierzwa 0 siblings, 0 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-07-30 10:30 UTC (permalink / raw) To: Hans Reiser, LKML, reiserfs-list Dnia Sun, 30 Jul 2006 03:08:55 +0200, Hans Reiser <reiser@namesys.com> napisał: > Maciej Sołtysiak wrote: > >> >> Hmm, what about linspire / freespire ? Linsire is a proud reiser4 >> debugging >> sponsor as the website (http://www.namesys.com) says. >> >> Wouldn't they want to include reiser4 in their distro first? >> >> >> > Not if the mainstream kernel is not going to add it..... If I got it all rigth then the VFS issue is there from the moment when Hans made a request to merge reiser4 into mainstream, nobody nows what will happen tomorrow, it will go in without moving things to vfs, it can go in with moving some code to vfs or it can stay forever outside. If it's not sure then no distro will use it as a main fs (or at all) becouse if tommarow Hans will say "Ok, will move things to VFS" then for the next year or so reiser4 will be simply dead, it will take a lot of time to recode it and this change will provide a lot of bugs that will need to be debuged, it won't a be a single patch anymore as You will need to patch that adds reiser4 and a patch that alters VFS. What distro would toy with VFS? None until it's all done, fully and trully tested and merged into mainline, it won't happen quickly and it is not said that there won't be any more complains when this is done. In my opinion there is no best option for reiser4 only less evil ways, it's just too different from other fs, too independent. Again I'm no expert, just enduser. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) @ 2006-07-30 10:30 ` Łukasz Mierzwa 0 siblings, 0 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-07-30 10:30 UTC (permalink / raw) To: Hans Reiser, LKML, reiserfs-list Dnia Sun, 30 Jul 2006 03:08:55 +0200, Hans Reiser <reiser@namesys.com> napisa³: > Maciej So³tysiak wrote: > >> >> Hmm, what about linspire / freespire ? Linsire is a proud reiser4 >> debugging >> sponsor as the website (http://www.namesys.com) says. >> >> Wouldn't they want to include reiser4 in their distro first? >> >> >> > Not if the mainstream kernel is not going to add it..... If I got it all rigth then the VFS issue is there from the moment when Hans made a request to merge reiser4 into mainstream, nobody nows what will happen tomorrow, it will go in without moving things to vfs, it can go in with moving some code to vfs or it can stay forever outside. If it's not sure then no distro will use it as a main fs (or at all) becouse if tommarow Hans will say "Ok, will move things to VFS" then for the next year or so reiser4 will be simply dead, it will take a lot of time to recode it and this change will provide a lot of bugs that will need to be debuged, it won't a be a single patch anymore as You will need to patch that adds reiser4 and a patch that alters VFS. What distro would toy with VFS? None until it's all done, fully and trully tested and merged into mainline, it won't happen quickly and it is not said that there won't be any more complains when this is done. In my opinion there is no best option for reiser4 only less evil ways, it's just too different from other fs, too independent. Again I'm no expert, just enduser. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 18:11 ` David Masover 2006-07-29 20:02 ` Sarath Menon 2006-07-30 0:45 ` Maciej Sołtysiak @ 2006-08-07 22:00 ` Pavel Machek 2 siblings, 0 replies; 639+ messages in thread From: Pavel Machek @ 2006-08-07 22:00 UTC (permalink / raw) To: David Masover Cc: Arjan van de Ven, Hans Reiser, Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List Hi! > >> Most users not only cannot patch a kernel, they don't know what a patch > >> is. It most certainly does. > > > > > > obviously you can provide complete kernels, including precompiled ones. > > Most distros have a yum or apt or similar tool to suck down packages, > > it's trivial for users to add a site to that, so you could provide > > packages if you want and make it easy for them. > > What's more, many distros patch their kernels extensively. They listen > to their users, too. So if there are a lot of users wanting this to be > in the kernel, let them complain -- loudly -- to their distro to patch > for Reiser4. This is not true any more. SUSE (and RedHat) really try not to add patches to their kernels unless patch is obvious bugfix or merged to some later mainline. Yeah, we do exceptions, but getting exception is *way* harder than it used to be. Pavel -- Thanks for all the (sleeping) penguins. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 1:36 ` Hans Reiser 2006-07-29 9:12 ` Arjan van de Ven @ 2006-07-29 12:28 ` Nikita Danilov 2006-07-29 18:31 ` David Masover 2006-07-29 22:58 ` Hans Reiser 2006-07-29 18:25 ` David Masover 2006-07-31 13:28 ` Horst H. von Brand 3 siblings, 2 replies; 639+ messages in thread From: Nikita Danilov @ 2006-07-29 12:28 UTC (permalink / raw) To: Hans Reiser Cc: Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List Hans Reiser writes: > David Masover wrote: > > > > > If indeed it can be changed easily at all. I think the burden is on > > you to prove that you can change it to be more generic, rather than > > saying "Well, we could do it later, if people want us to..." > > None of the filesystems other than reiser4 have any interest in using > plugins, and this whole argument over how it should be in VFS is > nonsensical because nobody but us has any interest in using the > functionality. The burden is on the generic code authors to prove that > they will ever ever do anything at all besides complain. Frankly, I > don't think they will. I think they will never produce one line of code. > > Please cite one ext3 developer who is signed up to implement ext3 using > plugins if they are supported by VFS. In fact, they all do: struct inode_operations ext2_file_inode_operations; struct inode_operations ext2_dir_inode_operations; struct inode_operations ext2_special_inode_operations; struct inode_operations ext2_symlink_inode_operations; struct inode_operations ext2_fast_symlink_inode_operations; As you see, ext2 code already has multiple file "plugins", with persistent "plugin id" (stored in i_mode field of on-disk struct ext2_inode). > > Hans > Nikita. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 12:28 ` Nikita Danilov @ 2006-07-29 18:31 ` David Masover 2006-07-29 18:57 ` Nikita Danilov ` (2 more replies) 2006-07-29 22:58 ` Hans Reiser 1 sibling, 3 replies; 639+ messages in thread From: David Masover @ 2006-07-29 18:31 UTC (permalink / raw) To: Nikita Danilov Cc: Hans Reiser, Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 309 bytes --] Nikita Danilov wrote: > As you see, ext2 code already has multiple file "plugins", with > persistent "plugin id" (stored in i_mode field of on-disk struct > ext2_inode). Aha! So here's another question: Is it fair to ask Reiser4 to make its plugins generic, or should we be asking ext2/3 first? [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 890 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 18:31 ` David Masover @ 2006-07-29 18:57 ` Nikita Danilov 2006-07-29 23:01 ` Hans Reiser 2006-07-30 10:13 ` Łukasz Mierzwa 2 siblings, 0 replies; 639+ messages in thread From: Nikita Danilov @ 2006-07-29 18:57 UTC (permalink / raw) To: David Masover Cc: Hans Reiser, Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List David Masover writes: > Nikita Danilov wrote: > > > As you see, ext2 code already has multiple file "plugins", with > > persistent "plugin id" (stored in i_mode field of on-disk struct > > ext2_inode). > > Aha! So here's another question: Is it fair to ask Reiser4 to make its > plugins generic, or should we be asking ext2/3 first? ext2/3 plugins are generic: in Linux every file system can implement per-object behavior by specifying {file,inode,dentry,address_space}_operations. This mechanism is provided by VFS (and, in fact, is the only way that VFS interacts with file system) and is completely generic. > Nikita. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 18:31 ` David Masover 2006-07-29 18:57 ` Nikita Danilov @ 2006-07-29 23:01 ` Hans Reiser 2006-07-30 10:13 ` Łukasz Mierzwa 2 siblings, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-29 23:01 UTC (permalink / raw) To: David Masover Cc: Nikita Danilov, Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List David Masover wrote: >Nikita Danilov wrote: > > > >>As you see, ext2 code already has multiple file "plugins", with >>persistent "plugin id" (stored in i_mode field of on-disk struct >>ext2_inode). >> >> > >Aha! So here's another question: Is it fair to ask Reiser4 to make its >plugins generic, or should we be asking ext2/3 first? > > > Shhhhh....., ext* already made their plugins generic, job is done....:) ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 18:31 ` David Masover @ 2006-07-30 10:13 ` Łukasz Mierzwa 2006-07-29 23:01 ` Hans Reiser 2006-07-30 10:13 ` Łukasz Mierzwa 2 siblings, 0 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-07-30 10:13 UTC (permalink / raw) To: David Masover, LKML, reiserfs-list Dnia Sat, 29 Jul 2006 20:31:59 +0200, David Masover <ninja@slaphack.com> napisał: > Nikita Danilov wrote: > >> As you see, ext2 code already has multiple file "plugins", with >> persistent "plugin id" (stored in i_mode field of on-disk struct >> ext2_inode). > > Aha! So here's another question: Is it fair to ask Reiser4 to make its > plugins generic, or should we be asking ext2/3 first? > Doesn't iptables have plugins? Maybe we should make them generic so other packet filters can use them ;) ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) @ 2006-07-30 10:13 ` Łukasz Mierzwa 0 siblings, 0 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-07-30 10:13 UTC (permalink / raw) To: David Masover, LKML, reiserfs-list Dnia Sat, 29 Jul 2006 20:31:59 +0200, David Masover <ninja@slaphack.com> napisa³: > Nikita Danilov wrote: > >> As you see, ext2 code already has multiple file "plugins", with >> persistent "plugin id" (stored in i_mode field of on-disk struct >> ext2_inode). > > Aha! So here's another question: Is it fair to ask Reiser4 to make its > plugins generic, or should we be asking ext2/3 first? > Doesn't iptables have plugins? Maybe we should make them generic so other packet filters can use them ;) ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-30 10:13 ` Łukasz Mierzwa @ 2006-07-30 15:33 ` David Masover -1 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-07-30 15:33 UTC (permalink / raw) To: Łukasz Mierzwa; +Cc: LKML, reiserfs-list Łukasz Mierzwa wrote: > Dnia Sat, 29 Jul 2006 20:31:59 +0200, David Masover <ninja@slaphack.com> > napisał: > >> Nikita Danilov wrote: >> >>> As you see, ext2 code already has multiple file "plugins", with >>> persistent "plugin id" (stored in i_mode field of on-disk struct >>> ext2_inode). >> >> Aha! So here's another question: Is it fair to ask Reiser4 to make its >> plugins generic, or should we be asking ext2/3 first? >> > > Doesn't iptables have plugins? Maybe we should make them generic so > other packet filters can use them ;) Hey, yeah! I mean, not everyone wants to run the ipchains emulation on top of iptables! Some people really want to run ipchains with iptables plugins! </sarcasm> It is REALLY time for this discussion to get technical again, and to go way, way over my head. And it's time for me to go build my MythTV box, and see if I can shake out some Reiser4 bugs. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) @ 2006-07-30 15:33 ` David Masover 0 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-07-30 15:33 UTC (permalink / raw) To: Łukasz Mierzwa; +Cc: LKML, reiserfs-list £ukasz Mierzwa wrote: > Dnia Sat, 29 Jul 2006 20:31:59 +0200, David Masover <ninja@slaphack.com> > napisa³: > >> Nikita Danilov wrote: >> >>> As you see, ext2 code already has multiple file "plugins", with >>> persistent "plugin id" (stored in i_mode field of on-disk struct >>> ext2_inode). >> >> Aha! So here's another question: Is it fair to ask Reiser4 to make its >> plugins generic, or should we be asking ext2/3 first? >> > > Doesn't iptables have plugins? Maybe we should make them generic so > other packet filters can use them ;) Hey, yeah! I mean, not everyone wants to run the ipchains emulation on top of iptables! Some people really want to run ipchains with iptables plugins! </sarcasm> It is REALLY time for this discussion to get technical again, and to go way, way over my head. And it's time for me to go build my MythTV box, and see if I can shake out some Reiser4 bugs. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 12:28 ` Nikita Danilov 2006-07-29 18:31 ` David Masover @ 2006-07-29 22:58 ` Hans Reiser 2006-07-31 11:58 ` Nikita Danilov 1 sibling, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-07-29 22:58 UTC (permalink / raw) To: Nikita Danilov Cc: Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List Nikita Danilov wrote: >Hans Reiser writes: > > David Masover wrote: > > > > > > > > If indeed it can be changed easily at all. I think the burden is on > > > you to prove that you can change it to be more generic, rather than > > > saying "Well, we could do it later, if people want us to..." > > > > None of the filesystems other than reiser4 have any interest in using > > plugins, and this whole argument over how it should be in VFS is > > nonsensical because nobody but us has any interest in using the > > functionality. The burden is on the generic code authors to prove that > > they will ever ever do anything at all besides complain. Frankly, I > > don't think they will. I think they will never produce one line of code. > > > > Please cite one ext3 developer who is signed up to implement ext3 using > > plugins if they are supported by VFS. > >In fact, they all do: > >struct inode_operations ext2_file_inode_operations; >struct inode_operations ext2_dir_inode_operations; >struct inode_operations ext2_special_inode_operations; >struct inode_operations ext2_symlink_inode_operations; >struct inode_operations ext2_fast_symlink_inode_operations; > >As you see, ext2 code already has multiple file "plugins", with >persistent "plugin id" (stored in i_mode field of on-disk struct >ext2_inode). > > > > > Hans > > > >Nikita. > > > > > So the job is already done. Good. Reiser4 can be included then.:) Hans "The Easily Agreeable" Reiser ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 22:58 ` Hans Reiser @ 2006-07-31 11:58 ` Nikita Danilov 0 siblings, 0 replies; 639+ messages in thread From: Nikita Danilov @ 2006-07-31 11:58 UTC (permalink / raw) To: Hans Reiser Cc: Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List Hans Reiser writes: > Nikita Danilov wrote: [...] > > > >As you see, ext2 code already has multiple file "plugins", with > >persistent "plugin id" (stored in i_mode field of on-disk struct > >ext2_inode). > > > >Nikita. > > > > > So the job is already done. Good. Reiser4 can be included then.:) Indeed, namesys already implemented requirements pointed to by Christoph Hellwig, as far as meta-data plugins are concerned, so this issue is not an obstacle. > > Hans "The Easily Agreeable" Reiser > PS: please, don't trim CC lists. Nikita. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 1:36 ` Hans Reiser 2006-07-29 9:12 ` Arjan van de Ven 2006-07-29 12:28 ` Nikita Danilov @ 2006-07-29 18:25 ` David Masover 2006-07-31 13:28 ` Horst H. von Brand 3 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-07-29 18:25 UTC (permalink / raw) To: Hans Reiser Cc: Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 2442 bytes --] Hans Reiser wrote: > David Masover wrote: > >> If indeed it can be changed easily at all. I think the burden is on >> you to prove that you can change it to be more generic, rather than >> saying "Well, we could do it later, if people want us to..." > > None of the filesystems other than reiser4 have any interest in using > plugins, and this whole argument over how it should be in VFS is > nonsensical because nobody but us has any interest in using the > functionality. The burden is on the generic code authors to prove that > they will ever ever do anything at all besides complain. Frankly, I > don't think they will. I think they will never produce one line of code. I think it's fair to say that 5-10 years from now, with different ext3 maintainers, when the Reiser4 concept has proven itself, people will want plugins for ext3, and the ext3 developers will like the idea. ext* is one of those things that just refuses to die. I use ext3 for my /boot fs, so that I don't have to patch Grub for Reiser4, and so that at least I can mess with the bootloader from any rescue CD if something goes wrong. It's for kind of the same reason that Gentoo builds a 32-bit Grub, even though I'm booting a 64-bit OS -- just in case. I also use ext2 for my initrd. There are other monstrosities that will likely never die, also. ISO9660, UDF, and VFAT probably all have worse storage characteristics than Reiser4, in that as I understand it, they won't pack multiple files into a block. So Reiser4 might even make a good boot cd FS, storing things more efficiently -- but even if I'm right, those three filesystems will last forever, because they are currently well supported on every major OS, and I think one of ISO/UDF is required for DVDs. So for whatever reason someone's using another filesystem, even if all they need is the on-disk format (my reason for ext3 /boot and vfat on USB thumbdrives), I think it's reasonable to expect that they may one day want plugin functionality. People who like Reiser filesystems will do just fine running Reiser4 with a (udf|iso|vfat) storage driver, but people who don't will just want the higher level stuff. You're probably right and this is years of work for something that may not be worth anything, but I think this is what is going through people's heads as they look at this plugin system. So see my comments about distro inclusion. [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 890 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-29 1:36 ` Hans Reiser ` (2 preceding siblings ...) 2006-07-29 18:25 ` David Masover @ 2006-07-31 13:28 ` Horst H. von Brand 3 siblings, 0 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-07-31 13:28 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List Hans Reiser <reiser@namesys.com> wrote: > David Masover wrote: > > If indeed it can be changed easily at all. I think the burden is on > > you to prove that you can change it to be more generic, rather than > > saying "Well, we could do it later, if people want us to..." > None of the filesystems other than reiser4 have any interest in using > plugins, Then they are probably nonsense... > and this whole argument over how it should be in VFS is > nonsensical because nobody but us has any interest in using the > functionality. The burden is on the generic code authors to prove that > they will ever ever do anything at all besides complain. Frankly, I > don't think they will. I think they will never produce one line of code. It is /your/ burden to show that they are worthwhile. And if they are, thay aren't worthwhile only in your pet filesystem... > Please cite one ext3 developer who is signed up to implement ext3 using > plugins if they are supported by VFS. Have you asked? > >> . It also prevents users from getting > >> advances they could be getting today, for no reason. > > It prevents users from doing nothing. > Most users not only cannot patch a kernel, they don't know what a patch > is. It most certainly does. Simple answer: Fork the kernel with all your grandiose plans, and see which one wins. Open source, remember? It is not as if your only chance is to get it into the official kernel. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 13:34 ` Hans Reiser 2006-07-28 20:58 ` Łukasz Mierzwa 2006-07-28 21:53 ` David Masover @ 2006-07-28 23:08 ` Jeff Garzik 2006-07-29 3:20 ` Hans Reiser 2006-07-29 10:30 ` metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) Łukasz Mierzwa ` (2 subsequent siblings) 5 siblings, 1 reply; 639+ messages in thread From: Jeff Garzik @ 2006-07-28 23:08 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Linus Torvalds, Horst H. von Brand, Andrew Morton, Theodore Tso, LKML, ReiserFS List Hans Reiser wrote: > As for this "we are all too grand to be bothered with money to feed our > families" business, building a system in which those who contribute can > find a way to be rewarded is what managers do. Free software > programmers may be willing to live on less than others, but they cannot > live on nothing, and code that does not ever ship means living on nothing. Using guilt as an argument in a technical discussion is a flashing red sign that says "I have no technical rebuttal" > If reiser4 is delayed enough, for reasons that have nothing to do with > its needs, and without it having encumbered anyone else, it won't be > ahead of the other filesystems when it ships. What delay? This is open source. There is nothing stopping you from worldwide distribution of the coolest Linux filesystem in the world. Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 23:08 ` Jeff Garzik @ 2006-07-29 3:20 ` Hans Reiser 2006-08-07 21:56 ` ext3 vs reiserfs speed (was Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)) Pavel Machek 0 siblings, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-07-29 3:20 UTC (permalink / raw) To: Jeff Garzik Cc: David Masover, Linus Torvalds, Horst H. von Brand, Andrew Morton, Theodore Tso, LKML, ReiserFS List Jeff Garzik wrote: > > Using guilt as an argument in a technical discussion is a flashing red > sign that says "I have no technical rebuttal" Wow, that is really nervy. Let's recap this all: * reiser4 has a 2x performance advantage over the next fastest FS (ext3), and when compression ships in a month that will double again as well as save space. See http://www.namesys.com/benchmarks.html, and then ask the reiserfs-list@namesys.com whether those benchmarks are fair representations of their experiences. This is in a field where a 25% advantage is a hard won big deal. * we described our plugin architecture in 2001. No other FS developers were interested, only the users were, and it was presented quite a lot. * So we implemented plugins for ourselves, because no other FS developers would possibly have supported us touching their code. (I do not say that they erred in this.) * No one has actually made a serious case for it being genericizable when you get to the details, it is all just handwaving. I'd be surprised if >10% of it was FS independent, and unsurprised if making that 10% FS independent made the code ossified and hard to maintain. I do not in anyway claim that those who choose to implement Reiser4 plugins are not deeply affected by Reiser4 design choices. Most of the value of writing Reiser4 plugins comes from being able to reuse Reiser4 code as you choose to in the process, and if Reiser4 is not to your taste as a whole, then nobody should impose our plugins upon you. VFS is a bad enough straight jacket for FS developers, we don't need even more mandated design decisions for the FS developers to come who will be brighter than us. Actually, I would like to see Nate Diller implement a competing VFS layer, I think he would do a very good job of that. * Here we are today, and Reiser4 plugins work. Now some say that because we did it for Reiser4 and not for every other FS, that we should be excluded from the kernel. So we are supposed to re-implement it as generic code, which will involve years of time, and then finally something will be coded and nobody but us will use it, and then they will tell us that because nobody but us wants to use it it cannot go in. If you disagree, find one ext3 developer who wants to rewrite ext3 to use plugins and change its disk format to do it. And you have the nerve to say that this ever was a technical discussion? Our code measurably works the best. If folks want to imitate it, go ahead, but don't blame us for making our code work without first making those other folks's code work. The technical rebuttal you ask for is http://www.namesys.com/benchmarks.html. The only time this argument gets technical is when akpm is involved. He was right about what should technically be done about batch write, which, by the way, was greeted upon completion with an if only reiser4 uses it then it should not go in response. We are being penalized for thinking too differently, and this whole ping-pong between "no we don't want to do it your way" and "you did it your way for only you, redo it for us even though we won't ever use it" and "oh, you redid it for us but none of us want to use it, so no it is an imposition and cannot go in" is the Kafka-esque manifestation of that. If only reiser4 wants to use something, then just let us do it in our little corner without bothering anybody else. (Though any advice from akpm that he has time for giving us is always welcome.) David, we aren't asking to be in the band, we are asking to be in the jukebox. I think enough users want to go 2x as fast that the users would benefit from our being in the jukebox. Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* ext3 vs reiserfs speed (was Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)) 2006-07-29 3:20 ` Hans Reiser @ 2006-08-07 21:56 ` Pavel Machek 0 siblings, 0 replies; 639+ messages in thread From: Pavel Machek @ 2006-08-07 21:56 UTC (permalink / raw) To: Hans Reiser Cc: Jeff Garzik, David Masover, Linus Torvalds, Horst H. von Brand, Andrew Morton, Theodore Tso, LKML, ReiserFS List Hi! > > Using guilt as an argument in a technical discussion is a flashing red > > sign that says "I have no technical rebuttal" > > Wow, that is really nervy. Let's recap this all: > > * reiser4 has a 2x performance advantage over the next fastest FS > (ext3), and when compression ships in a month that will double again as > well as save space. See http://www.namesys.com/benchmarks.html, and Does that mean that ext3 is faster than reiser3? Wow, that would be good reason to switch default filesystem to ext3 (or reiser4?) in next suse release. Pavel -- Thanks for all the (sleeping) penguins. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 13:34 ` Hans Reiser ` (2 preceding siblings ...) 2006-07-28 23:08 ` Jeff Garzik @ 2006-07-29 10:30 ` Łukasz Mierzwa 2006-07-29 10:31 ` Łukasz Mierzwa 2006-07-30 18:41 ` Horst H. von Brand 5 siblings, 0 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-07-29 10:30 UTC (permalink / raw) To: LKML Dnia Fri, 28 Jul 2006 15:34:36 +0200, Hans Reiser <reiser@namesys.com> napisał: > Let me put it from my perspective and stop pretending to be unbiased, so > others can see where I am coming from. No one was interested in our > plugins. We put the design on a website, spoke at conferences, no one > but users were interested. No one would have conceived of having > plugins if not for us. Our plugins affect no one else. Our > self-contained code should not be delayed because other people delayed > getting interested in our ideas and now they don't want us to have an > advantage from leading. If they want to some distant day implement > generic plugins, for which they have written not one line of code to > date, fine, we'll use it when it exists, but right now those who haven't > coded should get out of the way of people with working code. It is not > fair or just to do otherwise. It also prevents users from getting > advances they could be getting today, for no reason. Our code will not > be harder to change once it is in the kernel, it will be easier, because > there will be more staff funded to work on it. > > As for this "we are all too grand to be bothered with money to feed our > families" business, building a system in which those who contribute can > find a way to be rewarded is what managers do. Free software > programmers may be willing to live on less than others, but they cannot > live on nothing, and code that does not ever ship means living on > nothing. > > If reiser4 is delayed enough, for reasons that have nothing to do with > its needs, and without it having encumbered anyone else, it won't be > ahead of the other filesystems when it ships. > It just hited me that 90% of mails (those I've read and remember) in which You guys are talking why r4 should or should not be merged did not contain a patch or not even a line of code as a reference, most of complains feels so abstract and ahead of time, there is nothing wrong with planing ahead but does it still makes sense when nobody else actually said that he would want to use it in future? Can't it be pusshed up to vfs later if it proves itself and there is demand for it? My question is what does it break now? It's been in mm for some time now, what troubles does it couse? Did anyone complained that it breaks mm and should be dropped? I'm just a end user and have no idea of linux internals, but all the fuzz about r4 seems so political and it's not just me who things so. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 13:34 ` Hans Reiser ` (3 preceding siblings ...) 2006-07-29 10:30 ` metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) Łukasz Mierzwa @ 2006-07-29 10:31 ` Łukasz Mierzwa 2006-07-30 18:41 ` Horst H. von Brand 5 siblings, 0 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-07-29 10:31 UTC (permalink / raw) To: LKML Dnia Fri, 28 Jul 2006 15:34:36 +0200, Hans Reiser <reiser@namesys.com> napisał: > Let me put it from my perspective and stop pretending to be unbiased, so > others can see where I am coming from. No one was interested in our > plugins. We put the design on a website, spoke at conferences, no one > but users were interested. No one would have conceived of having > plugins if not for us. Our plugins affect no one else. Our > self-contained code should not be delayed because other people delayed > getting interested in our ideas and now they don't want us to have an > advantage from leading. If they want to some distant day implement > generic plugins, for which they have written not one line of code to > date, fine, we'll use it when it exists, but right now those who haven't > coded should get out of the way of people with working code. It is not > fair or just to do otherwise. It also prevents users from getting > advances they could be getting today, for no reason. Our code will not > be harder to change once it is in the kernel, it will be easier, because > there will be more staff funded to work on it. > > As for this "we are all too grand to be bothered with money to feed our > families" business, building a system in which those who contribute can > find a way to be rewarded is what managers do. Free software > programmers may be willing to live on less than others, but they cannot > live on nothing, and code that does not ever ship means living on > nothing. > > If reiser4 is delayed enough, for reasons that have nothing to do with > its needs, and without it having encumbered anyone else, it won't be > ahead of the other filesystems when it ships. > It just hited me that 90% of mails (those I've read and remember) in which You guys are talking why r4 should or should not be merged did not contain a patch or not even a line of code as a reference, most of complains feels so abstract and ahead of time, there is nothing wrong with planing ahead but does it still makes sense when nobody else actually said that he would want to use it in future? Can't it be pusshed up to vfs later if it proves itself and there is demand for it? My question is what does it break now? It's been in mm for some time now, what troubles does it couse? Did anyone complained that it breaks mm and should be dropped? I'm just a end user and have no idea of linux internals, but all the fuzz about r4 seems so political and it's not just me who thinks so. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 13:34 ` Hans Reiser ` (4 preceding siblings ...) 2006-07-29 10:31 ` Łukasz Mierzwa @ 2006-07-30 18:41 ` Horst H. von Brand 2006-07-30 21:15 ` Toby Thain 5 siblings, 1 reply; 639+ messages in thread From: Horst H. von Brand @ 2006-07-30 18:41 UTC (permalink / raw) To: Hans Reiser Cc: David Masover, Linus Torvalds, Horst H. von Brand, Jeff Garzik, Andrew Morton, Theodore Tso, LKML, ReiserFS List Hans Reiser <reiser@namesys.com> wrote: > Let me put it from my perspective and stop pretending to be unbiased, so > others can see where I am coming from. OK, but /that/ was pretty clear from day one... > No one was interested in our > plugins. Should tell you something... > We put the design on a website, spoke at conferences, no one > but users were interested. Again, should tell you something... "Look, a cool new gadget nobody ever imagined before" sure attracts lots of people. Intriguing. Play around a bit. Go for next "last novel gadget". Rinse, repeat. > No one would have conceived of having > plugins if not for us. Perhaps because nobody else sees any sense in them? > Our plugins affect no one else. Wrong. Others will want plugins if they are any use, at the very least. And then there is the possibility of /massive/ maintainance problems ("So, you /are/ using Reiser 4, the broken file has attached plugin A version 5.3, and is reached through a "directory" with plugin D 3.4rc5, and ..."). /I/ wouldn't want to be at the receiving end of bug reports which after a dozen rounds boil down to this. > Our > self-contained code should not be delayed Not your call to make. > because other people delayed > getting interested in our ideas and now they don't want us to have an > advantage from leading. Your problem, not Linux'. > If they want to some distant day implement > generic plugins, for which they have written not one line of code to > date, fine, we'll use it when it exists, but right now those who haven't > coded should get out of the way of people with working code. But they are! Just branch the kernel, and be done with it. > It is not > fair or just to do otherwise. /You/ are asking the kernel developers for a /huge/ favor. Even totally go out of their ways, and acting contrary to their set ways and beliefs. Saying "no" to that can't be called "unfair"... > It also prevents users from getting > advances they could be getting today, for no reason. OK, so you have /never/ seen any reasons given here for not placing Reiser 4 into the kernel? Strange... > Our code will not > be harder to change once it is in the kernel, it will be easier, because > there will be more staff funded to work on it. Right. Just like Reiser 3 right now. > As for this "we are all too grand to be bothered with money to feed our > families" business, building a system in which those who contribute can > find a way to be rewarded is what managers do. Free software > programmers may be willing to live on less than others, but they cannot > live on nothing, and code that does not ever ship means living on > nothing. Then go do something else... > If reiser4 is delayed enough, for reasons that have nothing to do with > its needs, and without it having encumbered anyone else, it won't be > ahead of the other filesystems when it ships. How is that important in any way for the Linux kernel? This is not (and has not been for quite some time now) an experimental operating system. And it has /never/ been a dumpling ground for the next grand idea, it has always been about sound engineering. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-30 18:41 ` Horst H. von Brand @ 2006-07-30 21:15 ` Toby Thain 0 siblings, 0 replies; 639+ messages in thread From: Toby Thain @ 2006-07-30 21:15 UTC (permalink / raw) To: Horst H.von Brand; +Cc: ReiserFS List > >> If reiser4 is delayed enough, for reasons that have nothing to do >> with >> its needs, and without it having encumbered anyone else, it won't be >> ahead of the other filesystems when it ships. > > How is that important in any way for the Linux kernel? This is not > (and has > not been for quite some time now) an experimental operating system. > And it > has /never/ been a dumpling ground for the next grand idea, it has > always > been about sound engineering. One of the groundbreaking things about Linux' (modular) development model was, I thought, that it can succeed at both... not necessarily at the same time, but for different users? --T ^ permalink raw reply [flat|nested] 639+ messages in thread
* RE: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 19:44 ` David Masover 2006-07-28 13:34 ` Hans Reiser @ 2006-07-28 20:14 ` Hua Zhong 2006-07-28 13:40 ` Hans Reiser 1 sibling, 1 reply; 639+ messages in thread From: Hua Zhong @ 2006-07-28 20:14 UTC (permalink / raw) To: 'David Masover', 'Hans Reiser' Cc: 'Linus Torvalds', 'Horst H. von Brand', 'Jeff Garzik', 'Andrew Morton', 'Theodore Tso', 'LKML', 'ReiserFS List' > Doesn't have to be in fstab, I hope, but think of it this > way: ext3 uses JBD for its journaling. As I understand it, > any other filesystem can also use JBD, and ext3 is mostly ext2 + JDB. The fact that no other major journaling filesystems use JBD except EXT3 might make this idea less appealing.. You can also say "other filesystems CAN use plugin". But who actually want to use it now? Even if there are people that want to use it, they should help doing the work. I remember someone said something along the lines of "Linux is evolution, not revolution". To me it seems unreasonable to put all the "revolutionary" VFS burden upon reiserfs team. It's not practical. Hua ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 20:14 ` Hua Zhong @ 2006-07-28 13:40 ` Hans Reiser 0 siblings, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-28 13:40 UTC (permalink / raw) To: Hua Zhong Cc: 'David Masover', 'Linus Torvalds', 'Horst H. von Brand', 'Jeff Garzik', 'Andrew Morton', 'Theodore Tso', 'LKML', 'ReiserFS List' Hua Zhong wrote: >I remember someone said something along the lines of "Linux is evolution, not revolution". To me it seems unreasonable to put all >the "revolutionary" VFS burden upon reiserfs team. It's not practical. > > > > Thanks for saying that Hua. We have a guy named Nate Diller, who probably could fix VFS up pretty nicely if Namesys did not have him earning the consulting income the rest of the team lives on (he is doing io scheduler work at the moment). That said, he would need a lot of time, and stopping reiser4 inclusion to await his work merely ensures his work will never happen. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-07-28 16:33 ` Linus Torvalds @ 2006-08-01 15:32 ` Łukasz Mierzwa 2006-08-01 15:32 ` Łukasz Mierzwa 1 sibling, 0 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-08-01 15:32 UTC (permalink / raw) To: Linus Torvalds, LKML, reiserfs-list Dnia Fri, 28 Jul 2006 18:33:56 +0200, Linus Torvalds <torvalds@osdl.org> napisał: > In other words, if a filesystem wants to do something fancy, it needs to > do so WITH THE VFS LAYER, not as some plugin architecture of its own. We > already have exactly the plugin interface we need, and it literally _is_ > the VFS interfaces - you can plug in your own filesystems with > "register_filesystem()", which in turn indirectly allows you to plug in > your per-file and per-directory operations for things like lookup etc. What fancy (beside cryptocompress) does reiser4 do now? Can someone point me to a list of things that are required by kernel mainteiners to merge reiser4 into vanilla? I feel like I'm getting lost with current reiser4 status and things that are need to be done. Łukasz Mierzwa ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) @ 2006-08-01 15:32 ` Łukasz Mierzwa 0 siblings, 0 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-08-01 15:32 UTC (permalink / raw) To: Linus Torvalds, LKML, reiserfs-list Dnia Fri, 28 Jul 2006 18:33:56 +0200, Linus Torvalds <torvalds@osdl.org> napisa³: > In other words, if a filesystem wants to do something fancy, it needs to > do so WITH THE VFS LAYER, not as some plugin architecture of its own. We > already have exactly the plugin interface we need, and it literally _is_ > the VFS interfaces - you can plug in your own filesystems with > "register_filesystem()", which in turn indirectly allows you to plug in > your per-file and per-directory operations for things like lookup etc. What fancy (beside cryptocompress) does reiser4 do now? Can someone point me to a list of things that are required by kernel mainteiners to merge reiser4 into vanilla? I feel like I'm getting lost with current reiser4 status and things that are need to be done. £ukasz Mierzwa ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-08-01 15:32 ` Łukasz Mierzwa @ 2006-08-01 16:43 ` Vladimir V. Saveliev -1 siblings, 0 replies; 639+ messages in thread From: Vladimir V. Saveliev @ 2006-08-01 16:43 UTC (permalink / raw) To: Łukasz Mierzwa; +Cc: LKML, reiserfs-list Hello On Tue, 2006-08-01 at 17:32 +0200, Łukasz Mierzwa wrote: > Dnia Fri, 28 Jul 2006 18:33:56 +0200, Linus Torvalds <torvalds@osdl.org> > napisał: > > > In other words, if a filesystem wants to do something fancy, it needs to > > do so WITH THE VFS LAYER, not as some plugin architecture of its own. We > > already have exactly the plugin interface we need, and it literally _is_ > > the VFS interfaces - you can plug in your own filesystems with > > "register_filesystem()", which in turn indirectly allows you to plug in > > your per-file and per-directory operations for things like lookup etc. > What fancy (beside cryptocompress) does reiser4 do now? it is supposed to provide an ability to easy modify filesystem behaviour in various aspects without breaking compatibility. > Can someone point me to a list of things that are required by kernel > mainteiners to merge reiser4 into vanilla? list of features reiser4 does not have now: O_DIRECT support - we are working on it now various block size support quota support xattrs and acls list of warnings about reiser4 code: I think that last big list of useful comments (from Christoph Hellwig <hch@infradead.org>) is addressed. Well, except for one minor (I believe) place in file release. Currently, Andrew is trying to find some time to review reiser4 code. > I feel like I'm getting lost with current reiser4 status and things that > are need to be done. > > Łukasz Mierzwa > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) @ 2006-08-01 16:43 ` Vladimir V. Saveliev 0 siblings, 0 replies; 639+ messages in thread From: Vladimir V. Saveliev @ 2006-08-01 16:43 UTC (permalink / raw) To: Łukasz Mierzwa; +Cc: LKML, reiserfs-list Hello On Tue, 2006-08-01 at 17:32 +0200, Łukasz Mierzwa wrote: > Dnia Fri, 28 Jul 2006 18:33:56 +0200, Linus Torvalds <torvalds@osdl.org> > napisał: > > > In other words, if a filesystem wants to do something fancy, it needs to > > do so WITH THE VFS LAYER, not as some plugin architecture of its own. We > > already have exactly the plugin interface we need, and it literally _is_ > > the VFS interfaces - you can plug in your own filesystems with > > "register_filesystem()", which in turn indirectly allows you to plug in > > your per-file and per-directory operations for things like lookup etc. > What fancy (beside cryptocompress) does reiser4 do now? it is supposed to provide an ability to easy modify filesystem behaviour in various aspects without breaking compatibility. > Can someone point me to a list of things that are required by kernel > mainteiners to merge reiser4 into vanilla? list of features reiser4 does not have now: O_DIRECT support - we are working on it now various block size support quota support xattrs and acls list of warnings about reiser4 code: I think that last big list of useful comments (from Christoph Hellwig <hch@infradead.org>) is addressed. Well, except for one minor (I believe) place in file release. Currently, Andrew is trying to find some time to review reiser4 code. > I feel like I'm getting lost with current reiser4 status and things that > are need to be done. > > Łukasz Mierzwa > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-08-01 16:43 ` Vladimir V. Saveliev (?) @ 2006-08-02 18:45 ` Horst H. von Brand 2006-08-02 17:07 ` Łukasz Mierzwa 2006-08-05 1:01 ` David Masover -1 siblings, 2 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-08-02 18:45 UTC (permalink / raw) To: Vladimir V. Saveliev; +Cc: Łukasz Mierzwa, LKML, reiserfs-list [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain, Size: 1727 bytes --] Vladimir V. Saveliev <vs@namesys.com> wrote: > On Tue, 2006-08-01 at 17:32 +0200, Łukasz Mierzwa wrote: > > Dnia Fri, 28 Jul 2006 18:33:56 +0200, Linus Torvalds <torvalds@osdl.org> > > napisał: > > > In other words, if a filesystem wants to do something fancy, it needs to > > > do so WITH THE VFS LAYER, not as some plugin architecture of its own. We > > > already have exactly the plugin interface we need, and it literally _is_ > > > the VFS interfaces - you can plug in your own filesystems with > > > "register_filesystem()", which in turn indirectly allows you to plug in > > > your per-file and per-directory operations for things like lookup etc. > > What fancy (beside cryptocompress) does reiser4 do now? > > it is supposed to provide an ability to easy modify filesystem behaviour > in various aspects without breaking compatibility. If it just modifies /behaviour/ it can't really do much. And what can be done here is more the job of the scheduler, not of the filesystem. Keep your hands off it! If it somehow modifies /on disk format/, it (by *definition*) isn't compatible. Ditto. > > Can someone point me to a list of things that are required by kernel > > mainteiners to merge reiser4 into vanilla? > > list of features reiser4 does not have now: > O_DIRECT support - we are working on it now > various block size support Is this required? > quota support > xattrs and acls Without those, it is next to useless anyway. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-08-02 18:45 ` Horst H. von Brand @ 2006-08-02 17:07 ` Łukasz Mierzwa 2006-08-05 1:01 ` David Masover 1 sibling, 0 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-08-02 17:07 UTC (permalink / raw) To: Horst H. von Brand, LKML, reiserfs-list Dnia Wed, 02 Aug 2006 20:45:07 +0200, Horst H. von Brand <vonbrand@inf.utfsm.cl> napisał: > Vladimir V. Saveliev <vs@namesys.com> wrote: >> On Tue, 2006-08-01 at 17:32 +0200, Å�ukasz Mierzwa wrote: >> > Dnia Fri, 28 Jul 2006 18:33:56 +0200, Linus Torvalds >> <torvalds@osdl.org> >> > napisaÅ‚: >> > > In other words, if a filesystem wants to do something fancy, it >> needs to >> > > do so WITH THE VFS LAYER, not as some plugin architecture of its >> own. We >> > > already have exactly the plugin interface we need, and it literally >> _is_ >> > > the VFS interfaces - you can plug in your own filesystems with >> > > "register_filesystem()", which in turn indirectly allows you to >> plug in >> > > your per-file and per-directory operations for things like lookup >> etc. > >> > What fancy (beside cryptocompress) does reiser4 do now? >> >> it is supposed to provide an ability to easy modify filesystem behaviour >> in various aspects without breaking compatibility. > > If it just modifies /behaviour/ it can't really do much. And what can be > done here is more the job of the scheduler, not of the filesystem. Keep > your > hands off it! You modify the way the fs stores files or let You access them, since when it is a job for a scheduler? > If it somehow modifies /on disk format/, it (by *definition*) isn't > compatible. Ditto. > >> > Can someone point me to a list of things that are required by kernel >> > mainteiners to merge reiser4 into vanilla? >> >> list of features reiser4 does not have now: >> O_DIRECT support - we are working on it now >> various block size support > > Is this required? > >> quota support >> xattrs and acls > > Without those, it is next to useless anyway. I don't use any of this and I live quite happly. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) @ 2006-08-02 17:07 ` Łukasz Mierzwa 0 siblings, 0 replies; 639+ messages in thread From: Łukasz Mierzwa @ 2006-08-02 17:07 UTC (permalink / raw) To: Horst H. von Brand, LKML, reiserfs-list Dnia Wed, 02 Aug 2006 20:45:07 +0200, Horst H. von Brand <vonbrand@inf.utfsm.cl> napisał: > Vladimir V. Saveliev <vs@namesys.com> wrote: >> On Tue, 2006-08-01 at 17:32 +0200, Å�ukasz Mierzwa wrote: >> > Dnia Fri, 28 Jul 2006 18:33:56 +0200, Linus Torvalds >> <torvalds@osdl.org> >> > napisaÅ‚: >> > > In other words, if a filesystem wants to do something fancy, it >> needs to >> > > do so WITH THE VFS LAYER, not as some plugin architecture of its >> own. We >> > > already have exactly the plugin interface we need, and it literally >> _is_ >> > > the VFS interfaces - you can plug in your own filesystems with >> > > "register_filesystem()", which in turn indirectly allows you to >> plug in >> > > your per-file and per-directory operations for things like lookup >> etc. > >> > What fancy (beside cryptocompress) does reiser4 do now? >> >> it is supposed to provide an ability to easy modify filesystem behaviour >> in various aspects without breaking compatibility. > > If it just modifies /behaviour/ it can't really do much. And what can be > done here is more the job of the scheduler, not of the filesystem. Keep > your > hands off it! You modify the way the fs stores files or let You access them, since when it is a job for a scheduler? > If it somehow modifies /on disk format/, it (by *definition*) isn't > compatible. Ditto. > >> > Can someone point me to a list of things that are required by kernel >> > mainteiners to merge reiser4 into vanilla? >> >> list of features reiser4 does not have now: >> O_DIRECT support - we are working on it now >> various block size support > > Is this required? > >> quota support >> xattrs and acls > > Without those, it is next to useless anyway. I don't use any of this and I live quite happly. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 2006-08-02 18:45 ` Horst H. von Brand 2006-08-02 17:07 ` Łukasz Mierzwa @ 2006-08-05 1:01 ` David Masover 1 sibling, 0 replies; 639+ messages in thread From: David Masover @ 2006-08-05 1:01 UTC (permalink / raw) To: Horst H. von Brand Cc: Vladimir V. Saveliev, Łukasz Mierzwa, LKML, reiserfs-list Horst H. von Brand wrote: > Vladimir V. Saveliev <vs@namesys.com> wrote: >> On Tue, 2006-08-01 at 17:32 +0200, Åukasz Mierzwa wrote: >>> What fancy (beside cryptocompress) does reiser4 do now? >> it is supposed to provide an ability to easy modify filesystem behaviour >> in various aspects without breaking compatibility. > > If it just modifies /behaviour/ it can't really do much. And what can be > done here is more the job of the scheduler, not of the filesystem. Keep your > hands off it! Say wha? There's a lot you can do with the _representation_ of the on-disk format without changing the _physical_ on-disk format. As a very simple example, a plugin could add a sysfs-like folder with information about that particular filesystem. Yes, I know there are better ways to do things, but there are things you can change about behavior without (I think) touching the scheduler. Or am I wrong about the scope of the "scheduler"? > If it somehow modifies /on disk format/, it (by *definition*) isn't > compatible. Ditto. Cryptocompress is compatible with kernels that have a working cryptocompress plugin. Other kernels will notice that they are meant to be read by cryptocompress, and (I hope) refuse to read files they won't be able to. Same would be true of any plugin that changes the disk format. But, the above comments about behavior still hold. There's a lot you can do with plugins without changing the on-disk format. If you want a working example, look to your own favorite filesystems that support quotas, xattrs, and acls -- is an on-disk FS format with those enabled compatible with a kernel that doesn't support them (has them turned off)? How about ext3, with its journaling -- is the journaling all in the scheduler? But isn't the ext3 disk format compatible with ext2? >> quota support >> xattrs and acls > > Without those, it is next to useless anyway. What is? The FS? I use neither on desktop machines, though I'd appreciate xattrs for Beagle. Or are you talking about the plugins? See above, then. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-23 7:20 ` Hans Reiser 2006-07-23 9:12 ` Matt Heler @ 2006-07-23 11:48 ` Jan-Benedict Glaw 2006-07-23 17:43 ` Toby Thain 2006-07-23 20:46 ` Jeff Mahoney ` (3 subsequent siblings) 5 siblings, 1 reply; 639+ messages in thread From: Jan-Benedict Glaw @ 2006-07-23 11:48 UTC (permalink / raw) To: Hans Reiser; +Cc: Jeff Garzik, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1378 bytes --] On Sun, 2006-07-23 01:20:40 -0600, Hans Reiser <reiser@namesys.com> wrote: > There is nothing about small patches that makes them better code. There Erm, a small patch is something which should _obviously_ fix one issue. A small patch, containing at max some 100 lines, can easily be read and understood. A complete filesystem (I'm co-maintaining one for an ancient on-disk format, too) isn't really easy to understand or to verify from looking at it for 5min. > is no reason we should favor them, if the developers are willing to work > on something for 5 years to escape a local optimum, that is often the > RIGHT thing to do. I give a shit of nothing to some 5 year work if I cannot verify that it won't hurt me at some point. > It is importand that we embrace our diversity, and be happy for the > strength it gives us. Some of us are good at small patches that evolve, > and some are good at escaping local optimums. We all have value, both > trees and grass have their place in the world. Just put reiser4 in some GIT tree and publish it. Maybe you can place it on git.kernel.org . MfG, JBG -- Jan-Benedict Glaw jbglaw@lug-owl.de +49-172-7608481 Signature of: ...und wenn Du denkst, es geht nicht mehr, the second : kommt irgendwo ein Lichtlein her. [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion @ 2006-07-23 17:43 ` Toby Thain 0 siblings, 0 replies; 639+ messages in thread From: Toby Thain @ 2006-07-23 17:43 UTC (permalink / raw) To: ReiserFS List On 23-Jul-06, at 7:48 AM, Jan-Benedict Glaw wrote: > On Sun, 2006-07-23 01:20:40 -0600, Hans Reiser <reiser@namesys.com> > wrote: >> There is nothing about small patches that makes them better code. >> There > > Erm, a small patch is something which should _obviously_ fix one > issue. A small patch, containing at max some 100 lines, can easily be > read and understood. > > A complete filesystem (I'm co-maintaining one for an ancient on-disk > format, too) isn't really easy to understand or to verify from looking > at it for 5min. Nonetheless, "There is nothing about small patches that makes them better code". Hans is quite right. Long patches just take longer to read. This can make them harder to penetrate review, as he describes, with analogy. > >> is no reason we should favor them, if the developers are willing >> to work >> on something for 5 years to escape a local optimum, that is often the >> RIGHT thing to do. > > I give a shit of nothing to some 5 year work if I cannot verify that > it won't hurt me at some point. Do you really review all patches to ensure this? It is well understood that only once r4 reaches mainline will it get the wider testing it must have to shake down. Lucky Namesys is not deterred by ingratitude or there would be no "5 year work" for us to contemplate at all. > >> It is importand that we embrace our diversity, and be happy for the >> strength it gives us. Some of us are good at small patches that >> evolve, >> and some are good at escaping local optimums. We all have value, >> both >> trees and grass have their place in the world. > > Just put reiser4 in some GIT tree and publish it. Maybe you can place > it on git.kernel.org . Why should Hans give up the aspiration to have r4 in mainline due to a small number of regressive personalities (a.k.a. politics)? To much of the Linux world R3 has been an extremely valuable contribution; r4 promises to be even more so. --T > > MfG, JBG > > -- > Jan-Benedict Glaw jbglaw@lug-owl.de > +49-172-7608481 > Signature of: ...und wenn Du denkst, es geht > nicht mehr, > the second : kommt irgendwo ein > Lichtlein her. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion @ 2006-07-23 17:43 ` Toby Thain 0 siblings, 0 replies; 639+ messages in thread From: Toby Thain @ 2006-07-23 17:43 UTC (permalink / raw) To: ReiserFS List On 23-Jul-06, at 7:48 AM, Jan-Benedict Glaw wrote: > On Sun, 2006-07-23 01:20:40 -0600, Hans Reiser <reiser@namesys.com> > wrote: >> There is nothing about small patches that makes them better code. >> There > > Erm, a small patch is something which should _obviously_ fix one > issue. A small patch, containing at max some 100 lines, can easily be > read and understood. > > A complete filesystem (I'm co-maintaining one for an ancient on-disk > format, too) isn't really easy to understand or to verify from looking > at it for 5min. Nonetheless, "There is nothing about small patches that makes them better code". Hans is quite right. Long patches just take longer to read. This can make them harder to penetrate review, as he describes, with analogy. > >> is no reason we should favor them, if the developers are willing >> to work >> on something for 5 years to escape a local optimum, that is often the >> RIGHT thing to do. > > I give a shit of nothing to some 5 year work if I cannot verify that > it won't hurt me at some point. Do you really review all patches to ensure this? It is well understood that only once r4 reaches mainline will it get the wider testing it must have to shake down. Lucky Namesys is not deterred by ingratitude or there would be no "5 year work" for us to contemplate at all. > >> It is importand that we embrace our diversity, and be happy for the >> strength it gives us. Some of us are good at small patches that >> evolve, >> and some are good at escaping local optimums. We all have value, >> both >> trees and grass have their place in the world. > > Just put reiser4 in some GIT tree and publish it. Maybe you can place > it on git.kernel.org . Why should Hans give up the aspiration to have r4 in mainline due to a small number of regressive personalities (a.k.a. politics)? To much of the Linux world R3 has been an extremely valuable contribution; r4 promises to be even more so. --T > > MfG, JBG > > -- > Jan-Benedict Glaw jbglaw@lug-owl.de > +49-172-7608481 > Signature of: ...und wenn Du denkst, es geht > nicht mehr, > the second : kommt irgendwo ein > Lichtlein her. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-23 7:20 ` Hans Reiser 2006-07-23 9:12 ` Matt Heler 2006-07-23 11:48 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Jan-Benedict Glaw @ 2006-07-23 20:46 ` Jeff Mahoney 2006-07-23 21:15 ` Hans Reiser 2006-07-25 19:13 ` Russell Cattelan ` (2 subsequent siblings) 5 siblings, 1 reply; 639+ messages in thread From: Jeff Mahoney @ 2006-07-23 20:46 UTC (permalink / raw) To: Hans Reiser; +Cc: Jeff Garzik, Theodore Tso, LKML, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hans Reiser wrote: > I just got an email from the programmer who wrote the MythTV bug saying > that he is just too busy to bother fixing the bug in his code..... so > my response is that a Namesys programmer is going to fix it on Monday. Hans - I'll accept blame when it's my bug, but the MythTV one isn't. I've been working with the bitmap code and did the analysis to track down what was happening, but that doesn't make it a bug in my code. That particular bug isn't in the bitmap scanning code, it's a side effect of the write batching higher up. It's looking for a window of 32 blocks, and there's just no window that large available. It ends up scanning all the bitmaps looking for the window, and then backs off to single block allocations. The scanning code works fine, and it does skip where there aren't enough free blocks available in a particular bitmap. It's a pathological case when the file system is seriously fragmented. A quick fix would be to set a flag indicating that future writes shouldn't bother trying to find a window that large, but that's a hack. A better allocation algorithm would keep track of free space extents in memory, subject to getting dropped by memory pressure. Since that information would be separate from the bitmaps themselves, we could get rid of that nasty "is this block free, but in the journal?" check that we need to do as well. It's invasive, and a quicker fix would just be to track the largest window, and rescan when it gets used or a block in that bitmap gets freed. That said, I actually did start work on a fix for this one, but I really just don't have the time right now. - -Jeff - -- Jeff Mahoney SUSE Labs -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFEw+BBLPWxlyuTD7IRAtG8AKCOWW/AH3NAen6gd6BToJGVfzdnNACfYkVS j2/6yAAeWKAhs4ng9fdGW0Y= =gB+v -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-23 20:46 ` Jeff Mahoney @ 2006-07-23 21:15 ` Hans Reiser 2006-07-23 23:22 ` Jeff Mahoney 0 siblings, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-07-23 21:15 UTC (permalink / raw) To: Jeff Mahoney; +Cc: Jeff Garzik, Theodore Tso, LKML, ReiserFS List Jeff Mahoney wrote: > > > That particular bug isn't in the bitmap scanning code, it's a side > effect of the write batching higher up. Did you write the code that looks for a window of 32 blocks? If not, and if this code has been around for a long time, I apologize. I thought you did write it and added it in recent months. > > > It's a pathological case when the file system is seriously fragmented. Most bugs are pathological cases.;-) > A > quick fix would be to set a flag indicating that future writes shouldn't > bother trying to find a window that large, There are lots of quick fixes. 1) The quickest is to not scan for the window at all. 2) The second quickest is to limit the number of bitmaps that will be scanned to some number like 3. 3) The not at all quickest is to track free extents like XFS does, which is not a hack, but it belongs in a development branch. I am not sure it is worth the complexity, but my mind is not closed. On monday we will do 1) or 2), probably 1). After the repacker is done, we should review all our block allocation algorithms. I have an idea for how to do things more optimally for streaming media that will avoid fragmentation over time, and when combined with the repacker may make 3 not worthwhile. I am grateful that you and Chris do bug fixes, but when you guys are too busy, (and that can and will happen to any of us), the baton needs to get passed. V3 needs to be a zero defect product, and once we know it is a bug I don't want bugs in V3 to remain unfixed for more than a day plus the time it takes to fix it. If you do add code, I want any bugs that show up in the aftermath of mainstream merging to get jumped on. Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-23 21:15 ` Hans Reiser @ 2006-07-23 23:22 ` Jeff Mahoney 2006-07-23 22:35 ` Hans Reiser 0 siblings, 1 reply; 639+ messages in thread From: Jeff Mahoney @ 2006-07-23 23:22 UTC (permalink / raw) To: Hans Reiser; +Cc: Jeff Garzik, Theodore Tso, LKML, ReiserFS List -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hans Reiser wrote: > Jeff Mahoney wrote: > >> >> That particular bug isn't in the bitmap scanning code, it's a side >> effect of the write batching higher up. > > Did you write the code that looks for a window of 32 > blocks? If not, and if this code has been around for a long time, I > apologize. I thought you did write it and added it in recent months. Nope. The scan_bitmap_block() code that looks for windows was added in a changeset from a bk merge against 2.5.33 in September 2002. The change to want a minimum size window was added in June 2004 to 2.6.8-rc2. That patch is actually credited to both Mason and I, but I don't recall who wrote that bit. It may well have been my code, after all, but it's certainly not a new bug. *shrug* I guess MythTV might just be generating an i/o pattern that hadn't been seen before. >> A >> quick fix would be to set a flag indicating that future writes shouldn't >> bother trying to find a window that large, > > There are lots of quick fixes. 1) The quickest is to not scan for the > window at all. 2) The second quickest is to limit the number of bitmaps > that will be scanned to some number like 3. 3) The not at all quickest > is to track free extents like XFS does, which is not a hack, but it > belongs in a development branch. I am not sure it is worth the > complexity, but my mind is not closed. > > On monday we will do 1) or 2), probably 1). After the repacker is > done, we should review all our block allocation algorithms. I have an > idea for how to do things more optimally for streaming media that will > avoid fragmentation over time, and when combined with the repacker may > make 3 not worthwhile. If you want to go the 1) route, it's trivial. See patch below. It will restore the one-block-at-a-time behavior. > I am grateful that you and Chris do bug fixes, but when you guys are too > busy, (and that can and will happen to any of us), the baton needs to > get passed. V3 needs to be a zero defect product, and once we know it > is a bug I don't want bugs in V3 to remain unfixed for more than a day > plus the time it takes to fix it. If you do add code, I want any bugs > that show up in the aftermath of mainstream merging to get jumped on. Anyone up for it? :) There are changes I'd like to see in reiser3, particularly ones that address the severe problems observed in David Chinner's high bandwidth file system talk this year at OLS. Specifically, it ended up making very little progress and spending the majority of the time in the journal when the workload is streaming data at the disk at a very high rate on a very large file system. Yes, that is certainly XFS's sweet spot, but barely making progress at all is a bit more severe than "poor performance." Perhaps mkreiserfs should be a bit saner about choosing journal sizes, since a 32 MB journal is not a good fit for all cases. Also, I'd like to see the usage of the BKL gone as it severely limits performance when more than one thread is writing to the file system, or even another reiserfs file system. It's not entirely low hanging fruit since the nested cases need to be audited, but it shouldn't be too hard to eliminate the inter-filesystem lock contention by replacing the BKL with a per-sb mutex. I have some more things, but I have nowhere near the time to do them, and other file systems will perform fine. - -Jeff Patch: - --- linux-2.6.17.orig/fs/reiserfs/bitmap.c 2006-01-02 22:21:10.000000000 -0500 +++ linux-2.6.17.orig.devel/fs/reiserfs/bitmap.c 2006-07-23 19:10:57.000000000 -0400 @@ -1020,7 +1020,6 @@ b_blocknr_t finish = SB_BLOCK_COUNT(s) - 1; int passno = 0; int nr_allocated = 0; - - int bigalloc = 0; determine_prealloc_size(hint); if (!hint->formatted_node) { @@ -1047,28 +1046,9 @@ hint->preallocate = hint->prealloc_size = 0; } /* for unformatted nodes, force large allocations */ - - bigalloc = amount_needed; } do { - - /* in bigalloc mode, nr_allocated should stay zero until - - * the entire allocation is filled - - */ - - if (unlikely(bigalloc && nr_allocated)) { - - reiserfs_warning(s, "bigalloc is %d, nr_allocated %d\n", - - bigalloc, nr_allocated); - - /* reset things to a sane value */ - - bigalloc = amount_needed - nr_allocated; - - } - - /* - - * try pass 0 and pass 1 looking for a nice big - - * contiguous allocation. Then reset and look - - * for anything you can find. - - */ - - if (passno == 2 && bigalloc) { - - passno = 0; - - bigalloc = 0; - - } switch (passno++) { case 0: /* Search from hint->search_start to end of disk */ start = hint->search_start; @@ -1106,8 +1086,7 @@ new_blocknrs + nr_allocated, start, finish, - - bigalloc ? - - bigalloc : 1, + 1, amount_needed - nr_allocated, hint-> - -- Jeff Mahoney SUSE Labs -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFExATJLPWxlyuTD7IRAmeKAJsFI/awPPAXpB2DI+kO19EZtr3tRwCfWduO Re+5kXNtj6St/LuUy9lbNm4= =anQd -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-23 23:22 ` Jeff Mahoney @ 2006-07-23 22:35 ` Hans Reiser 0 siblings, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-23 22:35 UTC (permalink / raw) To: Jeff Mahoney; +Cc: Jeff Garzik, Theodore Tso, LKML, ReiserFS List Jeff Mahoney wrote: > > > Anyone up for it? :) There are changes I'd like to see in reiser3, > particularly ones that address the severe problems observed in David > Chinner's high bandwidth file system talk this year at OLS. Specifically, > it ended up making very little progress and spending the majority of the > time in the journal when the workload is streaming data at the disk at a > very high rate on a very large file system. Yes, that is certainly XFS's > sweet spot, but barely making progress at all is a bit more severe than > "poor performance." Perhaps mkreiserfs should be a bit saner about > choosing > journal sizes, since a 32 MB journal is not a good fit for all cases. > Also, > I'd like to see the usage of the BKL gone as it severely limits > performance > when more than one thread is writing to the file system, or even another > reiserfs file system. It's not entirely low hanging fruit since the nested > cases need to be audited, but it shouldn't be too hard to eliminate the > inter-filesystem lock contention by replacing the BKL with a per-sb mutex. Getting rid of the BKL is a huge task that was done in V4 for a reason. You are talking about 6+ man-months, and years of shake-out to fully debug. Actually, it is a tribute to Zam's skill that V4's locking got debugged so fast: I gave him the task knowing it was going to be the hardest code to debug, and he did it very well. These things you discuss, except for the journal size, are not things to fix in a stable branch. My apologies that I thought this was a new bug. Let us be glad that a user gave us enough detail we saw it. > I have some more things, but I have nowhere near the time to do them, > and other file systems will perform fine. > > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-23 7:20 ` Hans Reiser ` (2 preceding siblings ...) 2006-07-23 20:46 ` Jeff Mahoney @ 2006-07-25 19:13 ` Russell Cattelan 2006-07-25 23:51 ` David Masover 2006-07-26 0:44 ` David Masover 2006-07-26 10:38 ` David Weinehall 5 siblings, 1 reply; 639+ messages in thread From: Russell Cattelan @ 2006-07-25 19:13 UTC (permalink / raw) To: Hans Reiser; +Cc: Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Sun, 2006-07-23 at 01:20 -0600, Hans Reiser wrote: > Jeff, I think that a large part of what is going on is that any patch > that can be read in 15 minutes gets reviewed immediately, and any patch > that is worked on for 5 years and then takes a week to read gets > neglected. This is true even if line for line the 1 week to read patch > is more valuable. What is more is that people know this is > irrational, but aren't able to cure it in themselves. Even I have a > problem of paying too much attention to endless 5 minute emails when I > know I should instead, say, read the compression plugin from beginning > to end. > > There is nothing about small patches that makes them better code. There > is no reason we should favor them, if the developers are willing to work > on something for 5 years to escape a local optimum, that is often the > RIGHT thing to do. > > It is importand that we embrace our diversity, and be happy for the > strength it gives us. Some of us are good at small patches that evolve, > and some are good at escaping local optimums. We all have value, both > trees and grass have their place in the world. > Which is summed up quite well by: http://en.wikipedia.org/wiki/Color_of_the_bikeshed It seem to be a well know tendency for people to want to be involved in some way, thus keeping to much of the development cycle internal tends to generate friction. -Russell Cattelan cattelan@xfs.org ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 19:13 ` Russell Cattelan @ 2006-07-25 23:51 ` David Masover 2006-07-26 14:26 ` Hans Reiser 0 siblings, 1 reply; 639+ messages in thread From: David Masover @ 2006-07-25 23:51 UTC (permalink / raw) To: Russell Cattelan Cc: Hans Reiser, Jeff Garzik, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 3367 bytes --] Russell Cattelan wrote: > On Sun, 2006-07-23 at 01:20 -0600, Hans Reiser wrote: >> Jeff, I think that a large part of what is going on is that any patch >> that can be read in 15 minutes gets reviewed immediately, and any patch >> that is worked on for 5 years and then takes a week to read gets [...] >> It is importand that we embrace our diversity, and be happy for the >> strength it gives us. Some of us are good at small patches that evolve, >> and some are good at escaping local optimums. We all have value, both >> trees and grass have their place in the world. >> > Which is summed up quite well by: > http://en.wikipedia.org/wiki/Color_of_the_bikeshed > > It seem to be a well know tendency for people to want to > be involved in some way, thus keeping to much of the development > cycle internal tends to generate friction. No, I think Hans is right. Although I should mention, Hans, that there is a really good reason to prefer the 15 minute patches. The patches that take a week are much harder to read during that week than any number of 15 minute incremental patches, because the incremental patches are already broken down into nice, small, readable, ordered chunks. And since development follows some sort of logical, orderly pattern, it can be much easier to read it that way than to try to consider the whole. Think of it this way -- why are debuggers useful? One of the nicest thing about a debugger, especially for newbies, is the ability to step through a program a line at a time. It's the same principle -- you can understand the program state at one point in time, and the impact of one line of code, much more easily than the overall model of the program state (and all of its edge cases), or the impact of several hundred (thousand? tens of thousands?) lines of code. So while I don't blame the Namesys team for putting off inclusion till it's done, I also can't really blame the kernel guys for not wanting to read it, especially if it's revolutionary. Revolutionary ideas are hard to grasp, and it's not their fault. I mean, if revolutionary ideas were easy, why didn't you write Reiser4 for a system like, say, Tunes? (http://tunes.org/) Say what you will, but there are ways of doing fast filesystems which don't require that said filesystems be written in kernel C. Consider this: http://www.cs.utah.edu/flux/oskit/ If I understand that right, it's a mechanism for writing kernel code in languages like "Java, Lisp, Scheme, or ML"... If we could all grasp every single good (best) idea from every corner of software engineering, and write completely new software (including the OS) using those ideas, we could potentially replace all existing software in something like 3-5 years with software which has none of the problems ours does now. We'd never have inflexibility, insecurity, instability, user interface issues... Never have to worry about getting software out the door (it'd be so fast to develop), but always have it designed the right way the first time, yet be able to rearrange it completely with only 5-10 line patches. So it's not always the computer hardware that's the limitation. Often it's our hardware as well. Human beings usually aren't equipped to be able to grok the whole universe all at once. If we were... see above. [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 890 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 23:51 ` David Masover @ 2006-07-26 14:26 ` Hans Reiser 2006-07-26 18:16 ` Russell Cattelan 0 siblings, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-07-26 14:26 UTC (permalink / raw) To: David Masover Cc: Russell Cattelan, Jeff Garzik, Theodore Tso, LKML, ReiserFS List David Masover wrote: > > >Although I should mention, Hans, that there is a really good reason to >prefer the 15 minute patches. The patches that take a week are much >harder to read during that week than any number of 15 minute incremental >patches, because the incremental patches are already broken down into >nice, small, readable, ordered chunks. And since development follows >some sort of logical, orderly pattern, it can be much easier to read it >that way than to try to consider the whole. > > No, I disagree, if the code is well commented, it is easier to read the whole thing at the end when it has its greatest coherence and refinement. A problem with Reiser4 is that its core algorithms are simply complex. We pushed the envelope in multiple areas all at once. Benchmarks don't always suggest simple algorithms are the ones that will be highest performance. Tree algorithms are notorious in the database industry for being simple on web pages but complex as code. Some people program in small increments, some program things that require big increments of change, both kinds of people are needed. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 14:26 ` Hans Reiser @ 2006-07-26 18:16 ` Russell Cattelan 2006-07-27 1:19 ` David Masover 0 siblings, 1 reply; 639+ messages in thread From: Russell Cattelan @ 2006-07-26 18:16 UTC (permalink / raw) To: Hans Reiser; +Cc: David Masover, Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Wed, 2006-07-26 at 08:26 -0600, Hans Reiser wrote: > David Masover wrote: > > > > > > >Although I should mention, Hans, that there is a really good reason to > >prefer the 15 minute patches. The patches that take a week are much > >harder to read during that week than any number of 15 minute incremental > >patches, because the incremental patches are already broken down into > >nice, small, readable, ordered chunks. And since development follows > >some sort of logical, orderly pattern, it can be much easier to read it > >that way than to try to consider the whole. > > > > > No, I disagree, if the code is well commented, it is easier to read the > whole thing at the end when it has its greatest coherence and > refinement. A problem with Reiser4 is that its core algorithms are > simply complex. We pushed the envelope in multiple areas all at once. > Benchmarks don't always suggest simple algorithms are the ones that will > be highest performance. Tree algorithms are notorious in the database > industry for being simple on web pages but complex as code. > > Some people program in small increments, some program things that > require big increments of change, both kinds of people are needed. > FWIW I think both points are valid. Personally I find it difficult to completely stop what I'm doing and spend several days studying a large patch/complete new subsystem. But on the other hand it's important that code that goes in the kernel gets a proper review, I think we all come to rely on the gatekeepers job of making sure any given part of the kernel does not destabilize to the point were it interferes with our individual areas of development. So exactly what level of granularity is appropriate for changes? Well that should probably be left of to the gatekeeper for each particular area. In the case of filesystems generic vfs changes obviously need to be small and easy to digest, and more importantly easy to bisect regressions. The core of the file system can be left to the person who can best manage the code, but hopefully that person applies a reasonable of granularity to the changes, thus allowing non familiar developers to at least keep up and possibly make helpful suggestions. If we look at the current "beliefs" surrounding XFS you can see the affects of a code base that did not have an incremental development with regards to linux anyways. Ok so ya XFS was a close sourced IRIX FS for the first 8 years of it's life, and even once the Linux project started there was another year or so encumbrance investigation and cleaning before legal would sign off on its release. So to this day the major hang up with XFS seems to be "it's to complex because it has x thousands lines of code". Hell by that argument Linux has way more lines of code than IRIX could ever hope to have and is therefore Linux is more complex than IRIX and to hard to understand. :-) Fortunately many developers (especially ones that have worked on other OS's) do not use "wc -l" as a tool to measure code quality/readability/complexity. XFS unfortunately will probably always suffer from skepticism since it did not grow up in in Linux. Guess it's sort of like adopting a 8 year old child vs a new born, hard to tell what happened in first 8 years. -Russell Cattelan cattelan@xfs.org ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 18:16 ` Russell Cattelan @ 2006-07-27 1:19 ` David Masover 2006-07-27 20:11 ` Maciej Sołtysiak 0 siblings, 1 reply; 639+ messages in thread From: David Masover @ 2006-07-27 1:19 UTC (permalink / raw) To: Russell Cattelan Cc: Hans Reiser, Jeff Garzik, Theodore Tso, LKML, ReiserFS List Russell Cattelan wrote: > Guess it's sort of like adopting a 8 year old child vs a new born, hard > to tell what happened in first 8 years. And, by that token, the newborn is sometimes preferable. It hasn't had time to develop severe emotional problems, it's physically harmless, and you get to help it form its ideas and beliefs. On the other hand, the 8 year old is potty trained, you won't have to change a single diaper, it's intelligent and makes you question things, it understands what you want it to do... So, it depends what kind of developer you are, what kind of gatekeeper, and what kind of project it is. I still believe in releasing early and often, but I can see many reasons not to. Some are financial -- Namesys is trying to operate a business, so any features they open up too early could be that much harder to sell as a commercial product. The repacker, for instance. I'm not arguing for closed source, I'm just saying that once you open, there's no going back. Many times it's a good thing, but sometimes you want to wait and see. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 1:19 ` David Masover @ 2006-07-27 20:11 ` Maciej Sołtysiak 2006-07-27 20:23 ` David Masover 0 siblings, 1 reply; 639+ messages in thread From: Maciej Sołtysiak @ 2006-07-27 20:11 UTC (permalink / raw) To: ReiserFS List Hello David, Thursday, July 27, 2006, 3:19:15 AM, you wrote: > I'm not arguing for closed source, I'm just saying that once you open, > there's no going back. Many times it's a good thing, but sometimes you A sidenote. Reiser4 is open and still we don't see people writing plugins as crazy. I belive there is one group that tried to be the first outside of namesys to write a plugin but still no success. Regards, Maciej ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 20:11 ` Maciej Sołtysiak @ 2006-07-27 20:23 ` David Masover 0 siblings, 0 replies; 639+ messages in thread From: David Masover @ 2006-07-27 20:23 UTC (permalink / raw) To: Maciej Sołtysiak; +Cc: ReiserFS List Maciej SoÅ‚tysiak wrote: > Hello David, > > Thursday, July 27, 2006, 3:19:15 AM, you wrote: >> I'm not arguing for closed source, I'm just saying that once you open, >> there's no going back. Many times it's a good thing, but sometimes you > A sidenote. > > Reiser4 is open and still we don't see people writing plugins as crazy. > I belive there is one group that tried to be the first outside of > namesys to write a plugin but still no success. Kernel inclusion would help a lot. Decent documentation would be better, though. Someone should look at what FUSE is doing right. Plugins fill a lot of the same niches, but with significantly better performance. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-23 7:20 ` Hans Reiser ` (3 preceding siblings ...) 2006-07-25 19:13 ` Russell Cattelan @ 2006-07-26 0:44 ` David Masover 2006-07-26 7:59 ` the ' 'official' point of view' " Luigi Genoni 2006-07-26 14:33 ` the " 'official' point of view" " Hans Reiser 2006-07-26 10:38 ` David Weinehall 5 siblings, 2 replies; 639+ messages in thread From: David Masover @ 2006-07-26 0:44 UTC (permalink / raw) To: Hans Reiser; +Cc: Jeff Garzik, Theodore Tso, LKML, ReiserFS List [-- Attachment #1: Type: text/plain, Size: 1123 bytes --] Hans Reiser wrote: > to use as his default. Now that we paid the 5 year development price > tag to get everything as plugins, we can now upgrade in littler pieces > than any other FS. Hmm, I need a buzz phrase, its not extreme > programming, maybe "moderate programming". Does that sound exciting to Hah! No, it doesn't sound exciting. Plugins don't work well either, not as a marketing concept. People have had so many bad experiences with plugins, and they're only ever visible when you have a bad experience. Think about it -- missing plugin (so you have to download it), On the other hand, it works for WordPress. My day job is work on a plugin for WordPress. Not including a link because I feel dirty for having to work with PHP... Fluid programming? If you build a solution from the bottom up with gravel or large rocks, you leave gaps that are hard to fill without ripping off the top layer and redoing it. But if you can do fluid programming, your program just flows around any obstacle, and into every crack / between every space (metaphor for new customer requirements)... [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 890 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 0:44 ` David Masover @ 2006-07-26 7:59 ` Luigi Genoni 2006-07-26 14:33 ` the " 'official' point of view" " Hans Reiser 1 sibling, 0 replies; 639+ messages in thread From: Luigi Genoni @ 2006-07-26 7:59 UTC (permalink / raw) To: David Masover; +Cc: Hans Reiser, Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Wed, July 26, 2006 02:44, David Masover wrote: > Hans Reiser wrote: > > >> to use as his default. Now that we paid the 5 year development price tag >> to get everything as plugins, we can now upgrade in littler pieces than >> any other FS. Hmm, I need a buzz phrase, its not extreme programming, >> maybe "moderate programming". Does that sound exciting to > > Hah! No, it doesn't sound exciting. > > > Plugins don't work well either, not as a marketing concept. People have > had so many bad experiences with plugins, and they're only ever visible when > you have a bad experience. Think about it -- missing plugin (so you have to > download it), > marketing? </joke mode on> if you do not like the word "plugin" why don't you suggest some alternative? like "modules"? </joke mode off> Seriously, please leave out this kind of marketing. The plugin concept in reiser4 is probably the most interessant feature I filesystem some users could need. > > Fluid programming? If you build a solution from the bottom up with > gravel or large rocks, you leave gaps that are hard to fill without ripping > off the top layer and redoing it. But if you can do fluid programming, your > program just flows around any obstacle, and into every crack / between every > space (metaphor for new customer requirements)... probably I do not know english enought well to appraciate this metaphor and to understand what it means. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 0:44 ` David Masover 2006-07-26 7:59 ` the ' 'official' point of view' " Luigi Genoni @ 2006-07-26 14:33 ` Hans Reiser 1 sibling, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-26 14:33 UTC (permalink / raw) To: David Masover; +Cc: Jeff Garzik, Theodore Tso, LKML, ReiserFS List David Masover wrote: >Hans Reiser wrote: > > > >>to use as his default. Now that we paid the 5 year development price >>tag to get everything as plugins, we can now upgrade in littler pieces >>than any other FS. Hmm, I need a buzz phrase, its not extreme >>programming, maybe "moderate programming". >> This phrase was a bit tongue-in-cheek. >> Does that sound exciting to >> >> > > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-23 7:20 ` Hans Reiser ` (4 preceding siblings ...) 2006-07-26 0:44 ` David Masover @ 2006-07-26 10:38 ` David Weinehall 5 siblings, 0 replies; 639+ messages in thread From: David Weinehall @ 2006-07-26 10:38 UTC (permalink / raw) To: Hans Reiser; +Cc: Jeff Garzik, Theodore Tso, LKML, ReiserFS List On Sun, Jul 23, 2006 at 01:20:40AM -0600, Hans Reiser wrote: > Jeff, I think that a large part of what is going on is that any patch > that can be read in 15 minutes gets reviewed immediately, and any patch > that is worked on for 5 years and then takes a week to read gets > neglected. This is true even if line for line the 1 week to read patch > is more valuable. What is more is that people know this is > irrational, but aren't able to cure it in themselves. Even I have a > problem of paying too much attention to endless 5 minute emails when I > know I should instead, say, read the compression plugin from beginning > to end. Well, the problem is that someone actually went through the effort of doing the week-long reviews of your code, you flamed him every time he suggested that there was need for improvement, instead of saying thanks for all the hard work and implementing the needed fixes. Not exactly the right way to make people fond of reviewing your code. Getting flamed by someone having a hard time taking criticism after spending 5 minutes to review a patch is bearable, after all, what's 5 minutes, right? Flaming someone that has put down a week or two of hard work on review is just disrespectful. [snip] Regards: David Weinehall -- /) David Weinehall <tao@acc.umu.se> /) Northern lights wander (\ // Maintainer of the v2.0 kernel // Dance across the winter sky // \) http://www.acc.umu.se/~tao/ (/ Full colour fire (/ ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-22 20:29 ` Jeff Garzik 2006-07-23 7:20 ` Hans Reiser @ 2006-07-24 8:41 ` Matthias Andree 2006-07-24 8:09 ` Hans Reiser 2006-07-24 13:49 ` Horst H. von Brand 1 sibling, 2 replies; 639+ messages in thread From: Matthias Andree @ 2006-07-24 8:41 UTC (permalink / raw) To: Jeff Garzik; +Cc: Hans Reiser, Theodore Tso, LKML On Sat, 22 Jul 2006, Jeff Garzik wrote: > Anyone who fails to respect the kernel development process, the process > of building consensus, is turn not respected, flamed, and/or ignored. That reminds me of the old "layer 8 and 9" extensions to the OSI/ISO layering model. Layer 8: financial, Layer 9: policital. SCNR. > With you in particular, you demonstrated NO interest in maintaining > reiser3, once reiser4 began to make a splash. Linux kernel code exists > for DECADES, and as such, long term maintenance is a CRITICAL aspect of > development. > > Regardless of whatever new whiz-bang technology exists in reiser4, there > is a very real worry that you will abandon reiser4 once its in the tree > for a few years, just like what happened with reiser3. The most worrying point was that reiser3 maintenance was given up at the point where it was just about to transition from usable to mature. -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 8:41 ` Matthias Andree @ 2006-07-24 8:09 ` Hans Reiser 2006-07-24 10:32 ` Matthias Andree 2006-07-24 13:49 ` Horst H. von Brand 1 sibling, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-07-24 8:09 UTC (permalink / raw) To: Matthias Andree; +Cc: Jeff Garzik, Theodore Tso, LKML Matthias Andree wrote: >The most worrying point was that reiser3 maintenance was given up at the >point where it was just about to transition from usable to mature. > > > Name a bug (not a feature) that has not been fixed by us, and which is not also so deep that it reasonably required waiting for a major release to fix it (for example, bug fixes that require disk format changes belong in v4 not v3 even if a case can be made that they are bug fixes). I mean, god, sometimes I think users are like little children waiting for the pie that is in the oven and who want to take it out now before it finishes cooking so they can eat it, and they are very angry about it, and I should just understand that and not try to reason with it (but also not give them the pie before it finishes cooking either). Someone please tell me I don't understand the users and it all makes more sense than that, please.... No code before its time. No features in stable branches. Wait for it. Stop complaining about how you are abandoned, we are working hard. It's going to be the best pie ever. Wait for it. Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 8:09 ` Hans Reiser @ 2006-07-24 10:32 ` Matthias Andree 0 siblings, 0 replies; 639+ messages in thread From: Matthias Andree @ 2006-07-24 10:32 UTC (permalink / raw) To: Hans Reiser; +Cc: Matthias Andree, Jeff Garzik, Theodore Tso, LKML On Mon, 24 Jul 2006, Hans Reiser wrote: > I mean, god, sometimes I think users are like little children waiting > for the pie that is in the oven and who want to take it out now before > it finishes cooking so they can eat it, and they are very angry about > it, and I should just understand that and not try to reason with it (but > also not give them the pie before it finishes cooking either). Someone > please tell me I don't understand the users and it all makes more sense > than that, please.... namesys.com doesn't list reiserfs 3.6 hash collision limits in an easy to find place (which would be the same place that boasts about 100,000 files per directory if it wants to be honest). > No code before its time. No features in stable branches. Wait for it. > Stop complaining about how you are abandoned, we are working hard. It's > going to be the best pie ever. Wait for it. I'm not going to eat it while it's still steaming and fogging my glasses. You're now making the same noise about how good reiser4 is that was made when reiserfs 3.5 was a patch for Linux 2.2 and that wedged local access when NFS exported, and 3.6 didn't fix the hash collision issue (but required a format change, too, right). -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 8:41 ` Matthias Andree 2006-07-24 8:09 ` Hans Reiser @ 2006-07-24 13:49 ` Horst H. von Brand 1 sibling, 0 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-07-24 13:49 UTC (permalink / raw) To: Jeff Garzik, Hans Reiser, Theodore Tso, LKML Matthias Andree <matthias.andree@gmx.de> wrote: > On Sat, 22 Jul 2006, Jeff Garzik wrote: > > > Anyone who fails to respect the kernel development process, the process > > of building consensus, is turn not respected, flamed, and/or ignored. > > That reminds me of the old "layer 8 and 9" extensions to the OSI/ISO > layering model. Layer 8: financial, Layer 9: policital. Got that one wrong. Layer 8: User Layer 9: Political Layer 10: Financial -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-22 13:02 ` Theodore Tso 2006-07-22 14:30 ` Eric Sandeen 2006-07-22 18:33 ` Hans Reiser @ 2006-07-23 7:20 ` Rene Rebe 2006-07-24 7:49 ` Nikita Danilov 2006-07-24 2:08 ` Steve Lord 3 siblings, 1 reply; 639+ messages in thread From: Rene Rebe @ 2006-07-23 7:20 UTC (permalink / raw) To: LKML; +Cc: Hans Reiser Hi, On Saturday 22 July 2006 15:02, Theodore Tso wrote: > > The code isn't even written, benchmarked, or tested yet, > > Actually, the first bits that we plan to merge have already been > written and in use by hundreds of clusterfs customers, posted to LKML > for comments (and we don't attack our reviewers, we thank them for > their comments), and in fact they were written about at last year's > OLS complete with benchmarks and graphs. > (http://ext2.sourceforge.net/2005-ols/2005-ols-ext3.html) However I would estimate that Reiser4 is used by more people than yet aother ext2 patchups. Yours, -- René Rebe - Rubensstr. 64 - 12157 Berlin (Europe / Germany) http://exactcode.de | http://t2-project.org | http://rebe.name +49 (0)30 / 255 897 45 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-23 7:20 ` Rene Rebe @ 2006-07-24 7:49 ` Nikita Danilov 2006-07-25 12:35 ` Andrea Arcangeli 0 siblings, 1 reply; 639+ messages in thread From: Nikita Danilov @ 2006-07-24 7:49 UTC (permalink / raw) To: Rene Rebe; +Cc: Hans Reiser, Linux Kernel Mailing List Rene Rebe writes: > Hi, > > On Saturday 22 July 2006 15:02, Theodore Tso wrote: > > > The code isn't even written, benchmarked, or tested yet, > > > > Actually, the first bits that we plan to merge have already been > > written and in use by hundreds of clusterfs customers, posted to LKML > > for comments (and we don't attack our reviewers, we thank them for > > their comments), and in fact they were written about at last year's > > OLS complete with benchmarks and graphs. > > (http://ext2.sourceforge.net/2005-ols/2005-ols-ext3.html) > > However I would estimate that Reiser4 is used by more people than yet > aother ext2 patchups. Any data backing up that estimation? Just to give an example, that "patchup" is used by (tens of) thousands of computers in governmental laboratories the US "national security" depends upon. > > Yours, Nikita. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 7:49 ` Nikita Danilov @ 2006-07-25 12:35 ` Andrea Arcangeli 2006-07-25 12:51 ` Rene Rebe 2006-07-25 17:47 ` Hans Reiser 0 siblings, 2 replies; 639+ messages in thread From: Andrea Arcangeli @ 2006-07-25 12:35 UTC (permalink / raw) To: Nikita Danilov; +Cc: Rene Rebe, Hans Reiser, Linux Kernel Mailing List I'm no reiserfs user (nor v3 nor v4), but I respect views of people that needs the fs to handle zillons of tiny files, for that reiser3 did a good job, and my humble opinion is that the only chance for reiser4 to stabilize is to grow the userbase and including it in mainline or in some distro would obviously help (like it happened to reiser3 in the past with SUSE inclusion first and mainline inclusion later, nothing has changed dramatically since then, reiser3 was very buggy at the time too, it was probably my backport of lfs that made us notice it couldn't even handle lfs on-disk so they had to change the fs format later on with some backwards compatibility mess with older kernels, hopefully this time the fixage of v4 for production usage won't require a change of the on-disk format). On Mon, Jul 24, 2006 at 11:49:43AM +0400, Nikita Danilov wrote: > Any data backing up that estimation? Just to give an example, that Probably KLive is the only tool that can provide an estimate of the part of the reiser4 userbase compared to the other fs. reiser4 had so far 1926 days of total uptime and 4409 users using it at least once for root or anyway mounted nearly at boot. Avg uptime is 16 hours, max uptime is 28 days. I can also provide the exact storage scsi/SATA/ATA chipsets used in combination with it, but it doesn't seem necessary. klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where f = 'reiser4' group by f order by sum(uptime) desc; f | count | sum | avg | max ----------------------------------+-------+------------------------------+-----------------+------------------ reiser4 | 4409 | 1926 days 27479:22:27.459534 | 16:42:59.666015 | 28 days 16:16:15 (1 row) klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where f = 'ext4' group by f order by sum(uptime) desc; f | count | sum | avg | max ---+-------+-----+-----+----- (0 rows) klive=> Full data of all fs (all kernels not just mainline ones) follows. In total uptime and avg uptime terms it beats many other common fs like isofs, autofs and ramfs. It'd be cool if I'd be able to receive a packet during sigterm so I could then differentiate sessions that rebooted cleanly from the ones that didn't. Receiving oopses (optionally encrypted) is also a long term planned feature. At the moment I can't (and I've no much time to work on klive), but this is still good to quantify the current userbase compared to the other more common fs. klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive group by f order by sum(uptime) desc; f | count | sum | avg | max ----------------------------------+-------+--------------------------------+-------------------------+------------------------- sysfs | 40998 | 35138 days 259797:16:29.93576 | 26:54:23.100394 | 173 days 16:16:15 tmpfs | 40507 | 35025 days 257298:57:48.690973 | 27:06:14.154311 | 173 days 16:16:15 usbfs | 38803 | 26681 days 241169:00:36.268676 | 22:43:03.543444 | 173 days 16:16:15 reiserfs | 19934 | 19257 days 133294:48:27.256508 | 29:52:18.181361 | 155 days 16:16:15 ext3 | 22077 | 16346 days 132190:42:38.862752 | 23:45:27.062502 | 173 days 16:16:15 ext2 | 9396 | 5629 days 63099:50:27.500173 | 21:05:37.103821 | 139 days 14:54:11 nfs | 2824 | 6373 days 23368:45:22.667638 | 2 days 14:26:11.502361 | 86 days 16:16:15 binfmt_misc | 4396 | 5421 days 33069:20:47.871638 | 1 day 13:07:06.944466 | 173 days 16:16:15 nfsd | 2231 | 4727 days 19824:00:00.354481 | 2 days 11:44:11.187967 | 89 days 16:16:15 xfs | 4326 | 4067 days 28000:38:11.762818 | 29:02:08.685105 | 139 days 14:54:11 vfat | 8684 | 2470 days 46585:39:49.859957 | 12:11:27.193673 | 68 days 16:16:15 ntfs | 8482 | 2382 days 47344:00:52.902741 | 12:19:17.846369 | 68 days 16:16:15 rpc_pipefs | 1188 | 3078 days 11297:55:12.600343 | 2 days 23:41:30.667172 | 173 days 16:16:15 reiser4 | 4409 | 1926 days 27479:22:27.459534 | 16:42:59.666015 | 28 days 16:16:15 ^^^^^^^^^^^^^ autofs | 2030 | 2374 days 14512:14:16.617646 | 1 day 11:12:57.170748 | 54 days 16:16:15 ramfs | 2847 | 817 days 14809:30:42.122875 | 12:05:20.562741 | 32 days 16:16:15 selinuxfs | 600 | 1006 days 3466:41:58.91087 | 1 day 22:01:04.198185 | 173 days 16:16:15 smbfs | 222 | 1053 days 2155:11:48.374153 | 4 days 27:32:45.353037 | 122 days 16:20:01 jfs | 370 | 971 days 2435:07:17.805929 | 2 days 21:33:54.696773 | 160 days 16:16:15 iso9660 | 651 | 792 days 4698:41:27.153665 | 1 day 12:24:56.90807 | 84 days 16:06:15 subfs | 611 | 736 days 3482:38:06.342656 | 1 day 10:36:35.558662 | 122 days 16:20:01 devfs | 51 | 713 days 543:13:20.51142 | 13 days 34:10:50.99042 | 155 days 16:16:15 supermount | 260 | 437 days 1281:43:13.500606 | 1 day 21:16:05.359618 | 68 days 16:16:15 cifs | 506 | 222 days 2997:17:59.130448 | 16:27:11.381681 | 35 days 16:16:15 openpromfs | 53 | 296 days 631:17:42.10027 | 5 days 25:56:56.266043 | 43 days 16:16:15 fuse | 390 | 163 days 2443:31:37.846769 | 16:17:46.404735 | 17 days 16:16:15 squashfs | 62 | 198 days 622:55:15.099784 | 3 days 14:41:32.179029 | 34 days 16:16:15 configfs | 45 | 160 days 504:05:40.811143 | 3 days 24:32:07.573581 | 23 days 16:16:15 capifs | 67 | 135 days 569:04:11.286495 | 2 days 08:51:06.437112 | 39 days 16:16:15 minix | 35 | 128 days 428:57:59 | 3 days 28:01:39.40 | 16 days 16:16:15 cramfs | 6 | 140 days 69:58:03.269142 | 23 days 19:39:40.544857 | 58 days 16:16:15 udf | 15 | 102 days 161:35:54.111551 | 6 days 29:58:23.607437 | 51 days 16:16:15 debugfs | 314 | 39 days 1144:50:09.253475 | 06:37:36.717368 | 11 days 16:14:23.049312 ufs | 146 | 24 days 884:40:35.839055 | 10:00:16.683829 | 10 days 16:16:11.062917 usbdevfs | 139 | 1076:04:34.054012 | 07:44:29.597511 | 19:39:58 shm | 139 | 1076:04:34.054012 | 07:44:29.597511 | 19:39:58 hfsplus | 80 | 14 days 480:38:46.186027 | 10:12:29.077325 | 2 days 23:36:48 unionfs | 23 | 24 days 129:38:06 | 1 day 06:40:47.217391 | 22 days 16:16:15 ocfs2_dlmfs | 1 | 23 days 16:16:15 | 23 days 16:16:15 | 23 days 16:16:15 oprofilefs | 1 | 23 days 16:16:08.088967 | 23 days 16:16:08.088967 | 23 days 16:16:08.088967 nfs4 | 5 | 20 days 71:36:00 | 4 days 14:19:12 | 12 days 16:16:15 nnpfs | 9 | 18 days 78:16:50 | 2 days 08:41:52.222222 | 8 days 16:16:15 afs | 1 | 19 days 16:16:15 | 19 days 16:16:15 | 19 days 16:16:15 securityfs | 9 | 4 days 22:13:22 | 13:08:09.111111 | 4 days 16:16:15 ufsd | 15 | 75:38:57 | 05:02:35.80 | 23:01:05 vmware-hgfs | 1 | 1 day 04:56:21 | 1 day 04:56:21 | 1 day 04:56:21 shfs | 2 | 14:46:00 | 07:23:00 | 11:27:35 (47 rows) Now let's eliminate from the screen all sessions that rebooted (or crashed) in less than one day: klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where uptime > '1 day' group by f order by sum(uptime) desc; f | count | sum | avg | max ----------------------------------+-------+-------------------------------+-------------------------+------------------------- sysfs | 5385 | 35138 days 80491:02:11.763381 | 6 days 27:33:04.202742 | 173 days 16:16:15 tmpfs | 5344 | 35025 days 79890:55:31.804764 | 6 days 28:14:51.192329 | 173 days 16:16:15 usbfs | 4681 | 26681 days 69552:08:46.865054 | 5 days 31:39:17.301189 | 173 days 16:16:15 reiserfs | 2959 | 19257 days 44003:36:16.956207 | 6 days 27:03:42.161864 | 155 days 16:16:15 ext3 | 2520 | 16346 days 37501:47:54.627011 | 6 days 26:33:28.283582 | 173 days 16:16:15 nfs | 645 | 6373 days 10050:19:24.173861 | 9 days 36:43:00.409572 | 86 days 16:16:15 ext2 | 1012 | 5629 days 15173:13:30.06032 | 5 days 28:29:14.555396 | 139 days 14:54:11 binfmt_misc | 806 | 5421 days 12168:47:27.060755 | 6 days 32:31:01.59685 | 173 days 16:16:15 nfsd | 634 | 4727 days 9800:42:01.958542 | 7 days 26:23:54.892679 | 89 days 16:16:15 xfs | 607 | 4067 days 9311:21:11.733095 | 6 days 32:08:38.075343 | 139 days 14:54:11 rpc_pipefs | 445 | 3078 days 6789:18:13.704195 | 6 days 37:15:40.884729 | 173 days 16:16:15 vfat | 669 | 2470 days 9424:11:09.850314 | 3 days 30:41:48.624589 | 68 days 16:16:15 ntfs | 636 | 2382 days 9081:10:26.573867 | 3 days 32:09:55.324802 | 68 days 16:16:15 autofs | 405 | 2374 days 6335:32:22.311385 | 5 days 36:19:29.240275 | 54 days 16:16:15 reiser4 | 509 | 1926 days 7617:09:58.618041 | 3 days 33:46:41.961921 | 28 days 16:16:15 ^^^^^^^^^^ smbfs | 96 | 1053 days 1476:55:37.466481 | 10 days 38:38:04.765276 | 122 days 16:20:01 selinuxfs | 97 | 1006 days 1584:22:47.021437 | 10 days 25:14:27.701252 | 173 days 16:16:15 jfs | 71 | 971 days 1070:04:09.186991 | 13 days 31:17:48.298408 | 160 days 16:16:15 ramfs | 241 | 817 days 3394:18:10.874752 | 3 days 23:26:42.8667 | 32 days 16:16:15 iso9660 | 85 | 792 days 1321:54:28.689609 | 9 days 23:10:31.396348 | 84 days 16:06:15 subfs | 89 | 736 days 1346:31:28.394797 | 8 days 21:36:05.038144 | 122 days 16:20:01 devfs | 24 | 713 days 399:55:43.321687 | 29 days 33:39:49.30507 | 155 days 16:16:15 supermount | 34 | 437 days 466:58:54.119926 | 12 days 34:12:19.238821 | 68 days 16:16:15 openpromfs | 29 | 296 days 515:10:26.10027 | 10 days 22:43:48.486216 | 43 days 16:16:15 cifs | 39 | 222 days 622:53:46.261055 | 5 days 32:35:13.493873 | 35 days 16:16:15 squashfs | 23 | 198 days 410:38:26 | 8 days 32:27:45.478261 | 34 days 16:16:15 fuse | 36 | 163 days 534:27:25.006209 | 4 days 27:30:45.694617 | 17 days 16:16:15 configfs | 23 | 160 days 344:26:41.378742 | 6 days 37:55:56.581684 | 23 days 16:16:15 minix | 24 | 128 days 393:11:05 | 5 days 24:22:57.708333 | 16 days 16:16:15 cramfs | 4 | 140 days 66:29:40.119926 | 35 days 16:37:25.029981 | 58 days 16:16:15 capifs | 11 | 135 days 170:13:33 | 12 days 22:01:13.909091 | 39 days 16:16:15 udf | 4 | 102 days 64:26:53 | 25 days 28:06:43.25 | 51 days 16:16:15 debugfs | 11 | 39 days 182:23:03.211977 | 3 days 29:40:16.655634 | 11 days 16:14:23.049312 ufs | 11 | 24 days 151:36:11.289322 | 2 days 18:08:44.662666 | 10 days 16:16:11.062917 unionfs | 2 | 24 days 39:53:03 | 12 days 19:56:31.50 | 22 days 16:16:15 ocfs2_dlmfs | 1 | 23 days 16:16:15 | 23 days 16:16:15 | 23 days 16:16:15 oprofilefs | 1 | 23 days 16:16:08.088967 | 23 days 16:16:08.088967 | 23 days 16:16:08.088967 nfs4 | 4 | 20 days 62:33:56 | 5 days 15:38:29 | 12 days 16:16:15 nnpfs | 4 | 18 days 73:50:18 | 4 days 30:27:34.50 | 8 days 16:16:15 afs | 1 | 19 days 16:16:15 | 19 days 16:16:15 | 19 days 16:16:15 hfsplus | 11 | 14 days 109:35:09 | 1 day 16:30:28.090909 | 2 days 23:36:48 securityfs | 1 | 4 days 16:16:15 | 4 days 16:16:15 | 4 days 16:16:15 vmware-hgfs | 1 | 1 day 04:56:21 | 1 day 04:56:21 | 1 day 04:56:21 (43 rows) Now let's eliminate everything that rebooted or crashed in less than 10 days: klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where uptime > '10 day' group by f order by sum(uptime) desc; f | count | sum | avg | max ----------------------------------+-------+-------------------------------+-------------------------+------------------------- sysfs | 948 | 22969 days 15224:50:33.717548 | 24 days 21:33:13.073542 | 173 days 16:16:15 tmpfs | 949 | 22960 days 15240:45:20.729486 | 24 days 20:42:47.250505 | 173 days 16:16:15 usbfs | 689 | 16015 days 11048:34:25.54762 | 23 days 21:53:15.450722 | 173 days 16:16:15 reiserfs | 516 | 12330 days 8297:16:33.102261 | 23 days 37:34:06.110663 | 155 days 16:16:15 ext3 | 453 | 10925 days 7258:04:40.240741 | 24 days 18:49:48.698103 | 173 days 16:16:15 nfs | 193 | 4902 days 3138:54:10.459336 | 25 days 25:50:19.950567 | 86 days 16:16:15 binfmt_misc | 154 | 3581 days 2484:52:11.438999 | 23 days 22:12:48.385968 | 173 days 16:16:15 ext2 | 139 | 3359 days 2218:31:49.225427 | 24 days 19:55:54.742629 | 139 days 14:54:11 nfsd | 135 | 3226 days 2186:35:50.40609 | 23 days 37:42:29.262267 | 89 days 16:16:15 xfs | 116 | 2666 days 1848:31:20.130065 | 22 days 39:31:18.276983 | 139 days 14:54:11 rpc_pipefs | 84 | 2034 days 1356:31:00.312955 | 24 days 21:17:30.718011 | 173 days 16:16:15 autofs | 69 | 1366 days 1095:10:03.052435 | 19 days 35:00:08.73989 | 54 days 16:16:15 ntfs | 56 | 1058 days 905:01:16.41019 | 18 days 37:35:22.793039 | 68 days 16:16:15 vfat | 52 | 1060 days 846:41:12.168038 | 20 days 25:30:47.541693 | 68 days 16:16:15 reiser4 | 53 | 820 days 859:44:03.65926 | 15 days 27:32:31.767156 | 28 days 16:16:15 ^^^^^^^^^^^^ jfs | 21 | 837 days 336:58:57 | 39 days 36:37:05.571429 | 160 days 16:16:15 smbfs | 30 | 815 days 475:06:57.099919 | 27 days 19:50:13.903331 | 122 days 16:20:01 selinuxfs | 28 | 767 days 454:42:21.021437 | 27 days 25:40:05.03648 | 173 days 16:16:15 devfs | 14 | 686 days 233:05:03.154639 | 49 days 16:38:55.939617 | 155 days 16:16:15 iso9660 | 24 | 596 days 386:04:44.125293 | 24 days 36:05:11.838554 | 84 days 16:06:15 subfs | 20 | 542 days 306:50:12.359485 | 27 days 17:44:30.617974 | 122 days 16:20:01 supermount | 13 | 383 days 210:20:37.087554 | 29 days 27:15:25.929812 | 68 days 16:16:15 ramfs | 17 | 327 days 261:34:11.310952 | 19 days 21:02:00.66535 | 32 days 16:16:15 openpromfs | 13 | 238 days 211:24:12.08448 | 18 days 23:38:47.083422 | 43 days 16:16:15 squashfs | 8 | 142 days 130:10:00 | 17 days 34:16:15 | 34 days 16:16:15 cramfs | 3 | 137 days 48:48:44.087554 | 45 days 32:16:14.695851 | 58 days 16:16:15 cifs | 6 | 115 days 97:37:30 | 19 days 20:16:15 | 35 days 16:16:15 capifs | 4 | 116 days 65:05:00 | 29 days 16:16:15 | 39 days 16:16:15 configfs | 5 | 95 days 80:52:24.077093 | 19 days 16:10:28.815419 | 23 days 16:16:15 udf | 3 | 96 days 48:10:38 | 32 days 16:03:32.666667 | 51 days 16:16:15 fuse | 6 | 85 days 93:19:29 | 14 days 19:33:14.833333 | 17 days 16:16:15 minix | 3 | 42 days 48:48:45 | 14 days 16:16:15 | 16 days 16:16:15 ocfs2_dlmfs | 1 | 23 days 16:16:15 | 23 days 16:16:15 | 23 days 16:16:15 oprofilefs | 1 | 23 days 16:16:08.088967 | 23 days 16:16:08.088967 | 23 days 16:16:08.088967 unionfs | 1 | 22 days 16:16:15 | 22 days 16:16:15 | 22 days 16:16:15 afs | 1 | 19 days 16:16:15 | 19 days 16:16:15 | 19 days 16:16:15 nfs4 | 1 | 12 days 16:16:15 | 12 days 16:16:15 | 12 days 16:16:15 debugfs | 1 | 11 days 16:14:23.049312 | 11 days 16:14:23.049312 | 11 days 16:14:23.049312 ufs | 1 | 10 days 16:16:11.062917 | 10 days 16:16:11.062917 | 10 days 16:16:11.062917 (39 rows) klive=> Now let's eliminate everything that rebooted or crashed in less than 20 days: klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where uptime > '20 day' group by f order by sum(uptime) desc; f | count | sum | avg | max ----------------------------------+-------+------------------------------+-------------------------+------------------------- sysfs | 425 | 15940 days 6844:38:13.264145 | 37 days 28:14:46.337092 | 173 days 16:16:15 tmpfs | 425 | 15908 days 6844:19:50.221708 | 37 days 26:26:18.329933 | 173 days 16:16:15 usbfs | 266 | 10341 days 4258:02:54.770525 | 38 days 37:01:48.927709 | 173 days 16:16:15 reiserfs | 236 | 8553 days 3809:00:57.232965 | 36 days 21:56:11.428953 | 155 days 16:16:15 ext3 | 197 | 7465 days 3163:26:12.784865 | 37 days 37:29:58.846624 | 173 days 16:16:15 nfs | 127 | 4029 days 2066:09:04.104158 | 31 days 33:39:17.04019 | 86 days 16:16:15 binfmt_misc | 63 | 2318 days 1020:40:19.272961 | 36 days 35:14:55.544015 | 173 days 16:16:15 nfsd | 63 | 2243 days 1023:44:21.136625 | 35 days 30:43:33.668835 | 89 days 16:16:15 ext2 | 53 | 2220 days 827:52:02.580825 | 41 days 36:54:11.36945 | 139 days 14:54:11 xfs | 42 | 1665 days 662:12:10.297862 | 39 days 31:11:43.10233 | 139 days 14:54:11 rpc_pipefs | 30 | 1286 days 486:09:06.068985 | 42 days 37:00:18.202299 | 173 days 16:16:15 autofs | 23 | 756 days 374:13:45 | 32 days 37:08:25.434783 | 54 days 16:16:15 jfs | 10 | 677 days 158:20:12 | 67 days 32:38:01.20 | 160 days 16:16:15 vfat | 20 | 636 days 326:41:41.13894 | 31 days 35:32:05.056947 | 68 days 16:16:15 smbfs | 15 | 620 days 231:13:12.099919 | 41 days 23:24:52.806661 | 122 days 16:20:01 devfs | 9 | 609 days 151:46:53.100264 | 67 days 32:51:52.566696 | 155 days 16:16:15 ntfs | 20 | 584 days 325:04:00.023183 | 29 days 21:03:12.001159 | 68 days 16:16:15 selinuxfs | 9 | 485 days 145:33:36.021437 | 53 days 37:30:24.002382 | 173 days 16:16:15 iso9660 | 13 | 463 days 207:52:24 | 35 days 30:45:34.153846 | 84 days 16:06:15 subfs | 10 | 403 days 144:54:16.30619 | 40 days 21:41:25.630619 | 122 days 16:20:01 supermount | 9 | 325 days 145:38:07.087554 | 36 days 18:50:54.120839 | 68 days 16:16:15 ramfs | 10 | 249 days 147:43:34.143084 | 24 days 36:22:21.414308 | 32 days 16:16:15 reiser4 | 10 | 240 days 161:33:44.187484 | 24 days 16:09:22.418748 | 28 days 16:16:15 ^^^^^^^^^^^^^ cramfs | 3 | 137 days 48:48:44.087554 | 45 days 32:16:14.695851 | 58 days 16:16:15 openpromfs | 5 | 130 days 81:21:15 | 26 days 16:16:15 | 43 days 16:16:15 capifs | 3 | 102 days 48:48:45 | 34 days 16:16:15 | 39 days 16:16:15 udf | 3 | 96 days 48:10:38 | 32 days 16:03:32.666667 | 51 days 16:16:15 cifs | 3 | 81 days 48:48:45 | 27 days 16:16:15 | 35 days 16:16:15 squashfs | 3 | 79 days 48:48:45 | 26 days 24:16:15 | 34 days 16:16:15 configfs | 3 | 64 days 48:28:45 | 21 days 24:09:35 | 23 days 16:16:15 ocfs2_dlmfs | 1 | 23 days 16:16:15 | 23 days 16:16:15 | 23 days 16:16:15 oprofilefs | 1 | 23 days 16:16:08.088967 | 23 days 16:16:08.088967 | 23 days 16:16:08.088967 unionfs | 1 | 22 days 16:16:15 | 22 days 16:16:15 | 22 days 16:16:15 (33 rows) klive=> Currently the number of reiser4 users is 35, this lists only the kernels running in real time as I write this that are running reiser4: klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where last + '1 day' > 'now' group by f order by sum(uptime) desc; f | count | sum | avg | max ----------------------------------+-------+-----------------------------+-------------------------+------------------------- sysfs | 484 | 3928 days 4530:39:11.52799 | 8 days 12:08:15.76762 | 173 days 16:16:15 tmpfs | 480 | 3927 days 4501:01:21.406313 | 8 days 13:43:37.669596 | 173 days 16:16:15 usbfs | 437 | 2903 days 3824:39:39.983599 | 6 days 24:11:04.485088 | 173 days 16:16:15 ext3 | 279 | 2053 days 2355:05:46.852916 | 7 days 17:02:36.08191 | 173 days 16:16:15 reiserfs | 230 | 1949 days 2245:30:53.639863 | 8 days 21:08:13.276695 | 155 days 16:16:15 xfs | 58 | 624 days 587:32:09.569396 | 10 days 28:20:12.578783 | 139 days 14:54:11 ext2 | 91 | 608 days 933:47:49.912624 | 6 days 26:36:47.361677 | 139 days 14:54:11 binfmt_misc | 58 | 600 days 619:39:09.233036 | 10 days 18:57:34.297121 | 173 days 16:16:15 nfsd | 32 | 546 days 420:58:29.048813 | 17 days 14:39:19.657775 | 89 days 16:16:15 jfs | 10 | 546 days 143:40:39 | 54 days 28:46:03.90 | 160 days 16:16:15 nfs | 28 | 518 days 380:48:49.210675 | 18 days 25:36:01.757524 | 64 days 16:16:15 rpc_pipefs | 25 | 473 days 347:10:34.048813 | 18 days 35:58:01.361953 | 173 days 16:16:15 autofs | 17 | 211 days 218:50:13.090969 | 12 days 22:45:18.417116 | 54 days 16:16:15 selinuxfs | 10 | 204 days 105:21:47 | 20 days 20:08:10.70 | 173 days 16:16:15 devfs | 3 | 193 days 48:35:34.022351 | 64 days 24:11:51.340784 | 155 days 16:16:15 ntfs | 74 | 93 days 459:41:24.880971 | 1 day 12:22:27.092986 | 30 days 16:16:15 vfat | 64 | 83 days 437:49:49.197282 | 1 day 13:57:57.956208 | 37 days 16:16:15 iso9660 | 16 | 90 days 126:40:31.377898 | 5 days 22:55:01.961119 | 75 days 13:45:31 smbfs | 7 | 72 days 122:03:20 | 10 days 24:17:37.142857 | 27 days 16:16:15 reiser4 | 35 | 56 days 314:41:54.194701 | 1 day 23:23:28.976991 | 24 days 16:16:15 ^^^^^^^^^^^^^^^ udf | 1 | 51 days 16:16:15 | 51 days 16:16:15 | 51 days 16:16:15 ramfs | 11 | 45 days 82:10:59.027323 | 4 days 09:39:10.820666 | 30 days 16:16:15 configfs | 2 | 24 days 25:15:41 | 12 days 12:37:50.50 | 22 days 16:06:15 squashfs | 1 | 21 days 16:16:15 | 21 days 16:16:15 | 21 days 16:16:15 subfs | 2 | 20 days 18:08:18.063223 | 10 days 09:04:09.031611 | 20 days 16:15:43.063223 cifs | 6 | 15 days 66:54:31 | 2 days 23:09:05.166667 | 11 days 16:16:15 openpromfs | 3 | 14 days 61:21:33.08448 | 4 days 36:27:11.02816 | 11 days 16:09:12.08448 fuse | 4 | 10 days 34:01:09 | 2 days 20:30:17.25 | 8 days 16:16:15 debugfs | 6 | 9 days 46:25:01 | 1 day 19:44:10.166667 | 8 days 16:16:15 minix | 1 | 8 days 16:16:15 | 8 days 16:16:15 | 8 days 16:16:15 hfsplus | 1 | 1 day 12:20:26 | 1 day 12:20:26 | 1 day 12:20:26 ufs | 7 | 10:55:17 | 01:33:36.714286 | 04:18:01 supermount | 2 | 03:14:39 | 01:37:19.50 | 01:52:35 securityfs | 1 | 00:38:07 | 00:38:07 | 00:38:07 (34 rows) klive=> And these are the respective kernel versions: klive=> select kernel_release, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where last + '1 day' > 'now' and f = 'reiser4' group by kernel_release order by sum(uptime) desc; kernel_release | count | sum | avg | max -------------------------+-------+------------------------+------------------------+------------------------ 2.6.17-ckpp1 | 1 | 24 days 16:16:15 | 24 days 16:16:15 | 24 days 16:16:15 2.6.17-beyond2 | 3 | 9 days 61:26:13.021287 | 3 days 20:28:44.340429 | 6 days 16:13:52.021287 2.6.15-gentoo-r1-blip-1 | 1 | 10 days 16:16:15 | 10 days 16:16:15 | 10 days 16:16:15 2.6.16-gentoo-r12 | 1 | 6 days 16:06:15 | 6 days 16:06:15 | 6 days 16:06:15 2.6.17-beyond1.1 | 1 | 3 days 17:41:00 | 3 days 17:41:00 | 3 days 17:41:00 2.6.16-reiser4-r7 | 1 | 2 days 08:58:55.041969 | 2 days 08:58:55.041969 | 2 days 08:58:55.041969 2.6.18-rc1-mm2-non14 | 1 | 1 day 21:35:33 | 1 day 21:35:33 | 1 day 21:35:33 2.6.17-no4 | 1 | 1 day 04:56:21 | 1 day 04:56:21 | 1 day 04:56:21 2.6.17-beyond2.1 | 2 | 24:53:40 | 12:26:50 | 23:01:05 2.6.16-beyond4.1 | 1 | 22:51:05 | 22:51:05 | 22:51:05 2.6.17-PRX5.1 | 2 | 18:16:52 | 09:08:26 | 18:16:52 2.6.18-rc1-mm2 | 1 | 18:16:52 | 18:16:52 | 18:16:52 2.6.17-gentoo-r3 | 3 | 11:02:11 | 03:40:43.666667 | 07:05:39 2.6.17-rc1-no2 | 2 | 10:24:03.045926 | 05:12:01.522963 | 07:05:39 2.6.17-beyond1 | 1 | 09:02:04 | 09:02:04 | 09:02:04 2.6.17-beyond-git2 | 1 | 09:02:04 | 09:02:04 | 09:02:04 2.6.16-beyond3 | 1 | 07:05:39 | 07:05:39 | 07:05:39 2.6.17-no1 | 3 | 06:33:04 | 02:11:01.333333 | 03:18:25 2.6.17-beyond2-r2 | 1 | 05:22:31 | 05:22:31 | 05:22:31 2.6.17-gentoo-r4 | 1 | 04:18:01 | 04:18:01 | 04:18:01 2.6.16-beyond4 | 3 | 03:19:24.068344 | 01:06:28.022781 | 03:09:24.068344 2.6.17-reiser4-r4 | 2 | 00:57:37.017176 | 00:28:48.508588 | 00:57:37.017176 2.6.16-reiser4-r11 | 1 | 00:00:00 | 00:00:00 | 00:00:00 (23 rows) klive=> This is the whole list of kernel_releases that ever run reiser4: kernel_release | count | sum | avg | max -------------------------------------+-------+-------------------------------+--------------------------+-------------------------- 2.6.15-gentoo-r1 | 15726 | 12498 days 94186:27:26.881778 | 25:03:46.27794 | 139 days 14:54:11 2.6.15.6 | 262 | 6192 days 4014:29:48.915128 | 23 days 30:31:43.011126 | 114 days 16:16:15 2.6.16-gentoo-r7 | 5893 | 3852 days 34120:33:11.729484 | 21:28:40.005384 | 35 days 16:11:20.052466 2.6.16-gentoo-r9 | 7349 | 3505 days 39841:34:06.868008 | 16:52:04.132109 | 45 days 16:16:15 2.6.14-gentoo-r5 | 3363 | 3774 days 25871:49:28.023204 | 1 day 10:37:34.22778 | 85 days 16:16:15 2.6.16-rc5 | 1309 | 3909 days 10877:31:17.420649 | 2 days 31:58:47.179084 | 42 days 16:16:15 2.6.15-gentoo-r7 | 4183 | 2741 days 25823:28:50.761387 | 21:53:59.811322 | 48 days 16:10:16.072702 2.6.16-gentoo-r8 | 3300 | 2737 days 21674:06:24.528491 | 26:28:24.116524 | 55 days 16:16:15 2.6.16-gentoo-r3 | 4780 | 2369 days 26106:57:10.720939 | 17:21:22.558728 | 91 days 16:16:15 2.6.16-gentoo-r2 | 2303 | 2600 days 17412:40:17.134521 | 1 day 10:39:21.449038 | 89 days 16:16:15 2.6.15.1 | 1463 | 2818 days 10447:34:09.563652 | 1 day 29:22:10.177419 | 136 days 16:16:15 2.6.16-gentoo-r1 | 3608 | 2233 days 21934:44:07.471825 | 20:55:59.270364 | 64 days 16:03:18.045239 2.6.16 | 1165 | 2508 days 9354:44:26.049247 | 2 days 11:41:48.382875 | 53 days 16:06:15 2.6.16-gentoo-r6 | 2075 | 2300 days 13980:21:11.200159 | 1 day 09:20:23.745157 | 58 days 16:16:15 2.6.15-gentoo-r5 | 3321 | 1909 days 17474:22:48.504908 | 19:03:27.458147 | 104 days 16:15:50.041517 2.6.17-gentoo | 3291 | 1753 days 19508:59:15.100422 | 18:42:42.97633 | 30 days 16:16:15 2.6.17-rc1 | 785 | 2053 days 8234:01:41.533576 | 2 days 25:15:21.912782 | 69 days 16:16:15 2.6.15 | 2206 | 1673 days 14112:53:10.800042 | 24:35:55.571532 | 45 days 16:16:15 2.6.15-gentoo | 3284 | 1260 days 20322:09:18.149661 | 15:23:47.453761 | 26 days 16:16:15 2.6.16-beyond4 | 2123 | 1268 days 15252:17:49.21025 | 21:31:07.484319 | 21 days 16:16:15 2.6.16-archck1 | 1330 | 1388 days 10490:30:49.827973 | 1 day 08:56:03.195359 | 36 days 16:16:15 2.6.15.4 | 302 | 1683 days 2958:42:00.940999 | 5 days 23:32:43.314374 | 80 days 16:16:15 2.6.12-12mdk | 360 | 1630 days 3696:44:27.785247 | 4 days 22:56:07.410515 | 84 days 16:06:15 2.6.15-ck3-r1 | 769 | 1364 days 6171:30:41.606372 | 1 day 26:35:41.796627 | 43 days 16:16:15 2.6.15-nitro3 | 1709 | 1142 days 10979:07:40.254166 | 22:27:42.293888 | 31 days 16:16:15 2.6.16-beyond3 | 1806 | 1030 days 13334:56:02.827243 | 21:04:17.011532 | 22 days 16:14:05.037859 2.6.15-ck7 | 637 | 1357 days 5372:37:06.322867 | 2 days 11:33:41.07743 | 31 days 16:16:15 2.6.9-22.0.1.ELsmp | 13 | 1571 days 211:31:15 | 120 days 36:34:42.692308 | 173 days 16:16:15 2.6.15-archck | 2579 | 865 days 15948:06:33.240877 | 14:14:00.478186 | 28 days 16:16:15 2.6.15-1-k7 | 164 | 1376 days 2179:27:47.995174 | 8 days 22:39:18.95119 | 35 days 16:16:15 2.6.16-ck11 | 1402 | 1016 days 10372:57:34.350942 | 24:47:27.542333 | 39 days 16:14:38.079596 2.6.16-gentoo | 1927 | 942 days 11516:24:54.920025 | 17:42:30.853617 | 47 days 16:16:15 2.6.14.5 | 169 | 1318 days 1681:33:11 | 7 days 29:07:17.816568 | 86 days 16:16:15 2.6.16-ck10 | 814 | 1130 days 5987:58:35.725072 | 1 day 16:40:23.483692 | 40 days 16:16:15 2.6.14-hardened-r5 | 636 | 1194 days 4394:03:05.986415 | 1 day 27:57:55.76413 | 52 days 16:16:15 2.6.12-gentoo-r10 | 300 | 1213 days 3756:26:28.092538 | 4 days 13:33:41.293642 | 89 days 16:16:15 2.6.17 | 1038 | 918 days 10628:20:44.642394 | 31:27:52.875378 | 32 days 16:04:49.04156 2.6.16-ck3 | 1043 | 1053 days 6467:18:14.426646 | 1 day 06:25:50.809613 | 54 days 16:16:15 2.6.16.1 | 314 | 1161 days 3632:43:26.24459 | 3 days 28:18:28.937085 | 67 days 16:16:15 2.6.15-archck2 | 1738 | 727 days 12587:25:00.214091 | 17:16:53.751562 | 14 days 16:16:15 2.6.13-gentoo-r5 | 880 | 833 days 9688:10:28.922616 | 33:43:38.896503 | 34 days 16:16:15 2.6.15-ck2pwr | 92 | 1127 days 1357:25:27.655134 | 12 days 20:45:16.604947 | 52 days 16:16:15 [..] too long to list them all. > "patchup" is used by (tens of) thousands of computers in governmental > laboratories the US "national security" depends upon. I don't see any ext4 in the klive logs, it'd be nice if I would know how to track your ext4 patchset too to make a more accurate comparison. We can't compare the above numbers to the klive ones, the above is the interpolation of the reiser4 and klive userbase only. While if I could detect the ext4 "patchup" it would be feasible to attempt a comparison. Disclaimer: the data should not be trusted, use it at your risk. Furthermore it may not represent a reliable sample, because the fs are only submitted at the first connection with the server after reboot, so if reiser4 or fuse are mounted later on (normally after an average of 5 minutes) they may not get logged in KLive. OTOH this also means that all computers showing reiser4 in the logs had it mounted since about the boot time (so if reiser4 would corrupt memory badly by just mounting nobody could reasonably hope to reach 20 days of uptime). Hope this helps. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 12:35 ` Andrea Arcangeli @ 2006-07-25 12:51 ` Rene Rebe 2006-07-25 17:47 ` Hans Reiser 1 sibling, 0 replies; 639+ messages in thread From: Rene Rebe @ 2006-07-25 12:51 UTC (permalink / raw) To: Andrea Arcangeli; +Cc: Nikita Danilov, Hans Reiser, Linux Kernel Mailing List Hi, On Tuesday 25 July 2006 14:35, Andrea Arcangeli wrote: ... > average of 5 minutes) they may not get logged in KLive. OTOH this also > means that all computers showing reiser4 in the logs had it mounted > since about the boot time (so if reiser4 would corrupt memory badly by > just mounting nobody could reasonably hope to reach 20 days of > uptime). Well I tested Reiser4 on systems building our Linux flavour (based on the T2 framework) thus with excessive software compilation, a lot of tarball extraction, compilation of fat packages such as linux, mozilla, openoffice et.al. and huge per-package ccache directories -> zilions of small files and did not encounter any abnormal behaviour aside the yet-to-be-fixed commit storms. Performance was superiour over ext3, XFS*, JFS or Reiser3 in this workload (that is less time "wasted" for FS housekeeping and the build finished noticable earlier). Patchset was tested around 2.6.1{4,5]. But I do not have the numbers around anymore and was basically waiting for Reiser4 to make it into the kernel to schedule further testing. You can see I'm not a blind Reiser4 follower but would rather like to see a fair chance for it. *) XFS oopsed so often in this test that I'll never touch it again ... Yours, -- René Rebe - Rubensstr. 64 - 12157 Berlin (Europe / Germany) http://exactcode.de | http://t2-project.org | http://rebe.name +49 (0)30 / 255 897 45 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 12:35 ` Andrea Arcangeli 2006-07-25 12:51 ` Rene Rebe @ 2006-07-25 17:47 ` Hans Reiser 2006-07-25 19:20 ` Francesco Biscani ` (2 more replies) 1 sibling, 3 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-25 17:47 UTC (permalink / raw) To: Andrea Arcangeli; +Cc: Nikita Danilov, Rene Rebe, Linux Kernel Mailing List Wow, I would never have guessed our market share was that high as 1/5th of ext3. I mean, you can't even get a distro which allows you to install onto reiser4 without hours of work so far as I know. I guess there are people who really do care about twice as fast. Hans Andrea Arcangeli wrote: > reiserfs | 19934 | 19257 days 133294:48:27.256508 | 29:52:18.181361 | 155 days 16:16:15 > ext3 | 22077 | 16346 days 132190:42:38.862752 | 23:45:27.062502 | 173 days 16:16:15 > > reiser4 | 4409 | 1926 days 27479:22:27.459534 | 16:42:59.666015 | 28 days 16:16:15 >^^^^^^^^^^^^^ > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 17:47 ` Hans Reiser @ 2006-07-25 19:20 ` Francesco Biscani 2006-07-25 21:17 ` Horst H. von Brand 2006-07-26 12:45 ` Adrian Bunk 2 siblings, 0 replies; 639+ messages in thread From: Francesco Biscani @ 2006-07-25 19:20 UTC (permalink / raw) To: Hans Reiser Cc: Andrea Arcangeli, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Tuesday 25 July 2006 19:47, Hans Reiser wrote: > Wow, I would never have guessed our market share was that high as 1/5th > of ext3. I mean, you can't even get a distro which allows you to > install onto reiser4 without hours of work so far as I know. I guess > there are people who really do care about twice as fast. > > Hans > Yes, this was a pleasant surprise indeed :) This useless mail from a mere user is just to testify that I've used reiser4 on my laptop since June 2004, and that, barring occasional hiccups, I never had serious problems with it. In fact I haven't lost a single bit of data despite crashes and cold shutdowns due to power outage. I don't know if this means something or if I've just been lucky, but it seems to me that when reiser4 crashes (and sometimes it does, given its young age), it behaves very very well. That's the thing that impressed me the most (and it is really something, given for example the debacle of a mature filesystem like XFS in 2.6.17). Oh, and performance, of course ;) I really hope that the technical difficulties that are preventing reiser4 from entering mainline will be sorted out soon. Thanks, Francesco (crawling back in the shadow) -- Dr. Francesco Biscani Dipartimento di Astronomia Università di Padova biscani@pd.astro.it ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 17:47 ` Hans Reiser 2006-07-25 19:20 ` Francesco Biscani @ 2006-07-25 21:17 ` Horst H. von Brand 2006-07-25 22:48 ` Jim Crilly 2006-07-25 23:45 ` Luigi Genoni 2006-07-26 12:45 ` Adrian Bunk 2 siblings, 2 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-07-25 21:17 UTC (permalink / raw) To: Hans Reiser Cc: Andrea Arcangeli, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List Hans Reiser <reiser@namesys.com> wrote: > Wow, I would never have guessed our market share was that high as 1/5th > of ext3. I mean, you can't even get a distro which allows you to > install onto reiser4 without hours of work so far as I know. I guess > there are people who really do care about twice as fast. That makes the data /higly/ suspect: Most of the Linux users I know wouldn't know where to start to compile a kernel, forget about patching around. And of the minority who can, they mostly run machines in production, and won't dream of using non-distribution kernels, let alone custom patched ones. BTW, where do you get the "twice as fast" number from? -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 21:17 ` Horst H. von Brand @ 2006-07-25 22:48 ` Jim Crilly 2006-07-25 23:48 ` andrea 2006-07-25 23:45 ` Luigi Genoni 1 sibling, 1 reply; 639+ messages in thread From: Jim Crilly @ 2006-07-25 22:48 UTC (permalink / raw) To: Horst H. von Brand Cc: Hans Reiser, Andrea Arcangeli, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On 07/25/06 05:17:48PM -0400, Horst H. von Brand wrote: > Hans Reiser <reiser@namesys.com> wrote: > > Wow, I would never have guessed our market share was that high as 1/5th > > of ext3. I mean, you can't even get a distro which allows you to > > install onto reiser4 without hours of work so far as I know. I guess > > there are people who really do care about twice as fast. > > That makes the data /higly/ suspect: Most of the Linux users I know > wouldn't know where to start to compile a kernel, forget about patching > around. And of the minority who can, they mostly run machines in > production, and won't dream of using non-distribution kernels, let alone > custom patched ones. Well if you take a look at the KLive webpage you'll see that there's only 485 computers providing data. I'm not trying to knock Andrea's project, but the sample size is quite small and only really representative of people who would be inclined to read lkml, patch their kernels, etc. Jim. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 22:48 ` Jim Crilly @ 2006-07-25 23:48 ` andrea 0 siblings, 0 replies; 639+ messages in thread From: andrea @ 2006-07-25 23:48 UTC (permalink / raw) To: Horst H. von Brand, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Tue, Jul 25, 2006 at 06:48:35PM -0400, Jim Crilly wrote: > Well if you take a look at the KLive webpage you'll see that there's only > 485 computers providing data. I'm not trying to knock Andrea's project, but > the sample size is quite small and only really representative of people who > would be inclined to read lkml, patch their kernels, etc. There are quite a few using the official distro kernels but the point remains that the KLive users are probably the ones most interested to try all new stuff and to live on the edge. They're not afraid to put the new technologies to work for them. That's probably why they're using reiser4 so much. OTOH they're probably the ones putting the fs under the most stress, a regular desktop user not capable of running klive.sh --install from the shell, would probably leave his CPU and harddisk idle most of the time too. Being this only a sample I can't come up with absolute figures, but obviously there are more users than what is being recorded by KLive. However the sample is not so small, in less then a year KLive logged 76740 days of uptime and 61355 reboots (or klive session restarts), and like you noticed an average of about 400-500 systems are always connected. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 21:17 ` Horst H. von Brand 2006-07-25 22:48 ` Jim Crilly @ 2006-07-25 23:45 ` Luigi Genoni 1 sibling, 0 replies; 639+ messages in thread From: Luigi Genoni @ 2006-07-25 23:45 UTC (permalink / raw) To: Horst H. von Brand; +Cc: Hans Reiser, Linux Kernel Mailing List I suppose from the ones, lime me, who has production servers, but also some test system to play with. Luigi On Tuesday 25 July 2006 23:17, Horst H. von Brand wrote: > Hans Reiser <reiser@namesys.com> wrote: > > Wow, I would never have guessed our market share was that high as 1/5th > > of ext3. I mean, you can't even get a distro which allows you to > > install onto reiser4 without hours of work so far as I know. I guess > > there are people who really do care about twice as fast. > > That makes the data /higly/ suspect: Most of the Linux users I know > wouldn't know where to start to compile a kernel, forget about patching > around. And of the minority who can, they mostly run machines in > production, and won't dream of using non-distribution kernels, let alone > custom patched ones. > > BTW, where do you get the "twice as fast" number from? ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-25 17:47 ` Hans Reiser 2006-07-25 19:20 ` Francesco Biscani 2006-07-25 21:17 ` Horst H. von Brand @ 2006-07-26 12:45 ` Adrian Bunk 2006-07-26 13:29 ` andrea 2 siblings, 1 reply; 639+ messages in thread From: Adrian Bunk @ 2006-07-26 12:45 UTC (permalink / raw) To: Hans Reiser Cc: Andrea Arcangeli, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Tue, Jul 25, 2006 at 11:47:29AM -0600, Hans Reiser wrote: > Wow, I would never have guessed our market share was that high as 1/5th > of ext3. I mean, you can't even get a distro which allows you to > install onto reiser4 without hours of work so far as I know. I guess > there are people who really do care about twice as fast. I doubt Andreas values have much value. According to the numbers on the klive website, Gentoo has 47 times as many users as SuSE (sic). Considering that klive is offered by someone working for SuSE, the Gentoo project could make a good news article ("Numbers by a SuSE developer confirm Gentoo has 47 times the market share of Suse.") out of it. ;-) > Hans cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 12:45 ` Adrian Bunk @ 2006-07-26 13:29 ` andrea 2006-07-26 13:43 ` Adrian Bunk 0 siblings, 1 reply; 639+ messages in thread From: andrea @ 2006-07-26 13:29 UTC (permalink / raw) To: Adrian Bunk Cc: Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 02:45:57PM +0200, Adrian Bunk wrote: > On Tue, Jul 25, 2006 at 11:47:29AM -0600, Hans Reiser wrote: > > > Wow, I would never have guessed our market share was that high as 1/5th > > of ext3. I mean, you can't even get a distro which allows you to > > install onto reiser4 without hours of work so far as I know. I guess > > there are people who really do care about twice as fast. > > I doubt Andreas values have much value. I think nobody ever pretended them to have much value, they only need to have some minor value to be useful. And as far as I can tell no better source of data about the kernel testing userbase exists as of today. > According to the numbers on the klive website, Gentoo has 47 times as > many users as SuSE (sic). > > Considering that klive is offered by someone working for SuSE, the > Gentoo project could make a good news article ("Numbers by a SuSE > developer confirm Gentoo has 47 times the market share of Suse.") > out of it. ;-) Oh well, you misunderstood KLive completely. Please read again the top of the page: "this project aims to provide kernel live feedback about the usage of every different Linux Kernel version". Where did I ever mention anything about distributions? There's absolutely no way to tell which distribution is running. I only can tell which _kernel_ is running. It could be 100% of the mainline users are running mainline on top of SUSE distro, or on top of Gentoo, or on top of Ubuntuu, there's absolutely no way to know about the distro. Nobody could ever make a claim like the above by using the KLive data. To make an example I run mainline myself on top of opensuse 10.1, and I get rightfully accounted as a "mainline" and not as a "SUSE". All you can tell is that there are many more people running KLive in combination with Gentoo kernels than with SUSE kernels. But you can't tell anything about what kind of distribution is running. One could guess the Gentoo kernels run on top of a Gentoo userland, but for the mainline ones you really can't tell, they could be slackware but they could be any other distro too. Also note that the cpushare.com domain is not related to any distro, so the fact I also do consulting for SUSE is irrelevant to KLive or any other dealings at the cpushare.com domain. As long there is people running KLive and browsing the website like now, it means somebody thinks it's useful and so I keep delivering the service. As of now, the Gentoo and mainline kernel users are the ones providing most of the feedback. Perhaps I should have filed a patent on KLive too just to make you happy, right? ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 13:29 ` andrea @ 2006-07-26 13:43 ` Adrian Bunk 2006-07-26 14:28 ` andrea 0 siblings, 1 reply; 639+ messages in thread From: Adrian Bunk @ 2006-07-26 13:43 UTC (permalink / raw) To: andrea; +Cc: Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 03:29:57PM +0200, andrea@cpushare.com wrote: > On Wed, Jul 26, 2006 at 02:45:57PM +0200, Adrian Bunk wrote: > > On Tue, Jul 25, 2006 at 11:47:29AM -0600, Hans Reiser wrote: > > > > > Wow, I would never have guessed our market share was that high as 1/5th > > > of ext3. I mean, you can't even get a distro which allows you to > > > install onto reiser4 without hours of work so far as I know. I guess > > > there are people who really do care about twice as fast. > > > > I doubt Andreas values have much value. > > I think nobody ever pretended them to have much value, they only need > to have some minor value to be useful. And as far as I can tell no > better source of data about the kernel testing userbase exists as of > today. >... Hans said "Wow, I would never have guessed our market share was that high as 1/5th of ext3." And estimate of the marked share based on klive data is simply wrong. That was my point. My distribution example was just one example of what happens when you wrongly try to estimate a marked share based on klive data (and it seems you missed the smiley after the last paragraph of my email). cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 13:43 ` Adrian Bunk @ 2006-07-26 14:28 ` andrea 2006-07-26 14:50 ` Adrian Bunk 0 siblings, 1 reply; 639+ messages in thread From: andrea @ 2006-07-26 14:28 UTC (permalink / raw) To: Adrian Bunk Cc: Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 03:43:26PM +0200, Adrian Bunk wrote: > My distribution example was just one example of what happens when you > wrongly try to estimate a marked share based on klive data (and it seems You're again wrong generalizing. With KLive I can attempt to estimate market share of _kernel_ code (what Hans did), but you can't attempt to estimate market share of userland (what you did). It won't be a smile at the end making your statement right. The estimate market share of kernel code may not be accurate but it has some minor significance. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 14:28 ` andrea @ 2006-07-26 14:50 ` Adrian Bunk 2006-07-26 16:06 ` andrea 0 siblings, 1 reply; 639+ messages in thread From: Adrian Bunk @ 2006-07-26 14:50 UTC (permalink / raw) To: andrea; +Cc: Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 04:28:54PM +0200, andrea@cpushare.com wrote: > On Wed, Jul 26, 2006 at 03:43:26PM +0200, Adrian Bunk wrote: > > My distribution example was just one example of what happens when you > > wrongly try to estimate a marked share based on klive data (and it seems > > You're again wrong generalizing. With KLive I can attempt to estimate > market share of _kernel_ code (what Hans did), but you can't attempt > to estimate market share of userland (what you did). It won't be a > smile at the end making your statement right. No, you can't estimate the market share of kernel code based on klive data. Someone said in this thread 'is used by (tens of) thousands of computers in governmental laboratories the US "national security" depends upon.' klive will never get any feedback from such users. But the computer freak using Gentoo and always trying the latest things like reiser4 is relatively likely to also use klive. > The estimate market share of kernel code may not be accurate but it > has some minor significance. You are able to say that based on klive data, reiser4 has at least 35 users in the world. But you can not tell based on klive data whether the ratio of reiser4:ext3 users in the world is more like 1:5, 1:500 or 1:50000. cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 14:50 ` Adrian Bunk @ 2006-07-26 16:06 ` andrea 2006-07-26 16:53 ` Adrian Bunk 2006-07-26 17:02 ` J. Bruce Fields 0 siblings, 2 replies; 639+ messages in thread From: andrea @ 2006-07-26 16:06 UTC (permalink / raw) To: Adrian Bunk Cc: Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 04:50:19PM +0200, Adrian Bunk wrote: > Someone said in this thread 'is used by (tens of) thousands of computers > in governmental laboratories the US "national security" depends upon.' There can be people using reiser4 behind the firewall too, what's the point? IIRC US .gov even sponsored part of reiser4 development, how do you know they're not testing it too? You don't believe KLive has any relation to reality, but you have no way to proof your claim. JFYI: all statistics only take a sample of the larger space, the whole point of having a statistic is because you can't measure the total. The smaller the sample compared to the total, the less the stats are accurate, but they still have some statistical significance. And one can always hope that KLive will grow larger. Last but not the least defining gentoo users as freaks isn't very nice from your part. For all new startups they're the ideal userbase to have, and they do a great deal of good work by testing all new stuff and they help speeding up innovation. Infact I think having a sample of what those brave users run is more important than the rest, the rest usually follows the ones living on the edge eventually. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 16:06 ` andrea @ 2006-07-26 16:53 ` Adrian Bunk 2006-07-26 17:02 ` J. Bruce Fields 1 sibling, 0 replies; 639+ messages in thread From: Adrian Bunk @ 2006-07-26 16:53 UTC (permalink / raw) To: andrea; +Cc: Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 06:06:04PM +0200, andrea@cpushare.com wrote: > On Wed, Jul 26, 2006 at 04:50:19PM +0200, Adrian Bunk wrote: > > Someone said in this thread 'is used by (tens of) thousands of computers > > in governmental laboratories the US "national security" depends upon.' > > There can be people using reiser4 behind the firewall too, what's the > point? IIRC US .gov even sponsored part of reiser4 development, how do > you know they're not testing it too? I don't know. The klive data might be inaccurate in any direction. In this case I'd suspect an inaccuraty in one specific direction. > You don't believe KLive has any relation to reality, but you have no > way to proof your claim. JFYI: all statistics only take a sample of > the larger space, the whole point of having a statistic is because you > can't measure the total. The smaller the sample compared to the total, > the less the stats are accurate, but they still have some statistical > significance. And one can always hope that KLive will grow larger. Size isn't everything. The problem is getting an accurate sample of data. Let me try to make the problem clearer by making an example: Consider you want to know the ratio between housewifes and women with a job for women aged between 30 and 40. Consider you get your data by asking women in shopping malls between 10 and 11 o'clock in the morning on workdays. Do you understand why the result might be quite different from the actual ratio? Do you understand why asking a million women in shopping malls between 10 and 11 o'clock in the morning on workdays wouldn't make the data better? And do you know the Linux Counter at [1]? I remember several years ago Debian had some default setting in some package to send reports there. smail was at that times the default Debian MTA, and therefore also the most popular MTA at the Linux counter... > Last but not the least defining gentoo users as freaks isn't very nice > from your part. For all new startups they're the ideal userbase to > have, and they do a great deal of good work by testing all new stuff Please read carefully what I said. I did NOT say: "All Gentoo users are freaks." It was more (implicitely) in the direction: "Many freaks use Gentoo." The latter is IMHO not that far from reality. And "freak" wasn't meant negative. > and they help speeding up innovation. Infact I think having a sample > of what those brave users run is more important than the rest, the > rest usually follows the ones living on the edge eventually. The majority of user will use the filesystem their distribution offers as default, no matter what people living on the edge today will use... cu Adrian [1] http://counter.li.org/ -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 16:06 ` andrea 2006-07-26 16:53 ` Adrian Bunk @ 2006-07-26 17:02 ` J. Bruce Fields 2006-07-26 17:20 ` andrea 1 sibling, 1 reply; 639+ messages in thread From: J. Bruce Fields @ 2006-07-26 17:02 UTC (permalink / raw) To: andrea Cc: Adrian Bunk, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 06:06:04PM +0200, andrea@cpushare.com wrote: > JFYI: all statistics only take a sample of the larger space, the whole > point of having a statistic is because you can't measure the total. > The smaller the sample compared to the total, the less the stats are > accurate Definitely not true in general. If I wanted to know the gender ratio at the latest OLS I'd take the results from a sample of a dozen chosen randomly over the results from a sample of hundreds all taken from the men's room. For exactly the same quality of sampling, yes, the larger the better, but the point of diminishing returns comes pretty quickly. So given limited resources it's probably more important to work on the quality of the sample rather than on its size.... --b. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 17:02 ` J. Bruce Fields @ 2006-07-26 17:20 ` andrea 2006-07-26 19:01 ` Jerome Pinot 2006-07-26 20:50 ` Adrian Bunk 0 siblings, 2 replies; 639+ messages in thread From: andrea @ 2006-07-26 17:20 UTC (permalink / raw) To: J. Bruce Fields Cc: Adrian Bunk, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 01:02:36PM -0400, J. Bruce Fields wrote: > On Wed, Jul 26, 2006 at 06:06:04PM +0200, andrea@cpushare.com wrote: > > JFYI: all statistics only take a sample of the larger space, the whole > > point of having a statistic is because you can't measure the total. > > The smaller the sample compared to the total, the less the stats are > > accurate > > Definitely not true in general. If I wanted to know the gender ratio at > the latest OLS I'd take the results from a sample of a dozen chosen > randomly over the results from a sample of hundreds all taken from the > men's room. Well, your example is perhaps the worst one since you wouldn't be decreasing the quality of your stats very much by only doing the sample in the men's room ;). I guess you meant the woman's room. > For exactly the same quality of sampling, yes, the larger the better, > but the point of diminishing returns comes pretty quickly. So given > limited resources it's probably more important to work on the quality of > the sample rather than on its size.... No matter how you see it, the larger the better (in the worst case it won't make a difference). Certainly if I could work on the quality, that would be more important than adding 1 more user. But I can't work on the quality. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 17:20 ` andrea @ 2006-07-26 19:01 ` Jerome Pinot 2006-07-26 20:50 ` andrea 2006-07-26 20:50 ` Adrian Bunk 1 sibling, 1 reply; 639+ messages in thread From: Jerome Pinot @ 2006-07-26 19:01 UTC (permalink / raw) To: andrea Cc: J. Bruce Fields, Adrian Bunk, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On 7/27/06, andrea@cpushare.com <andrea@cpushare.com> wrote: > On Wed, Jul 26, 2006 at 01:02:36PM -0400, J. Bruce Fields wrote: > > On Wed, Jul 26, 2006 at 06:06:04PM +0200, andrea@cpushare.com wrote: > > > JFYI: all statistics only take a sample of the larger space, the whole > > > point of having a statistic is because you can't measure the total. > > > The smaller the sample compared to the total, the less the stats are > > > accurate > > > > Definitely not true in general. If I wanted to know the gender ratio at > > the latest OLS I'd take the results from a sample of a dozen chosen > > randomly over the results from a sample of hundreds all taken from the > > men's room. > > Well, your example is perhaps the worst one since you wouldn't be > decreasing the quality of your stats very much by only doing the > sample in the men's room ;). I guess you meant the woman's room. > > > For exactly the same quality of sampling, yes, the larger the better, > > but the point of diminishing returns comes pretty quickly. So given > > limited resources it's probably more important to work on the quality of > > the sample rather than on its size.... > > No matter how you see it, the larger the better (in the worst case it > won't make a difference). Certainly if I could work on the quality, > that would be more important than adding 1 more user. But I can't work > on the quality. Maybe you could try pushing the use of klive by some distros arguing it's "a way of getting usage statistics in order to improve kernel quality and hardware support". I mean, not just an extra package but a service launch at the first boot so anyone can use it. klive is a nice project, it just needs more _different_ users and maybe, a support of some distros. Maybe, you could try add a page in the klive wiki for putting packages and RPMs of klive for several distros ? What do you think ? It's a first step to get it merge into distros. In my case, I got some .tgz that feel lonely... Maybe be in 2 years, we will know EXACTLY of much people use EXT4/Reiser5/CoincoinFS/CrashFS... -- Jerome Pinot http://ngc891.blogdns.net/ ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 19:01 ` Jerome Pinot @ 2006-07-26 20:50 ` andrea 0 siblings, 0 replies; 639+ messages in thread From: andrea @ 2006-07-26 20:50 UTC (permalink / raw) To: Jerome Pinot Cc: J. Bruce Fields, Adrian Bunk, Rene Rebe, Linux Kernel Mailing List Removed some people from CC since we're quickly changing topic and I doubt they're interested. On Thu, Jul 27, 2006 at 04:01:02AM +0900, Jerome Pinot wrote: > Maybe you could try pushing the use of klive by some distros arguing > it's "a way of getting usage statistics in order to improve kernel > quality and hardware support". I mean, not just an extra package but a > service launch at the first boot so anyone can use it. > > klive is a nice project, it just needs more _different_ users and > maybe, a support of some distros. That would be nice yes. KLive is already supported by Gentoo that created a proper emerge package, that's why there are so many gentoo kernels being tracked. Others are of course welcome. I'm open to suggestions on extending it as well. At the moment the todo list is like this: 1) create an optional http transport to pass through most firewalls 2) send a packet when system reboots so I can track which sessions didn't cleanly reboot (it's not reliable if you disconnect from the internet first and you shutdown later, but it's better than nothing) 3) send the oops from kernel to klive server, so no oopses will be lost and I can use the pciids details to track down bad hardware as well 4) possibly send info on usb buses too and not only about the pci buses > Maybe, you could try add a page in the klive wiki for putting packages > and RPMs of klive for several distros ? What do you think ? It's a > first step to get it merge into distros. If more distros will pickup KLive that will help like demonstrated by the emerge package from gentoo. It's already very simple to install klive, I wrote an universal installer that runs on any flavour of any weird distro out there: wget klive.cpushare.com/klive.sh sh klive.sh --install If you don't want it anymore: sh klive.sh --uninstall that's it. The rpm/deb packages would be just to make life easier for distro to pick it up but I hoped they had the resources to do the packaging work themself... Perhaps once I'll create the CPUShare rpm/deb packages I'll be easy to share the work to create the KLive ones too. As far as spare time work goes, CPUShare currently has a much higher prio than KLive, and I'm not yet at the point of creating rpm/deb packages, now that CPUShare transaction works and you can already compute remotely, I'm currently fighting with the paypal sendbox so I can start accepting cash into the system ASAP. But the hardest part of the ebusiness side of CPUShare is the handling of the invoicing and receipts generation which is a non computer related problem. I almost solved it though. > In my case, I got some .tgz that feel lonely... The .tgz is only for the ones interested hacking or studying it (both client and server). You should be using the .sh script above. Also note, the export of the whole database isn't available on the site, but I've no problem to publish it after filtering out all the ip addresses (which are collected only for purely paranoid security purposes, and tor doesn't route udp traffic). > Maybe be in 2 years, we will know EXACTLY of much people use > EXT4/Reiser5/CoincoinFS/CrashFS... BTW, CPUShare records a lot more than the fs already. 2 years or more, but I'm not in a hurry. The server will also soon require some caching trick to avoid recomputing the queries every time you click on it. Current KLive is the best I could do in a very little time I spent on it, it works well, but it still has an huge room for improvement. Also note, so far KLive logged 76989 cumulative days of uptime. When I see a number like that my mind thinks at the energy that can be recycled if all that time recorded on KLive would be routed inside CPUShare. That's why for me it's more urgent to give a chance to those brave 500 users to cash in with CPUShare than to push KLive beyond its current status ;). In due time KLive will get more attention too. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 17:20 ` andrea 2006-07-26 19:01 ` Jerome Pinot @ 2006-07-26 20:50 ` Adrian Bunk 2006-07-26 21:17 ` andrea 2006-07-27 4:35 ` Hans Reiser 1 sibling, 2 replies; 639+ messages in thread From: Adrian Bunk @ 2006-07-26 20:50 UTC (permalink / raw) To: andrea Cc: J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 07:20:29PM +0200, andrea@cpushare.com wrote: > On Wed, Jul 26, 2006 at 01:02:36PM -0400, J. Bruce Fields wrote: >... > > For exactly the same quality of sampling, yes, the larger the better, > > but the point of diminishing returns comes pretty quickly. So given > > limited resources it's probably more important to work on the quality of > > the sample rather than on its size.... > > No matter how you see it, the larger the better (in the worst case it > won't make a difference). Certainly if I could work on the quality, > that would be more important than adding 1 more user. But I can't work > on the quality. But depending on the nature of the error, the worst case might be the common case (as I've already explained in another email). If you can't ensure the quality of your data, please don't use this data to wrongly draw any conclusions from them [1]. cu Adrian [1] the conclusion itself might or might not be true e.g. there _could_ be an 1:5 ratio between reiser4 and ext3 users but your data is not in any way able to support or reject this statement -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 20:50 ` Adrian Bunk @ 2006-07-26 21:17 ` andrea 2006-07-26 21:37 ` J. Bruce Fields 2006-07-27 6:56 ` Adrian Bunk 2006-07-27 4:35 ` Hans Reiser 1 sibling, 2 replies; 639+ messages in thread From: andrea @ 2006-07-26 21:17 UTC (permalink / raw) To: Adrian Bunk Cc: J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 10:50:22PM +0200, Adrian Bunk wrote: > But depending on the nature of the error, the worst case might be the > common case (as I've already explained in another email). > > If you can't ensure the quality of your data, please don't use this data > to wrongly draw any conclusions from them [1]. Please read the footer of the KLive pages: "The use of the information and of the software in this website is at your own risk. KLive probably doesn't represent a reliable sample of the real usage of the Linux Kernel." Said that pretending that KLive data has absolutely no significance at all and that you can't draw any conclusion at all from it, to me seems as wrong as pretending it to perfect. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 21:17 ` andrea @ 2006-07-26 21:37 ` J. Bruce Fields 2006-07-26 22:17 ` andrea 2006-07-27 6:56 ` Adrian Bunk 1 sibling, 1 reply; 639+ messages in thread From: J. Bruce Fields @ 2006-07-26 21:37 UTC (permalink / raw) To: andrea Cc: Adrian Bunk, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 11:17:41PM +0200, andrea@cpushare.com wrote: > Said that pretending that KLive data has absolutely no significance at > all and that you can't draw any conclusion at all from it, to me seems > as wrong as pretending it to perfect. This is where we disagree. It may be wrong, but it's certainly not nearly *as* wrong.... --b. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 21:37 ` J. Bruce Fields @ 2006-07-26 22:17 ` andrea 0 siblings, 0 replies; 639+ messages in thread From: andrea @ 2006-07-26 22:17 UTC (permalink / raw) To: J. Bruce Fields Cc: Adrian Bunk, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 05:37:57PM -0400, J. Bruce Fields wrote: > This is where we disagree. It may be wrong, but it's certainly not > nearly *as* wrong.... ;) My point of view in saying you can't dismiss the current KLive stats completely, is simply that I didn't expect reiser4 numbers to be so high even in the small sample of the brave KLive project that is full of people willing to test new stuff (also note that I compared it to isofs, I didn't attempt a comparison with ext3 myself). That makes it a positive in my view. If you think KLive numbers aren't a positive point for reiser4, then it can only mean you expected them to be even higher than they were... ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 21:17 ` andrea 2006-07-26 21:37 ` J. Bruce Fields @ 2006-07-27 6:56 ` Adrian Bunk 2006-07-27 8:33 ` the ' 'official' point of view' " Luigi Genoni 2006-07-27 11:52 ` the " 'official' point of view" " andrea 1 sibling, 2 replies; 639+ messages in thread From: Adrian Bunk @ 2006-07-27 6:56 UTC (permalink / raw) To: andrea Cc: J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Wed, Jul 26, 2006 at 11:17:41PM +0200, andrea@cpushare.com wrote: > On Wed, Jul 26, 2006 at 10:50:22PM +0200, Adrian Bunk wrote: > > But depending on the nature of the error, the worst case might be the > > common case (as I've already explained in another email). > > > > If you can't ensure the quality of your data, please don't use this data > > to wrongly draw any conclusions from them [1]. > > Please read the footer of the KLive pages: > > "The use of the information and of the software in this website is at > your own risk. KLive probably doesn't represent a reliable sample of > the real usage of the Linux Kernel." It was you who wrongly said: "With KLive I can attempt to estimate market share of _kernel_ code" Hadn't you read your own disclaimer? > Said that pretending that KLive data has absolutely no significance at > all and that you can't draw any conclusion at all from it, to me seems > as wrong as pretending it to perfect. Possibly wrong conclusions about the general market share based on data not having the quality for being the basis of such statements are worse than having no data. Every time someone will repeat the "1:5 ratio for reiser4:ext3 users", this will be an additional proof it's really worse than no data. cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 6:56 ` Adrian Bunk @ 2006-07-27 8:33 ` Luigi Genoni 2006-07-27 10:04 ` Adrian Bunk 2006-07-27 11:52 ` the " 'official' point of view" " andrea 1 sibling, 1 reply; 639+ messages in thread From: Luigi Genoni @ 2006-07-27 8:33 UTC (permalink / raw) To: Adrian Bunk Cc: andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, July 27, 2006 08:56, Adrian Bunk wrote: > >> Said that pretending that KLive data has absolutely no significance at >> all and that you can't draw any conclusion at all from it, to me seems as >> wrong as pretending it to perfect. > > Possibly wrong conclusions about the general market share based on data > not having the quality for being the basis of such statements are worse than > having no data. Am I missing something, or it is not marketing what we are talking about? please I do not understand your point. For what I understand... I was not supposing reiser4 users to be so mutch, but this is very usefull because this means that there are more possibilities to test the code well and to fix bugs... This also means that there are people who feels the need of this filesystem for their work because they are not satisfied with the other filesystems, or because anyway with reiser4 they see that they get better results about the work they need to be done. And this is probably the most important point. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 8:33 ` the ' 'official' point of view' " Luigi Genoni @ 2006-07-27 10:04 ` Adrian Bunk 2006-07-27 11:07 ` Luigi Genoni ` (2 more replies) 0 siblings, 3 replies; 639+ messages in thread From: Adrian Bunk @ 2006-07-27 10:04 UTC (permalink / raw) To: Luigi Genoni Cc: andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 10:33:12AM +0200, Luigi Genoni wrote: > > > > On Thu, July 27, 2006 08:56, Adrian Bunk wrote: > > > >> Said that pretending that KLive data has absolutely no significance at > >> all and that you can't draw any conclusion at all from it, to me seems as > >> wrong as pretending it to perfect. > > > > Possibly wrong conclusions about the general market share based on data > > not having the quality for being the basis of such statements are worse than > > having no data. > > Am I missing something, or it is not marketing what we are talking about? > please I do not understand your point. > > For what I understand... I was not supposing reiser4 users to be so mutch, > but this is very usefull because this means that there are more > possibilities to test the code well and to fix bugs... >... I'm sorry for saying it that directly, and this is not meant against you personally: Your statement is a good example why this data is worse than having no data. As I already said in this thread: <-- snip --> You are able to say that based on klive data, reiser4 has at least 35 users in the world. But you can not tell based on klive data whether the ratio of reiser4:ext3 users in the world is more like 1:5, 1:500 or 1:50000. <-- snip --> I can't prove that the 1:5 ratio is wrong, but the point is that claiming a 1:5 ratio was true based on the klive data is not better than claiming it based on no data. But claiming it based on the klive data is worse since people like you are getting the wrong impression it was based on data that would have the quality for supporting such a statement. The data simply has not the quality for such a statement. Please read my two examples in [1] if you want to get an impression why such problems can occur. cu Adrian [1] http://lkml.org/lkml/2006/7/26/203 -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 10:04 ` Adrian Bunk @ 2006-07-27 11:07 ` Luigi Genoni 2006-07-27 11:35 ` Adrian Bunk 2006-07-27 13:30 ` Horst H. von Brand 2006-07-27 12:21 ` CaT 2006-07-28 2:25 ` Hans Reiser 2 siblings, 2 replies; 639+ messages in thread From: Luigi Genoni @ 2006-07-27 11:07 UTC (permalink / raw) To: Adrian Bunk Cc: Luigi Genoni, andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List Still I am missing something. I am not interested in discussing about 1:5 ora 5:1 or so on. I am not even interested if reiser4 users are 35 or 35000. But if some people felt they need reiser4 to get their work done, that means something. The numbers and the datum per se are meaningless. Anyway you have a datum. Some people need reiser4, period. Why they need reiser4? all of them are just playing with him to have a new game for their spare time? I don't think so. Some of them are using reiser4 because it is the best solution for their work? this is a concrete possibility. again... why that? Because the other FSs are not suitable for certain work? Because the other FSs are suitable, but reiser4 offers better results? Because even reiser4 is not complitelly suitable for some kind of work, but the other FSs are a worser solution? I don't think most reiser4 users decided to give it a try just to enjoy their time making tests just because they are curious. that does not mean reiser4 should be merged into linux kernel because of this. On the other side you can agree reiser4 has its place, because there are users who need it. What we would need to understand, sine ira nec studio, is where this place is. On Thu, July 27, 2006 12:04, Adrian Bunk wrote: > On Thu, Jul 27, 2006 at 10:33:12AM +0200, Luigi Genoni wrote: > >> >> >> >> On Thu, July 27, 2006 08:56, Adrian Bunk wrote: >> >>> >>>> Said that pretending that KLive data has absolutely no significance >>>> at all and that you can't draw any conclusion at all from it, to me >>>> seems as wrong as pretending it to perfect. >>> >>> Possibly wrong conclusions about the general market share based on data >>> not having the quality for being the basis of such statements are >>> worse than having no data. >> >> Am I missing something, or it is not marketing what we are talking about? >> please I do not understand your point. >> >> For what I understand... I was not supposing reiser4 users to be so >> mutch, but this is very usefull because this means that there are more >> possibilities to test the code well and to fix bugs... ... >> > > I'm sorry for saying it that directly, and this is not meant against you > personally: > > > Your statement is a good example why this data is worse than having no > data. > > As I already said in this thread: > > > <-- snip --> > > > You are able to say that based on klive data, reiser4 has at least > 35 users in the world. > > > But you can not tell based on klive data whether the ratio of > reiser4:ext3 users in the world is more like 1:5, 1:500 or 1:50000. > > > <-- snip --> > > > I can't prove that the 1:5 ratio is wrong, but the point is that > claiming a 1:5 ratio was true based on the klive data is not better than > claiming it based on no data. But claiming it based on the klive data is > worse since people like you are getting the wrong impression it was based on > data that would have the quality for supporting such a statement. > > The data simply has not the quality for such a statement. > Please read my two examples in [1] if you want to get an impression why > such problems can occur. > > cu Adrian > > > [1] http://lkml.org/lkml/2006/7/26/203 > > > -- > > > "Is there not promise of rain?" Ling Tan asked suddenly out > of the darkness. There had been need of rain for many days. "Only a promise," > Lao Er said. > Pearl S. Buck - Dragon Seed > > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org More majordomo info at > http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 11:07 ` Luigi Genoni @ 2006-07-27 11:35 ` Adrian Bunk 2006-07-27 11:43 ` Luigi Genoni 2006-07-27 13:30 ` Horst H. von Brand 1 sibling, 1 reply; 639+ messages in thread From: Adrian Bunk @ 2006-07-27 11:35 UTC (permalink / raw) To: Luigi Genoni Cc: andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 01:07:27PM +0200, Luigi Genoni wrote: > Still I am missing something. > > I am not interested in discussing about 1:5 ora 5:1 or so on. > I am not even interested if reiser4 users are 35 or 35000. > But if some people felt they need reiser4 to get their work done, that means > something. The numbers and the datum per se are meaningless. >... You answered to an email where I talked about the wrong assumption the klive data could support the 1:5 market share claim. This is completely independent from discussions about why users might need reiser4, or which technical or social problems prevent its merging (unless of course someone wants to justify its merging wrongly with the 1:5 claim), and therefore your answer does not apply to this part of the thread. cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 11:35 ` Adrian Bunk @ 2006-07-27 11:43 ` Luigi Genoni 2006-07-27 11:56 ` Adrian Bunk 0 siblings, 1 reply; 639+ messages in thread From: Luigi Genoni @ 2006-07-27 11:43 UTC (permalink / raw) To: Adrian Bunk Cc: Luigi Genoni, andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List I answered a mail about how klive data should not be took in account, and could even be dangerous... On Thu, July 27, 2006 13:35, Adrian Bunk wrote: > On Thu, Jul 27, 2006 at 01:07:27PM +0200, Luigi Genoni wrote: > > >> Still I am missing something. >> >> >> I am not interested in discussing about 1:5 ora 5:1 or so on. >> I am not even interested if reiser4 users are 35 or 35000. >> But if some people felt they need reiser4 to get their work done, that >> means something. The numbers and the datum per se are meaningless. ... >> > > You answered to an email where I talked about the wrong assumption the > klive data could support the 1:5 market share claim. > > This is completely independent from discussions about why users might > need reiser4, or which technical or social problems prevent its merging > (unless of course someone wants to justify its merging wrongly with the > 1:5 claim), and therefore your answer does not apply to this part of > the thread. > > cu Adrian > > > -- > > > "Is there not promise of rain?" Ling Tan asked suddenly out > of the darkness. There had been need of rain for many days. "Only a promise," > Lao Er said. > Pearl S. Buck - Dragon Seed > > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 11:43 ` Luigi Genoni @ 2006-07-27 11:56 ` Adrian Bunk 0 siblings, 0 replies; 639+ messages in thread From: Adrian Bunk @ 2006-07-27 11:56 UTC (permalink / raw) To: Luigi Genoni Cc: andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 01:43:53PM +0200, Luigi Genoni wrote: > I answered a mail about how klive data should not be took in account, and > could even be dangerous... ... and you talking about people who might need reiser4 was therefore a wrong answer. I never said reiser4 shouldn't be merged, but this wasn't the topic of this subthread. cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 11:07 ` Luigi Genoni 2006-07-27 11:35 ` Adrian Bunk @ 2006-07-27 13:30 ` Horst H. von Brand 2006-07-27 13:42 ` gmu 2k6 ` (2 more replies) 1 sibling, 3 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-07-27 13:30 UTC (permalink / raw) To: Luigi Genoni Cc: Adrian Bunk, andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List Luigi Genoni <genoni@sns.it> wrote: [...] > Anyway you have a datum. > Some people need reiser4, period. Nope. Some people run kernels that include reiser4. That is all you can infer, and that I knew beforehand. They are at least 35, and that I'd have guessed in any case. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 13:30 ` Horst H. von Brand @ 2006-07-27 13:42 ` gmu 2k6 2006-07-27 13:50 ` andrea 2006-07-27 14:31 ` Luigi Genoni 2 siblings, 0 replies; 639+ messages in thread From: gmu 2k6 @ 2006-07-27 13:42 UTC (permalink / raw) To: Horst H. von Brand Cc: Luigi Genoni, Adrian Bunk, andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On 7/27/06, Horst H. von Brand <vonbrand@inf.utfsm.cl> wrote: > Luigi Genoni <genoni@sns.it> wrote: > > [...] > > > Anyway you have a datum. > > Some people need reiser4, period. > > Nope. Some people run kernels that include reiser4. That is all you can > infer, and that I knew beforehand. They are at least 35, and that I'd have > guessed in any case. 35.5 as I'm testing it here on my workstation and it seems to be faster when you test some things involving many copies of large multi-level sourcetree directories each 3 to 6GiB big in size. 2.6.18-rc2-mm1 with Reiser4 looks ok so far and I had no sync() OOPS like the last time with one -mm revision. speed tells us nothing about reliability of course, but compared to ext3 with dir_index,sparse_super Reiser4 seems to handle "du -sh" and "rm -r" much faster and without eating all of the CPU cycles as it finishes quicker although Reiser4 was meant to be CPU-heavy compared to ext*/reiserfs3. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 13:30 ` Horst H. von Brand 2006-07-27 13:42 ` gmu 2k6 @ 2006-07-27 13:50 ` andrea 2006-07-27 14:31 ` Luigi Genoni 2 siblings, 0 replies; 639+ messages in thread From: andrea @ 2006-07-27 13:50 UTC (permalink / raw) To: Horst H. von Brand Cc: Luigi Genoni, Adrian Bunk, andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 09:30:36AM -0400, Horst H. von Brand wrote: > Nope. Some people run kernels that include reiser4. That is all you can > infer, and that I knew beforehand. They are at least 35, and that I'd have Well, if they were sure they could never get any benefit by reiser4 they wouldn't be testing it in the first place... But certainly the fact they're testing it, doesn't mean they're actually going to use it forever. You can monitor the 35 users live with this link and to see if they increase or decrease over time ;) http://klive.cpushare.com/?order_by=kernel_group&where_machine=all&branch=all&scheduler=all&smp=all&live=live&ip=all They're ordered by cumulative uptime and not by the number of users. For example there are only 10 jfs users but those 10 jfs users have a much larger average uptime (59 days) so they score much higher in the cumulative uptime too. Why the average reiser4 uptime is much lower is unknown, it could be because they shutdown the system by night, or because they upgrade kernel so frequently, or because the system crashes. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 13:30 ` Horst H. von Brand 2006-07-27 13:42 ` gmu 2k6 2006-07-27 13:50 ` andrea @ 2006-07-27 14:31 ` Luigi Genoni 2006-07-27 18:37 ` Jim Crilly 2006-07-27 18:43 ` Horst H. von Brand 2 siblings, 2 replies; 639+ messages in thread From: Luigi Genoni @ 2006-07-27 14:31 UTC (permalink / raw) To: Horst H. von Brand Cc: Luigi Genoni, Adrian Bunk, andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List Since reiser4 is not something enabled by default into a default kernel distribution, I assume they enabled it knowing what they where doing because they wanted to use it. On Thu, July 27, 2006 15:30, Horst H. von Brand wrote: > Luigi Genoni <genoni@sns.it> wrote: > > > [...] > > >> Anyway you have a datum. >> Some people need reiser4, period. >> > > Nope. Some people run kernels that include reiser4. That is all you can > infer, and that I knew beforehand. They are at least 35, and that I'd have > guessed in any case. -- > Dr. Horst H. von Brand User #22616 counter.li.org > Departamento de Informatica Fono: +56 32 654431 > Universidad Tecnica Federico Santa Maria +56 32 654239 > Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org More majordomo info at > http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > > > ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 14:31 ` Luigi Genoni @ 2006-07-27 18:37 ` Jim Crilly 2006-07-27 19:34 ` andrea 2006-07-27 18:43 ` Horst H. von Brand 1 sibling, 1 reply; 639+ messages in thread From: Jim Crilly @ 2006-07-27 18:37 UTC (permalink / raw) To: Luigi Genoni Cc: Horst H. von Brand, Adrian Bunk, andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On 07/27/06 04:31:07PM +0200, Luigi Genoni wrote: > Since reiser4 is not something enabled by default into a default kernel > distribution, I assume they enabled it knowing what they where doing because > they wanted to use it. > > Or because they did 'allmodconfig' or 'allyesconfig'. Whenever I build a kernel I enabled everything possible as a module in case I ever need it. For instance, a few weeks ago I had the reiserfs module loaded because I was testing something, if I had klive running it would have said that I use reiserfs when in fact I don't. Jim. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 18:37 ` Jim Crilly @ 2006-07-27 19:34 ` andrea 0 siblings, 0 replies; 639+ messages in thread From: andrea @ 2006-07-27 19:34 UTC (permalink / raw) To: Luigi Genoni, Horst H. von Brand, Adrian Bunk, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 02:37:33PM -0400, Jim Crilly wrote: > Or because they did 'allmodconfig' or 'allyesconfig'. Whenever I build > a kernel I enabled everything possible as a module in case I ever need > it. For instance, a few weeks ago I had the reiserfs module loaded because > I was testing something, if I had klive running it would have said that I > use reiserfs when in fact I don't. reiserfs would showup in the module list in such case, but _not_ in the fs list. KLive records both the modules loaded _and_ the mounted fs. This is the code that records the FS: if CONFIG_FS: fs = {} for _fs in [ re.search(r'^[^\s]+\s[^\s]+\s([^\s]+)', x).group(1) for x in file('/proc/mounts').readlines() ]: if _fs not in ['rootfs', 'proc', 'devpts']: fs[_fs] = None fs = ' '.join(fs.keys()) KERNEL += str_append(fs) /proc/mounts only lists the _mounted_ fs, not the fs loaded into the kernel statically or with insmod. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 14:31 ` Luigi Genoni 2006-07-27 18:37 ` Jim Crilly @ 2006-07-27 18:43 ` Horst H. von Brand 2006-07-27 20:11 ` andrea 1 sibling, 1 reply; 639+ messages in thread From: Horst H. von Brand @ 2006-07-27 18:43 UTC (permalink / raw) To: Luigi Genoni Cc: Horst H. von Brand, Adrian Bunk, andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List Luigi Genoni <genoni@sns.it> wrote: [...] > Since reiser4 is not something enabled by default into a default kernel > distribution, I assume they enabled it knowing what they where doing because > they wanted to use it. You know, I compile ReiserFS, JFS and XFS (and also CRAMFS and ROMFS) into my self-compiled kernels too. Haven't used any of them in ages (if ever), but one more go at compile-testing won't hurt... -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 18:43 ` Horst H. von Brand @ 2006-07-27 20:11 ` andrea 0 siblings, 0 replies; 639+ messages in thread From: andrea @ 2006-07-27 20:11 UTC (permalink / raw) To: Horst H. von Brand Cc: Luigi Genoni, Adrian Bunk, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 02:43:48PM -0400, Horst H. von Brand wrote: > You know, I compile ReiserFS, JFS and XFS (and also CRAMFS and ROMFS) into > my self-compiled kernels too. Haven't used any of them in ages (if ever), If you never mounted them shortly after boot time, they would have never showed up in KLive fs list if that's what you mean. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 10:04 ` Adrian Bunk 2006-07-27 11:07 ` Luigi Genoni @ 2006-07-27 12:21 ` CaT 2006-07-28 2:25 ` Hans Reiser 2 siblings, 0 replies; 639+ messages in thread From: CaT @ 2006-07-27 12:21 UTC (permalink / raw) To: Adrian Bunk Cc: Luigi Genoni, andrea, J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 12:04:46PM +0200, Adrian Bunk wrote: > I can't prove that the 1:5 ratio is wrong, but the point is that > claiming a 1:5 ratio was true based on the klive data is not better than > claiming it based on no data. But claiming it based on the klive data is > worse since people like you are getting the wrong impression it was > based on data that would have the quality for supporting such a > statement. > > The data simply has not the quality for such a statement. > Please read my two examples in [1] if you want to get an impression why > such problems can occur. > > [1] http://lkml.org/lkml/2006/7/26/203 Also, one may wish to invest in the purchase of 'How to Lie with Statistics'[1] by Darrel Huff (ISBN 0-393-31072-8). An easy yet powerful little read. [1] this is not meant to imply that anyone here is lieing, attempting to lie, thinking of lieing, wishing they were lieing or having anything to do with lieing, purposeful or otherwise. -- "To the extent that we overreact, we proffer the terrorists the greatest tribute." - High Court Judge Michael Kirby ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 10:04 ` Adrian Bunk 2006-07-27 11:07 ` Luigi Genoni 2006-07-27 12:21 ` CaT @ 2006-07-28 2:25 ` Hans Reiser 2006-07-28 10:01 ` Adrian Bunk 2006-07-28 14:05 ` Horst H. von Brand 2 siblings, 2 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-28 2:25 UTC (permalink / raw) To: Adrian Bunk Cc: Luigi Genoni, andrea, J. Bruce Fields, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List Adrian Bunk wrote: > > >But you can not tell based on klive data whether the ratio of >reiser4:ext3 users in the world is more like 1:5, 1:500 or 1:50000. > ><-- snip --> > >I can't prove that the 1:5 ratio is wrong, but the point is that >claiming a 1:5 ratio was true based on the klive data is not better than >claiming it based on no data. > Yes, but I have been surprised that linux conference attendees all know about reiser4, so I think it is consistent with the notion that reiser4 usage may well be much much higher than one would expect of a filesystem that is not in the main tree, and which requires a lot of hassle to install. It is a datapoint. I think the level of hobbyist enthusiasm seen suggests that distros may gain market advantage by adding reiser4 support..... Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-28 2:25 ` Hans Reiser @ 2006-07-28 10:01 ` Adrian Bunk 2006-07-28 14:05 ` Horst H. von Brand 1 sibling, 0 replies; 639+ messages in thread From: Adrian Bunk @ 2006-07-28 10:01 UTC (permalink / raw) To: Hans Reiser Cc: Luigi Genoni, andrea, J. Bruce Fields, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 08:25:23PM -0600, Hans Reiser wrote: > Adrian Bunk wrote: > > > > > > >But you can not tell based on klive data whether the ratio of > >reiser4:ext3 users in the world is more like 1:5, 1:500 or 1:50000. > > > ><-- snip --> > > > >I can't prove that the 1:5 ratio is wrong, but the point is that > >claiming a 1:5 ratio was true based on the klive data is not better than > >claiming it based on no data. > > > Yes, but I have been surprised that linux conference attendees all know > about reiser4, so I think it is consistent with the notion that reiser4 > usage may well be much much higher than one would expect of a filesystem > that is not in the main tree, >... There were already several long flamewars^Wdiscussions about reiser4 on linux-kernel. There has already been some amount of press articles covering reiser4. But how much usage is the result of this? There seem to be the the following possibilities how users get reiser4 into their kernel today: 1. -mm kernel 2. non -mm kernel a) patch downloaded from namesys.com b) through their distribution - in a kernel image - as patch c) patch forwarded through other channels 2b) seems to be the only number that is easily measurable. Do you have some download statistics for namesys.com? Multiplying the downloads for the 2.6.16 patches with 10 seems to be the best estimate for the order of magnitude of current reiser4 users that seems to be available. > Hans cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-28 2:25 ` Hans Reiser 2006-07-28 10:01 ` Adrian Bunk @ 2006-07-28 14:05 ` Horst H. von Brand 1 sibling, 0 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-07-28 14:05 UTC (permalink / raw) To: Hans Reiser Cc: Adrian Bunk, Luigi Genoni, andrea, J. Bruce Fields, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List Hans Reiser <reiser@namesys.com> wrote: > Adrian Bunk wrote: > >But you can not tell based on klive data whether the ratio of > >reiser4:ext3 users in the world is more like 1:5, 1:500 or 1:50000. > > > ><-- snip --> > > > >I can't prove that the 1:5 ratio is wrong, but the point is that > >claiming a 1:5 ratio was true based on the klive data is not better than > >claiming it based on no data. > Yes, but I have been surprised that linux conference attendees all know > about reiser4, so I think it is consistent with the notion that reiser4 > usage may well be much much higher than one would expect of a filesystem > that is not in the main tree, and which requires a lot of hassle to > install. It is a datapoint. I think the level of hobbyist enthusiasm > seen suggests that distros may gain market advantage by adding reiser4 > support..... Great! Then /sit down and do the work to get it into the vanilla kernel/. You know then it is not a wasted effort... and you /do/ know by now (or should) that playing the "politics"/"whining" card goes nowhere. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 6:56 ` Adrian Bunk 2006-07-27 8:33 ` the ' 'official' point of view' " Luigi Genoni @ 2006-07-27 11:52 ` andrea 2006-07-27 12:18 ` Adrian Bunk 2006-07-28 1:47 ` Hans Reiser 1 sibling, 2 replies; 639+ messages in thread From: andrea @ 2006-07-27 11:52 UTC (permalink / raw) To: Adrian Bunk Cc: J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 08:56:03AM +0200, Adrian Bunk wrote: > On Wed, Jul 26, 2006 at 11:17:41PM +0200, andrea@cpushare.com wrote: > > On Wed, Jul 26, 2006 at 10:50:22PM +0200, Adrian Bunk wrote: > > > But depending on the nature of the error, the worst case might be the > > > common case (as I've already explained in another email). > > > > > > If you can't ensure the quality of your data, please don't use this data > > > to wrongly draw any conclusions from them [1]. > > > > Please read the footer of the KLive pages: > > > > "The use of the information and of the software in this website is at > > your own risk. KLive probably doesn't represent a reliable sample of > > the real usage of the Linux Kernel." > > It was you who wrongly said: > "With KLive I can attempt to estimate market share of _kernel_ code" > > Hadn't you read your own disclaimer? There is no contradiction in the two statements. To attempt to estimate something I don't need a reliable sample of the whole population. Estimation is still a statistical thing. Also I said attempt to estimate, it doesn't mean I will make it. If you don't consider those results a positive for reiser4, it can only mean you expected reiser4 to have a much higher share among the KLive users. This is obvious. > Every time someone will repeat the "1:5 ratio for reiser4:ext3 users", > this will be an additional proof it's really worse than no data. If they say "1:5 ratio for reiser4:ext3 KLive users" everything will be correct and nobody can object because it's a fact. I said myself that I'm no reiserfs user, and I don't plan to become one any time soon (especially on my production systems), I'm only reporting plain numbers as KLive measured the stuff. I'm surprised as much as you are, but then I've to report facts, and not my own opinions. As far as I'm concerned the thing I like less of reiser4 is the plugin thing, I'd be less concerned if that was a microkernel (fuse-like) userland plugin system. Anyway with time perhaps things can change and become userland based, and the stuff can be moved into vfs if that code really belongs there as some kernel developer says. That doesn't mean reiser4 can't be merged first and the stuff moved into vfs later. xfs when was merged also pratically rewrote a vfs internally that was meant to work with irix, if Christoph didn't complain about xfs being merged, I don't see what's the problem of reiser4 being merged even if it rewrites some part of vfs. xfs is also still having various special features like the pinhole one that only belongs to the vfs instead but nobody complains. And if reiser4 is really so bad as they say, once people starts losing data they will spread the word of not using it. As long as it's marked experimental I don't see a big issue, the wireless driver for broadcom chip will eat your filesystem too if the reverse engineered dma operations writes into a buffer header instead of an skb. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 11:52 ` the " 'official' point of view" " andrea @ 2006-07-27 12:18 ` Adrian Bunk 2006-07-27 13:10 ` andrea 2006-07-28 1:47 ` Hans Reiser 1 sibling, 1 reply; 639+ messages in thread From: Adrian Bunk @ 2006-07-27 12:18 UTC (permalink / raw) To: andrea Cc: J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 01:52:29PM +0200, andrea@cpushare.com wrote: > On Thu, Jul 27, 2006 at 08:56:03AM +0200, Adrian Bunk wrote: >... > > It was you who wrongly said: > > "With KLive I can attempt to estimate market share of _kernel_ code" > > > > Hadn't you read your own disclaimer? > > There is no contradiction in the two statements. To attempt to > estimate something I don't need a reliable sample of the whole > population. Estimation is still a statistical thing. Also I said Sure, you don't need any data to estimate anything. But if your estimate is based on bad data it sounds better than if it was based on no data although the value of the estimate is the same. > attempt to estimate, it doesn't mean I will make it. Giving low quality raw data and then saying "it wasn't me who did the estimate" is silly. > If you don't consider those results a positive for reiser4, it can > only mean you expected reiser4 to have a much higher share among the > KLive users. This is obvious. That's completely wrong. I consider those results neither positive nor negative for reiser4. They could only be considered positive if someone expected less than 35 reiser4 users worldwide. > > Every time someone will repeat the "1:5 ratio for reiser4:ext3 users", > > this will be an additional proof it's really worse than no data. > > If they say "1:5 ratio for reiser4:ext3 KLive users" everything will > be correct and nobody can object because it's a fact. And 90% of all people will misread it. And many people will spread it wrong. That's exactly where such data is worse than no data. You could claim you know your the suboptimal quality of the data and that you had a disclaimer. But did you read Hans' reaction on your email? If you are intelligent (which I assume), you should have learned by this how to not present your data. > I said myself that I'm no reiserfs user, and I don't plan to become > one any time soon (especially on my production systems), I'm only > reporting plain numbers as KLive measured the stuff. I'm surprised as > much as you are, but then I've to report facts, and not my own > opinions. >... I have no plans to use reiser4 myself (I'm a happy ext2 user). But I'm not against mergng reiser4, either. I'll leave this technical discussion to the people who know more about this. cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 12:18 ` Adrian Bunk @ 2006-07-27 13:10 ` andrea 2006-07-27 13:58 ` Adrian Bunk 0 siblings, 1 reply; 639+ messages in thread From: andrea @ 2006-07-27 13:10 UTC (permalink / raw) To: Adrian Bunk Cc: J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 02:18:11PM +0200, Adrian Bunk wrote: > They could only be considered positive if someone expected less than > 35 reiser4 users worldwide. The only thing we know for sure is that 35 out of 500 KLive user are running reiser4, worldwide we have no clue. > If you are intelligent (which I assume), you should have learned by this > how to not present your data. I think readers are intelligent enough too to interpret the KLive data properly for themself, without you having to prevent their eyes to see the raw KLive data. When you tell me how I should not present my data, you're asking me to censor part or all of the very output of the KLive project. I'd rather wipe out KLive completely, than to censor it. The way I presented it was absolutely not biased, if you can make more transparent and unbiased sql queries than the ones I did, please post them and I'll be glad to run them. > [..] (I'm a happy ext2 user). Oh my, I hope you're only choosing the fs for your own workstation. Even though I don't pretend to fully understand someone who claims to be happy with ext2, I've no idea why you hate so much the stuff running at cpushare.com domain. But if it helps KLive is actually one of the non commercial projects I'm hosting there, and the only reason I keep it there, is to be sure not to find it filled by ads. Now let's try to get some work done instead of only sending emails ;) ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 13:10 ` andrea @ 2006-07-27 13:58 ` Adrian Bunk 2006-07-27 14:45 ` andrea 0 siblings, 1 reply; 639+ messages in thread From: Adrian Bunk @ 2006-07-27 13:58 UTC (permalink / raw) To: andrea Cc: J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 03:10:32PM +0200, andrea@cpushare.com wrote: > On Thu, Jul 27, 2006 at 02:18:11PM +0200, Adrian Bunk wrote: > > They could only be considered positive if someone expected less than > > 35 reiser4 users worldwide. > > The only thing we know for sure is that 35 out of 500 KLive user are > running reiser4, worldwide we have no clue. > > > If you are intelligent (which I assume), you should have learned by this > > how to not present your data. > > I think readers are intelligent enough too to interpret the KLive data > properly for themself, without you having to prevent their eyes to see > the raw KLive data. > > When you tell me how I should not present my data, you're asking me to > censor part or all of the very output of the KLive project. I'd rather > wipe out KLive completely, than to censor it. The way I presented it > was absolutely not biased, if you can make more transparent and > unbiased sql queries than the ones I did, please post them and I'll be > glad to run them. What about the following statement: "Gentoo is 47 times as popular as SuSE among KLive users (a service offered by a SuSE employee gathering data from many users worldwide)." Is any part of this information wrong? Is it therefore OK to put this information to /. ? Surely not, but you might get the point. > > [..] (I'm a happy ext2 user). > > Oh my, I hope you're only choosing the fs for your own workstation. > > Even though I don't pretend to fully understand someone who claims to > be happy with ext2, It's stable, and if the machine crashes the fsck is really great (Linus' tree contains at least a dozen patches that had their temporary home in my lost+found). (I don't care that the fsck takes 20 minutes on a 250 GB partition.) Other people have other preferences and do therefore prefer other filesystems. > I've no idea why you hate so much the stuff > running at cpushare.com domain. But if it helps KLive is actually one > of the non commercial projects I'm hosting there, and the only reason > I keep it there, is to be sure not to find it filled by ads. This isn't against cpushare.com, it's against publishing things people will easily misinterpret. The Linux Counter [1] is another non commercial project gathering similar data (including live data about running kernels and uptimes) from far more users, and I've already given an example in this thread how someone could have misinterpreted the data offered there. And if someone would argue with data from there in a similar easy to misinterpret way on this list, I'd react similarly. And another example would be that you could say "one third of all Linux Counter machines are still running kernels < 2.6". This would be based on current information submitted automatically from nearly 5000 machines. But people would read this is "one third of all machines are still running kernels < 2.6". But the data doesn't have the quality for this generalization. And this is exactly the problem. > Now let's try to get some work done instead of only sending emails ;) Agreed. ;-) cu Adrian [1] http://counter.li.org/ -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 13:58 ` Adrian Bunk @ 2006-07-27 14:45 ` andrea 2006-07-27 15:05 ` Adrian Bunk 0 siblings, 1 reply; 639+ messages in thread From: andrea @ 2006-07-27 14:45 UTC (permalink / raw) To: Adrian Bunk Cc: J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 03:58:23PM +0200, Adrian Bunk wrote: > What about the following statement: > > "Gentoo is 47 times as popular as SuSE among KLive users (a service > offered by a SuSE employee gathering data from many users worldwide)." > > Is any part of this information wrong? > > Is it therefore OK to put this information to /. ? > > Surely not, but you might get the point. I already told you once that such a claim is totally bogus because KLive _doesn't_ record anything about the _userland_, it only records kernel stuff. It's not Dlive (distributon Live), it's KLive and K stands for _kernel_ not distributions. Not sure how many times I'll have to remind you about this. Also note currently the current cumulative uptime ratio is 1:18 not 1:47 and even the ratio of the number of users is 1:42. The only claim based on actual facts you can try to publish on the press is this: "The Gentoo kernels are 18 times more popular than the SUSE kernels among KLive users (a service offered in his spare time by a SUSE contractor that tries to gather data from many users worldwide)." You can also add: "Another data point is that among the KLive users the Fedora kernels are 24 times less popular then Gentoo ones." Free to advertise the above if you find it interesting. > This isn't against cpushare.com, it's against publishing things people > will easily misinterpret. I think people deserve to think for themself without you acting as censorship filter for them. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 14:45 ` andrea @ 2006-07-27 15:05 ` Adrian Bunk 2006-07-27 16:11 ` andrea 0 siblings, 1 reply; 639+ messages in thread From: Adrian Bunk @ 2006-07-27 15:05 UTC (permalink / raw) To: andrea Cc: J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 04:45:29PM +0200, andrea@cpushare.com wrote: > On Thu, Jul 27, 2006 at 03:58:23PM +0200, Adrian Bunk wrote: > > What about the following statement: > > > > "Gentoo is 47 times as popular as SuSE among KLive users (a service > > offered by a SuSE employee gathering data from many users worldwide)." > > > > Is any part of this information wrong? > > > > Is it therefore OK to put this information to /. ? > > > > Surely not, but you might get the point. > > I already told you once that such a claim is totally bogus because > KLive _doesn't_ record anything about the _userland_, it only records > kernel stuff. > > It's not Dlive (distributon Live), it's KLive and K stands for > _kernel_ not distributions. > > Not sure how many times I'll have to remind you about this. > > > Also note currently the current cumulative uptime ratio is 1:18 not > 1:47 and even the ratio of the number of users is 1:42. The only claim > based on actual facts you can try to publish on the press is this: The 1:47 was what I calculated yesterday, but it seems to be 1:42 now. But I'd measure popularity in current usage, not in cumulative uptime. > "The Gentoo kernels are 18 times more popular than the SUSE kernels > among KLive users (a service offered in his spare time by a SUSE > contractor that tries to gather data from many users worldwide)." >... Slightly modified version is now in my .signature (I omitted the "spare time" part since the information is still true and sounds better without it - it would be cencorship if you'd try to force me to not omit it). cu Adrian -- The Gentoo kernels are 42 times more popular than the SUSE kernels among KLive users (a service by SuSE contractor Andrea Arcangeli that gathers data about kernels from many users worldwide). ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 15:05 ` Adrian Bunk @ 2006-07-27 16:11 ` andrea 2006-07-28 2:25 ` Hans Reiser 0 siblings, 1 reply; 639+ messages in thread From: andrea @ 2006-07-27 16:11 UTC (permalink / raw) To: Adrian Bunk Cc: J. Bruce Fields, Hans Reiser, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List On Thu, Jul 27, 2006 at 05:05:08PM +0200, Adrian Bunk wrote: > Slightly modified version is now in my .signature (I omitted the > "spare time" part since the information is still true and sounds better > without it - it would be cencorship if you'd try to force me to not > omit it). Well, a signature like that, perfectly match the IQ of who writes it, please add a link to the klive homepage too if you don't mind. This IMHO would be even more hilarious: "I'm happy with ext2" by Adrain Bunk, the guy who pretends to provide useful feedback in the reiser4 discussion. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 16:11 ` andrea @ 2006-07-28 2:25 ` Hans Reiser 2006-07-28 14:31 ` andrea 0 siblings, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-07-28 2:25 UTC (permalink / raw) To: andrea Cc: Adrian Bunk, J. Bruce Fields, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List Adrian and Andrea, you both write a lot of useful patches, so let's remember that, and try to all get along..... Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-28 2:25 ` Hans Reiser @ 2006-07-28 14:31 ` andrea 0 siblings, 0 replies; 639+ messages in thread From: andrea @ 2006-07-28 14:31 UTC (permalink / raw) To: Hans Reiser Cc: Adrian Bunk, J. Bruce Fields, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List > remember that, and try to all get along..... FWIW the last thing I can be interested about is to get along with somebody that gratuitously attacks me every time he can. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 11:52 ` the " 'official' point of view" " andrea 2006-07-27 12:18 ` Adrian Bunk @ 2006-07-28 1:47 ` Hans Reiser 1 sibling, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-28 1:47 UTC (permalink / raw) To: andrea Cc: Adrian Bunk, J. Bruce Fields, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List andrea@cpushare.com wrote: > > >As far as I'm concerned the thing I like less of reiser4 is the plugin >thing, I'd be less concerned if that was a microkernel (fuse-like) >userland plugin system. > Performance. If your plugin is performance valuing, it needs to be in kernel. Also, FUSE does not have per-file plugins (correct me if I err here). It would be nice to see it pay attention to reiser4 pluginids, and thus become per-file. Thanks for your other words, which I will not comment on because I agree with them.;-) ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 20:50 ` Adrian Bunk 2006-07-26 21:17 ` andrea @ 2006-07-27 4:35 ` Hans Reiser 2006-07-27 13:26 ` Horst H. von Brand 1 sibling, 1 reply; 639+ messages in thread From: Hans Reiser @ 2006-07-27 4:35 UTC (permalink / raw) To: Adrian Bunk Cc: andrea, J. Bruce Fields, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List Adrian Bunk wrote: >[1] the conclusion itself might or might not be true > e.g. there _could_ be an 1:5 ratio between reiser4 and ext3 users > but your data is not in any way able to support or reject this > statement > > It does however suggest that my surprise at how people at the last Linux Conference I went to all seemed to know that there exists a Reiser4 may be due to it being more widely used than I would have guessed. Maybe there is some coolness factor to having the faster FS that you can't get from any Distro that is enough to overcome the hassle of compiling reiser4progs and a kernel before inserting the DVD. I would not have guessed we had 1/5th of ext3's usage even among lkml readers..... I guess the market contains more people who like technology than I was guessing. Maybe there is a positive word of mouth effect going on too. It would be nice if SuSE and others at least made it an unsupported option at install time. I shall have to find the time to go asking them all..... Hans ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-27 4:35 ` Hans Reiser @ 2006-07-27 13:26 ` Horst H. von Brand 0 siblings, 0 replies; 639+ messages in thread From: Horst H. von Brand @ 2006-07-27 13:26 UTC (permalink / raw) To: Hans Reiser Cc: Adrian Bunk, andrea, J. Bruce Fields, Nikita Danilov, Rene Rebe, Linux Kernel Mailing List Hans Reiser <reiser@namesys.com> wrote: > Adrian Bunk wrote: > >[1] the conclusion itself might or might not be true > > e.g. there _could_ be an 1:5 ratio between reiser4 and ext3 users > > but your data is not in any way able to support or reject this > > statement > It does however suggest that my surprise at how people at the last > Linux Conference I went to all seemed to know that there exists a > Reiser4 I'd be rather surprised to find someone at a Linux conference who hasn't at least heard of the recurrent flamewars on the topic here... > may be due to it being more widely used than I would have > guessed. Maybe there is some coolness factor to having the faster FS > that you can't get from any Distro that is enough to overcome the hassle > of compiling reiser4progs and a kernel before inserting the DVD. Not seen any data backing up the "faster", let alone so much faster that the hassle (and the risk) would make it worth trying... No, Gentoo folks don't count, around here they are fond of claiming that their self-compiled systems are at least twice as fast as a binary distribution with the exact same software. > I > would not have guessed we had 1/5th of ext3's usage even among lkml > readers..... I'd guess something of the order of 1/100, for testing purposes and idle curiosity. > I guess the market contains more people who like > technology than I was guessing. Maybe there is a positive word of mouth > effect going on too. LKML (and Linux conferences) are exactly the places where you /only/ find this kind of people... > It would be nice if SuSE and others at least made it an unsupported > option at install time. I shall have to find the time to go asking them > all..... At least Fedora is trying hard to just follow upstream packages (in this case, Linus' kernels) with the absolute minimum of local patches. It makes good sense, as it reduces the up-front work, and (more important) minimizes the pain when the later official version is somehow incompatible with the previews. If you want ReiserFS 4 to get more exposure, do the legwork to get it into the official kernel. Distributions should be reluctant to pick it up as long as its status as an official Linux filesystem is in question. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-22 13:02 ` Theodore Tso ` (2 preceding siblings ...) 2006-07-23 7:20 ` Rene Rebe @ 2006-07-24 2:08 ` Steve Lord 2006-07-24 7:53 ` Nikita Danilov 3 siblings, 1 reply; 639+ messages in thread From: Steve Lord @ 2006-07-24 2:08 UTC (permalink / raw) To: Theodore Tso, Hans Reiser, LKML Theodore Tso wrote: > On Fri, Jul 21, 2006 at 01:46:18PM -0600, Hans Reiser wrote: >> Consider what happened with XFS as the article writer mentions. I met >> the original XFS team, led by two very senior developers (Jim Grey, and >> another fellow whose name I am blanking on, forgive me, I learned much >> from him in just a few conversations). Not sure who Jim Grey was, he never worked on XFS, ah well. > > I believe you are referring to Jim Mostek and Steve Lord, and yes, > they were very talented developers and engineers. I very much enjoyed > talking to them at various filesystem and Linux conferences and > workshops. > >> supervision. What happened? They got hassled. Instead of learning >> from them, welcoming into our community two very senior developers who >> knew a lot more than any of us about the topics they chose to speak >> about, they got hassled, they get ignored, they felt rejected, and left >> the Linux community forever, never to return. > > That's hardly what happened. SGI went through layoffs, and they were > hit. See: http://slashdot.org/articles/01/05/26/0743254.shtml > Ted, you of all people should know not believe all you read on slashdot ;-) 'Linuxcare helping out with the funding' ha! Both Jim Mostek and I left under our own steam at different times, Jim in 2000 and myself in 2003. SGI still has great technology to work on and, but the you can only take so many years of bad financial results and watching people get layed off. I still work on Linux, and follow development as much as I can. I keep trying to get back to OLS, but circumstances keep conspiring against me, maybe next year. Steve Lord ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 2:08 ` Steve Lord @ 2006-07-24 7:53 ` Nikita Danilov 2006-07-24 10:30 ` Theodore Tso 0 siblings, 1 reply; 639+ messages in thread From: Nikita Danilov @ 2006-07-24 7:53 UTC (permalink / raw) To: Steve Lord; +Cc: Linux Kernel Mailing List Steve Lord writes: > Theodore Tso wrote: > > On Fri, Jul 21, 2006 at 01:46:18PM -0600, Hans Reiser wrote: > > >> Consider what happened with XFS as the article writer mentions. I met > >> the original XFS team, led by two very senior developers (Jim Grey, and > >> another fellow whose name I am blanking on, forgive me, I learned much > >> from him in just a few conversations). > > Not sure who Jim Grey was, he never worked on XFS, ah well. I believe the (mis-)reference is to a famous data-base person, co-author of "Transaction Processing". He is with Microsoft now (http://research.microsoft.com/~Gray/JimGrayHomePageSummary.htm). > > Steve Lord Nikita. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 7:53 ` Nikita Danilov @ 2006-07-24 10:30 ` Theodore Tso 2006-07-24 11:35 ` Olivier Galibert 2006-07-25 21:44 ` Valdis.Kletnieks 0 siblings, 2 replies; 639+ messages in thread From: Theodore Tso @ 2006-07-24 10:30 UTC (permalink / raw) To: Nikita Danilov; +Cc: Steve Lord, Linux Kernel Mailing List On Mon, Jul 24, 2006 at 11:53:08AM +0400, Nikita Danilov wrote: > > I believe the (mis-)reference is to a famous data-base person, co-author > of "Transaction Processing". He is with Microsoft now > (http://research.microsoft.com/~Gray/JimGrayHomePageSummary.htm). > That's what I thought when I saw the name Jim Gray, but as far as I knew he never worked on XFS and never left the Linux community dejected because Linux kernel coding standards requirements before changes were allowed to be merged, when Hans did his name dropping thing. (I mean geez, if you want really high standards before new code is accepted, take a look at Open Solaris; they have *such* a heavyweight process, with two mandatory signoffs by core Solaris engineers who both have to do a line-by-line review, and with a promise of on-disk and ABI compatibility *forever* ---- that we do more commits in a week than they do in a year....) - Ted ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 10:30 ` Theodore Tso @ 2006-07-24 11:35 ` Olivier Galibert 2006-07-24 13:39 ` Theodore Tso 2006-07-25 21:44 ` Valdis.Kletnieks 1 sibling, 1 reply; 639+ messages in thread From: Olivier Galibert @ 2006-07-24 11:35 UTC (permalink / raw) To: Linux Kernel Mailing List; +Cc: Theodore Tso, Nikita Danilov, Steve Lord On Mon, Jul 24, 2006 at 06:30:23AM -0400, Theodore Tso wrote: > (I mean geez, if you want really high standards before new code is > accepted, take a look at Open Solaris; they have *such* a heavyweight > process, with two mandatory signoffs by core Solaris engineers who > both have to do a line-by-line review, and with a promise of on-disk > and ABI compatibility *forever* ---- that we do more commits in a week > than they do in a year....) That sounds almost like gcc, only worse. I think there is something of a problem currently, tough. It is getting too hard to get code in if you're not a maintainer for an existing subsystem (reiser4, suspend2...), and too easy when you're a maintainer (ext4, uswsusp...). Ext patches don't get reviewed much outside of the developpers, and they go in pretty much without discussion in any case, except when Linus blows a fuse. Reiser4 would have be in without discussion if it had been a set of patches through time to reiser3, and would have been called 4 only when Linus yelled. I suspect some balancing would be useful. OG. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 11:35 ` Olivier Galibert @ 2006-07-24 13:39 ` Theodore Tso 2006-07-24 15:38 ` Olivier Galibert 0 siblings, 1 reply; 639+ messages in thread From: Theodore Tso @ 2006-07-24 13:39 UTC (permalink / raw) To: Olivier Galibert, Linux Kernel Mailing List, Nikita Danilov, Steve Lord On Mon, Jul 24, 2006 at 01:35:34PM +0200, Olivier Galibert wrote: >Ext patches don't get reviewed much > outside of the developpers, and they go in pretty much without > discussion in any case, except when Linus blows a fuse. Um, you're kidding, right? We certainly don't make the assumption that we can violate CodingStyle willy nilly and stuff in yacc grammers into ext3 and assume that no one will push back. In fact we did a lot of work to make sure the patches were clean and mostly ready to be accepted to mainline even before we made the first proposal to push extents to LKML. > I think there is something of a problem currently, tough. It is > getting too hard to get code in if you're not a maintainer for an > existing subsystem (reiser4, suspend2...), and too easy when you're a > maintainer (ext4, uswsusp...). It's not fair to assume that the only reason why non-maintainers have a harder time getting changes is because their changes are getting more intensive review. (Although it is the case that we probably do need to get better at reviewing changes that go in via git trees.) A much more important effect is that non-maintainers aren't familiar with coding and patch submission guidelines. For example, in suspend2, Nigel first tried with patches that were too monolithic, and then his next series was too broken down such that it was too hard to review (and "git bisect" wouldn't work). And of course, there are people who assume that the rules shouldn't apply to their filesystem... - Ted ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 13:39 ` Theodore Tso @ 2006-07-24 15:38 ` Olivier Galibert 2006-07-24 16:17 ` Theodore Tso 2006-07-26 13:08 ` Pavel Machek 0 siblings, 2 replies; 639+ messages in thread From: Olivier Galibert @ 2006-07-24 15:38 UTC (permalink / raw) To: Theodore Tso, Linux Kernel Mailing List, Nikita Danilov, Steve Lord On Mon, Jul 24, 2006 at 09:39:39AM -0400, Theodore Tso wrote: > On Mon, Jul 24, 2006 at 01:35:34PM +0200, Olivier Galibert wrote: > >Ext patches don't get reviewed much > > outside of the developpers, and they go in pretty much without > > discussion in any case, except when Linus blows a fuse. > > Um, you're kidding, right? We certainly don't make the assumption > that we can violate CodingStyle willy nilly and stuff in yacc grammers > into ext3 and assume that no one will push back. I'm not kidding. I do recognise that the ext* maintainers do a very good and clean job. > In fact we did a lot of work to make sure the patches were clean and > mostly ready to be accepted to mainline even before we made the first > proposal to push extents to LKML. I'm no talking about extends only. Ext3 now is very, very different than the ext2+journal it was at the start, with backwards-incompatible format changes added all the time[1]. These changes went in no-questions-asked. > > I think there is something of a problem currently, tough. It is > > getting too hard to get code in if you're not a maintainer for an > > existing subsystem (reiser4, suspend2...), and too easy when you're a > > maintainer (ext4, uswsusp...). > > It's not fair to assume that the only reason why non-maintainers have > a harder time getting changes is because their changes are getting > more intensive review. "only", no, definitively not. The impact is non-negligible though. > (Although it is the case that we probably do > need to get better at reviewing changes that go in via git trees.) Ohh yes, a lot better. Just look at ALSA, most of SNDRV_HWDEP* should never have gotten in in the first place, especially some recent ones like SNDRV_HWDEP_IFACE_SB_RC, and even some (SNDRV_HWDEP_IFACE_USX2Y*) are security holes the size of Cleveland. I need to continue documenting the Alsa kernel interface, but I need a new bucket, the first one is overflowing. > A much more important effect is that non-maintainers aren't familiar > with coding and patch submission guidelines. For example, in > suspend2, Nigel first tried with patches that were too monolithic, > and then his next series was too broken down such that it was too > hard to review (and "git bisect" wouldn't work). All his submissions since 2004 or so? It's a little easy to limit oneself to the last two ones. > And of course, there are people who assume that the rules shouldn't > apply to their filesystem... It may be a little hard to remove XFS at that point though... And, while it's not a filesystem, I'd love to be pointed to the technical discussion deciding whether uswsusp is a good idea. OG. [1] All optional, I know. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 15:38 ` Olivier Galibert @ 2006-07-24 16:17 ` Theodore Tso 2006-07-24 17:50 ` Olivier Galibert 2006-07-26 13:08 ` Pavel Machek 1 sibling, 1 reply; 639+ messages in thread From: Theodore Tso @ 2006-07-24 16:17 UTC (permalink / raw) To: Olivier Galibert, Linux Kernel Mailing List, Nikita Danilov, Steve Lord On Mon, Jul 24, 2006 at 05:38:53PM +0200, Olivier Galibert wrote: > I'm no talking about extends only. Ext3 now is very, very different > than the ext2+journal it was at the start, with backwards-incompatible > format changes added all the time[1]. These changes went in > no-questions-asked. The patches indeed were reviewed and changes made in response to the reviews. The philosophical/design question about whether or not optional features which, if enabled, would prevent older kernels to mount the filesystem was not asked, no. But that doesn't mean that the code was not reviewed; it was. I would also note that we didn't intimate that we knew better than the reviewers, or question their motives, or otherwise insult the reviewers such that they might decide they have better things to do than to review our patches, and that might have had something to do with how the code got in relatively painlessly.... - Ted ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 16:17 ` Theodore Tso @ 2006-07-24 17:50 ` Olivier Galibert 2006-07-24 18:29 ` Jeff Garzik 0 siblings, 1 reply; 639+ messages in thread From: Olivier Galibert @ 2006-07-24 17:50 UTC (permalink / raw) To: Theodore Tso, Linux Kernel Mailing List, Nikita Danilov, Steve Lord On Mon, Jul 24, 2006 at 12:17:55PM -0400, Theodore Tso wrote: > I would also note that we didn't intimate that we knew better than the > reviewers, or question their motives, or otherwise insult the > reviewers such that they might decide they have better things to do > than to review our patches, and that might have had something to do > with how the code got in relatively painlessly.... Now that has a very high impact :-) Hans is not very good at the diplomacy game. The fact that the ext maintainers are very, very good helps quite a lot too. But I think it doesn't change the fact that if r4 has been a set of patches through time to r3, good or not, there wouldn't be a discussion. It's maybe the lack of an official development branch, but it looks like the kernel development has become very risk-averse, and the bar is set much higher to accept anything that looks relatively new. Any reason is good to have it dropped, cosmetic or not. Just to give you an idea, if the criteria applied to suspend2 or reiser4 had been applied to everything else, we wouldn't have at least XFS[1], ALSA[2], sysfs[3] and DRM[4]. Whether it is good or bad is an interesting question itself. But before, code just had to be reasonably sane, and it was expected to be fixed through time. Some even has been (sysfs got better). Now it has to attain an ever moving level of perfection before it has a chanc to be accepted. Unless you're a maintainer, that is, for which you can get pretty much anything in that doesn't immediatly break the compile through git trees. Thankfully, most of the maintainers are sane. Don't you think this is a problem? OG. [1] CodingStyle is for the weak [2] You call that a kernel interface? HWDEP? ioctl to read/write samples instead of read/write? [3] Lifetime rules and locking are for other people to care about [4] Oh my, people are still trying to fix that one ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 17:50 ` Olivier Galibert @ 2006-07-24 18:29 ` Jeff Garzik 0 siblings, 0 replies; 639+ messages in thread From: Jeff Garzik @ 2006-07-24 18:29 UTC (permalink / raw) To: Olivier Galibert, Theodore Tso, Linux Kernel Mailing List, Nikita Danilov, Steve Lord Olivier Galibert wrote: > The fact that the ext maintainers are very, very good helps quite a > lot too. But I think it doesn't change the fact that if r4 has been a > set of patches through time to r3, good or not, there wouldn't be a > discussion. That's a huuuuuge leap of logic. Metadata plugins found in reiser4 are far better done at the VFS level, than burying "reiser4 can look like ext2, if it wishes" functionality inside the filesystem. I guarantee such a patch to reiser3 would get rejected. > It's maybe the lack of an official development branch, but it looks > like the kernel development has become very risk-averse, and the bar > is set much higher to accept anything that looks relatively new. Any > reason is good to have it dropped, cosmetic or not. New stuff goes in all the time. The bar is set too high in some cases (read: SCSI subsystem submissions), but reiser4 submission cannot be generalized as you have done here. There are very real issues present, that need to be dealt with. > Just to give you an idea, if the criteria applied to suspend2 or > reiser4 had been applied to everything else, we wouldn't have at least > XFS[1], ALSA[2], sysfs[3] and DRM[4]. Whether it is good or bad is an > interesting question itself. But before, code just had to be > reasonably sane, and it was expected to be fixed through time. Some > even has been (sysfs got better). Now it has to attain an ever moving > level of perfection before it has a chanc to be accepted. reiser4 tries to be another VFS. That's a bit more than needing additional minor fixes over time. Jeff ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 15:38 ` Olivier Galibert 2006-07-24 16:17 ` Theodore Tso @ 2006-07-26 13:08 ` Pavel Machek 2006-07-27 15:52 ` Olivier Galibert 2006-07-27 16:43 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Alan Cox 1 sibling, 2 replies; 639+ messages in thread From: Pavel Machek @ 2006-07-26 13:08 UTC (permalink / raw) To: Olivier Galibert, Theodore Tso, Linux Kernel Mailing List, Nikita Danilov, Steve Lord Hi! > > A much more important effect is that non-maintainers aren't familiar > > with coding and patch submission guidelines. For example, in > > suspend2, Nigel first tried with patches that were too monolithic, > > and then his next series was too broken down such that it was too > > hard to review (and "git bisect" wouldn't work). > > All his submissions since 2004 or so? It's a little easy to limit > oneself to the last two ones. Nigel did not do any submissions in 2004 or so. Check your fact, that stuff was marked 'RFC' and yes I did comment on it. He did 1 (one) submission that looked like SubmittingPatches at the first sight, and that was very recent. Stop spreading lies. Pavel -- Thanks for all the (sleeping) penguins. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 13:08 ` Pavel Machek @ 2006-07-27 15:52 ` Olivier Galibert 2006-07-27 21:42 ` suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Pavel Machek 2006-07-27 16:43 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Alan Cox 1 sibling, 1 reply; 639+ messages in thread From: Olivier Galibert @ 2006-07-27 15:52 UTC (permalink / raw) To: Pavel Machek Cc: Theodore Tso, Linux Kernel Mailing List, Nikita Danilov, Steve Lord On Wed, Jul 26, 2006 at 01:08:06PM +0000, Pavel Machek wrote: > Hi! > > > > A much more important effect is that non-maintainers aren't familiar > > > with coding and patch submission guidelines. For example, in > > > suspend2, Nigel first tried with patches that were too monolithic, > > > and then his next series was too broken down such that it was too > > > hard to review (and "git bisect" wouldn't work). > > > > All his submissions since 2004 or so? It's a little easy to limit > > oneself to the last two ones. > > Nigel did not do any submissions in 2004 or so. Check your fact, that > stuff was marked 'RFC' and yes I did comment on it. 2004-09-16 submission: http://lkml.org/lkml/2004/9/16/76 http://lkml.org/lkml/2004/9/16/77 http://lkml.org/lkml/2004/9/16/78 http://lkml.org/lkml/2004/9/16/81 http://lkml.org/lkml/2004/9/16/82 http://lkml.org/lkml/2004/9/16/83 http://lkml.org/lkml/2004/9/16/86 http://lkml.org/lkml/2004/9/16/87 http://lkml.org/lkml/2004/9/16/89 http://lkml.org/lkml/2004/9/16/90 2004-11-24 submission: http://lkml.org/lkml/2004/11/24/93 http://lkml.org/lkml/2004/11/24/94 http://lkml.org/lkml/2004/11/24/95 http://lkml.org/lkml/2004/11/24/96 http://lkml.org/lkml/2004/11/24/97 http://lkml.org/lkml/2004/11/24/98 http://lkml.org/lkml/2004/11/24/100 http://lkml.org/lkml/2004/11/24/101 [58 mails or so total] > He did 1 (one) submission that looked like SubmittingPatches at the > first sight, and that was very recent. > > Stop spreading lies. I am awaiting your apologies. OG. ^ permalink raw reply [flat|nested] 639+ messages in thread
* suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-27 15:52 ` Olivier Galibert @ 2006-07-27 21:42 ` Pavel Machek 2006-07-27 23:22 ` Olivier Galibert 0 siblings, 1 reply; 639+ messages in thread From: Pavel Machek @ 2006-07-27 21:42 UTC (permalink / raw) To: Olivier Galibert, Theodore Tso, Linux Kernel Mailing List, Nikita Danilov, Steve Lord Hi! > > > > A much more important effect is that non-maintainers aren't familiar > > > > with coding and patch submission guidelines. For example, in > > > > suspend2, Nigel first tried with patches that were too monolithic, > > > > and then his next series was too broken down such that it was too > > > > hard to review (and "git bisect" wouldn't work). > > > > > > All his submissions since 2004 or so? It's a little easy to limit > > > oneself to the last two ones. > > > > Nigel did not do any submissions in 2004 or so. Check your fact, that > > stuff was marked 'RFC' and yes I did comment on it. > > 2004-09-16 submission: > > http://lkml.org/lkml/2004/9/16/76 ... > http://lkml.org/lkml/2004/9/16/90 Yes, Nigel submitted part of suspend2 here (I do not think he submitted compression code on that day, for example), but then http://lkml.org/lkml/2004/9/16/347 came: #I should mention, I'm planning on producing a BK tree with these patches #(and ones to fix issues raised), so you won't have to include a long #list in -mm eventually! It will be on suspend2.bkbits.net (not there #yet). That sounds like "don't merge it yet, I'll prepare better version" to me. > 2004-11-24 submission: > > http://lkml.org/lkml/2004/11/24/93 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Okay, this one is important. It starts with #Hi everyone. # #I know that I still have work to do on suspend2, but thought it was high #time I got around to properly submitting the code for review, so here goes. That looks like request for review, not request for merge. Plus it was to: lkml, while it should be to: maintainer (or maybe to: akpm). Well, it also says "Suspend 2 merge" in subject.. > > He did 1 (one) submission that looked like SubmittingPatches at the > > first sight, and that was very recent. > > > > Stop spreading lies. > > I am awaiting your apologies. Okay, 11/24 looked closer to submission than I recalled. So I guess I have to say "sorry". So we have 1 submission for review in 11/2004 and 1 submission for -mm merge in 2006, right? Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-27 21:42 ` suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Pavel Machek @ 2006-07-27 23:22 ` Olivier Galibert 2006-07-28 14:01 ` Pavel Machek 0 siblings, 1 reply; 639+ messages in thread From: Olivier Galibert @ 2006-07-27 23:22 UTC (permalink / raw) To: Linux Kernel Mailing List On Thu, Jul 27, 2006 at 11:42:25PM +0200, Pavel Machek wrote: > So we have 1 submission for review in 11/2004 and 1 submission for -mm > merge in 2006, right? Wrong. I gave a list of dates at the beginning of the month, do you think I threw dice to get them? And could you explain, as suspend maintainer for the linux kernel, how come code submitted for the first time two years ago and with a much better track record than the in-kernel one is still not in? OG. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-27 23:22 ` Olivier Galibert @ 2006-07-28 14:01 ` Pavel Machek 2006-07-29 19:23 ` Bill Davidsen 0 siblings, 1 reply; 639+ messages in thread From: Pavel Machek @ 2006-07-28 14:01 UTC (permalink / raw) To: Olivier Galibert, Linux Kernel Mailing List On Fri 28-07-06 01:22:49, Olivier Galibert wrote: > On Thu, Jul 27, 2006 at 11:42:25PM +0200, Pavel Machek wrote: > > So we have 1 submission for review in 11/2004 and 1 submission for -mm > > merge in 2006, right? > > Wrong. I gave a list of dates at the beginning of the month, do you > think I threw dice to get them? > > And could you explain, as suspend maintainer for the linux kernel, how > come code submitted for the first time two years ago and with a much > better track record than the in-kernel one is still not in? Because Nigel has too much of code to start with, and refuses to fix his design because it would invalidate all the stabilization work. Plus Nigel did not do very good job with submitting those patches. Pavel -- Thanks for all the (sleeping) penguins. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-28 14:01 ` Pavel Machek @ 2006-07-29 19:23 ` Bill Davidsen 2006-07-29 21:19 ` Rafael J. Wysocki 2006-07-29 23:31 ` Pavel Machek 0 siblings, 2 replies; 639+ messages in thread From: Bill Davidsen @ 2006-07-29 19:23 UTC (permalink / raw) To: Pavel Machek, Kernel Mailing List Pavel Machek wrote: > On Fri 28-07-06 01:22:49, Olivier Galibert wrote: >> On Thu, Jul 27, 2006 at 11:42:25PM +0200, Pavel Machek wrote: >>> So we have 1 submission for review in 11/2004 and 1 submission for -mm >>> merge in 2006, right? >> Wrong. I gave a list of dates at the beginning of the month, do you >> think I threw dice to get them? >> >> And could you explain, as suspend maintainer for the linux kernel, how >> come code submitted for the first time two years ago and with a much >> better track record than the in-kernel one is still not in? > > Because Nigel has too much of code to start with, and refuses to fix > his design because it would invalidate all the stabilization work. Why should he invalidate his stabilization work, and what's in need of fixing? The suspend in the kernel is great, but suspend2 includes both suspend and working resume code as well. > > Plus Nigel did not do very good job with submitting those patches. They apply, they work. What's not very good about that? Is this being blocked because of a spelling error, or did he mess up the indenting on "signed off by" or what? I realize you may have something other than the download version, but it's been years now. I would like to see the working suspend (suspend2) in the kernel, and users wanting to debug the resume stuff currently in the kernel could get it under EXPERIMENTAL or some such. -- Bill Davidsen <davidsen@tmr.com> Obscure bug of 2004: BASH BUFFER OVERFLOW - if bash is being run by a normal user and is setuid root, with the "vi" line edit mode selected, and the character set is "big5," an off-by-one errors occurs during wildcard (glob) expansion. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-29 19:23 ` Bill Davidsen @ 2006-07-29 21:19 ` Rafael J. Wysocki 2006-07-29 21:56 ` Hua Zhong 2006-07-30 12:57 ` Bill Davidsen 2006-07-29 23:31 ` Pavel Machek 1 sibling, 2 replies; 639+ messages in thread From: Rafael J. Wysocki @ 2006-07-29 21:19 UTC (permalink / raw) To: Bill Davidsen; +Cc: Pavel Machek, Kernel Mailing List On Saturday 29 July 2006 21:23, Bill Davidsen wrote: > Pavel Machek wrote: > > On Fri 28-07-06 01:22:49, Olivier Galibert wrote: > >> On Thu, Jul 27, 2006 at 11:42:25PM +0200, Pavel Machek wrote: > >>> So we have 1 submission for review in 11/2004 and 1 submission for -mm > >>> merge in 2006, right? > >> Wrong. I gave a list of dates at the beginning of the month, do you > >> think I threw dice to get them? > >> > >> And could you explain, as suspend maintainer for the linux kernel, how > >> come code submitted for the first time two years ago and with a much > >> better track record than the in-kernel one is still not in? > > > > Because Nigel has too much of code to start with, and refuses to fix > > his design because it would invalidate all the stabilization work. > > Why should he invalidate his stabilization work, and what's in need of > fixing? The suspend in the kernel is great, but suspend2 includes both > suspend and working resume code as well. > > > > Plus Nigel did not do very good job with submitting those patches. > > They apply, they work. What's not very good about that? Is this being > blocked because of a spelling error, or did he mess up the indenting on > "signed off by" or what? I realize you may have something other than the > download version, but it's been years now. > > I would like to see the working suspend (suspend2) in the kernel, and > users wanting to debug the resume stuff currently in the kernel could > get it under EXPERIMENTAL or some such. You probably don't realize how offensive this is. Actually some people have been working really hard to make the in-kernel code work and you could just respect that. Now, please note the swsusp resume code works for me 100% of the time and I know of many people who use it without any problems. Also I know of some people for whom suspend2 doesn't work. Please take this into consideration. Moreover, if the swsusp's resume doesn't work for you and suspend2's resume does, this probably means that suspend2 contains some driver fixes that haven't been submitted for merging. Greetings, Rafael ^ permalink raw reply [flat|nested] 639+ messages in thread
* RE: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-29 21:19 ` Rafael J. Wysocki @ 2006-07-29 21:56 ` Hua Zhong 2006-07-29 22:54 ` Rafael J. Wysocki ` (2 more replies) 2006-07-30 12:57 ` Bill Davidsen 1 sibling, 3 replies; 639+ messages in thread From: Hua Zhong @ 2006-07-29 21:56 UTC (permalink / raw) To: 'Rafael J. Wysocki', 'Bill Davidsen' Cc: 'Pavel Machek', 'Kernel Mailing List' > Moreover, if the swsusp's resume doesn't work for you and > suspend2's resume does, this probably means that suspend2 > contains some driver fixes that haven't been submitted for merging. This statement worries me for several reasons. First, I've seen repeatedly blame for "drivers". People might buy it if there was not a working suspend2. I never saw Nigal blame drivers; instead he makes sure to provide working code. In the end, users want a working suspend, and that's what counts. Second, if the current swsusp maintainers have genuine interest for the users (not just "it works on my machine"), I would think they'd have taken a serious look at why suspend2 has a higher success rate. But from the above comment, you are not even sure why that is, and could only speculate "this probably means (drivers)". I could hardly be convinced that the current maintainers have been trying to work with Nigal to improve Linux suspend. At last, "maintainer" is not just a title or a feeling of superiority, it's a responsibility of providing a great system to the users. I'll just quote Linus when he was flaming suspend-to-ram a couple of months ago. Replace it with suspend-to-disk at your own will: "The fact that worries me is that suspend-to-ram DOES NOT WORK FOR PEOPLE. I have never _ever_ met a laptop or machine of mine that "just worked". I've always had to fix something, and people always end up having to do something ridiculous like unlink all modules etc. If that isn't what worries you, you're on the wrong page." http://article.gmane.org/gmane.linux.power-management.general/2105 Hua ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-29 21:56 ` Hua Zhong @ 2006-07-29 22:54 ` Rafael J. Wysocki 2006-07-30 22:25 ` Hua Zhong 2006-07-29 23:19 ` Bill Davidsen 2006-07-29 23:47 ` Pavel Machek 2 siblings, 1 reply; 639+ messages in thread From: Rafael J. Wysocki @ 2006-07-29 22:54 UTC (permalink / raw) To: Hua Zhong Cc: 'Bill Davidsen', 'Pavel Machek', 'Kernel Mailing List' On Saturday 29 July 2006 23:56, Hua Zhong wrote: > > Moreover, if the swsusp's resume doesn't work for you and > > suspend2's resume does, this probably means that suspend2 > > contains some driver fixes that haven't been submitted for merging. > > This statement worries me for several reasons. > > First, I've seen repeatedly blame for "drivers". People might buy it if > there was not a working suspend2. I never saw Nigal blame > drivers; I don't _blame_ drivers. I only wanted to say this: "If Nigel knows that some drivers need to be fixed and he has working fixes for these drivers, he should have submitted these fixes for merging instead of just keeping them in suspend2". Period. If I know of a fix for a driver, I always do my best to make sure the fix will get considered for merging at least. The problem is I'm not a driver expert and I can't provide the fixes myself. > instead he makes sure to provide working code. In the end, users want > a working suspend, and that's what counts. > > Second, if the current swsusp maintainers have genuine interest for the > users (not just "it works on my machine"), I would think they'd have taken > a serious look at why suspend2 has a higher success rate. But from the > above comment, you are not even sure why that is, and could only speculate > "this probably means (drivers)". No, I'm not sure. If I were, I would fix the problem. Also I'm not quite sure about the higher success rate of suspend2. > I could hardly be convinced that the current maintainers have been trying to > work with Nigal to improve Linux suspend. This is an offensive comment. > At last, "maintainer" is not just a title or a feeling of superiority, it's > a responsibility of providing a great system to the users. I'll just quote > Linus when he was flaming suspend-to-ram a couple of months ago. > Replace it with suspend-to-disk at your own will: This also is offensive, because the current swsusp _does_ work for many people. Actually I've recently tried a couple of various machines on which it "just works", so I will not replace suspend-to-ram with suspend-to-disk below, sorry. Moreover, if it doesn't and the problem is reported, we do our best to trace it and fix it, but sometimes we just can't do this without access to the hardware in question. > "The fact that worries me is that suspend-to-ram DOES NOT WORK FOR PEOPLE. > I have never _ever_ met a laptop or machine of mine that "just worked". > I've always had to fix something, and people always end up having to do > something ridiculous like unlink all modules etc. > > If that isn't what worries you, you're on the wrong page." And that's very true. For example, the suspend-to-ram doesn't fully work on my own box and I'm not sure it ever will. However, that's not a fault of anyone who works on this, just broken BIOS and the lack of documentation. This is worrisome and many people work on improving these things, but the matter is notoriously difficult. Greetings, Rafael ^ permalink raw reply [flat|nested] 639+ messages in thread
* RE: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-29 22:54 ` Rafael J. Wysocki @ 2006-07-30 22:25 ` Hua Zhong 2006-07-30 23:07 ` Pavel Machek 0 siblings, 1 reply; 639+ messages in thread From: Hua Zhong @ 2006-07-30 22:25 UTC (permalink / raw) To: 'Rafael J. Wysocki' Cc: 'Bill Davidsen', 'Pavel Machek', 'Kernel Mailing List' > I don't _blame_ drivers. I only wanted to say this: "If > Nigel knows that some drivers need to be fixed and he has > working fixes for these drivers, he should have submitted > these fixes for merging instead of just keeping them in > suspend2". Period. > > If I know of a fix for a driver, I always do my best to make > sure the fix will get considered for merging at least. The > problem is I'm not a driver expert and I can't provide the > fixes myself. Suspend2 patch is open source. You can always take a look. Moreover, if someone claims suspend2 isn't ready for merge, or the patches Nigal has submitted aren't up to standards, one would guess he'd have done so already. So I don't understand what the above means. Have you asked Nigal whether he had any driver fixes? If you guys are really working together, isn't it very easy to do? > And that's very true. For example, the suspend-to-ram > doesn't fully work on my own box and I'm not sure it ever > will. However, that's not a fault of anyone who works on > this, just broken BIOS and the lack of documentation. > This is worrisome and many people work on improving these > things, but the matter is notoriously difficult. I'm not exactly an expert, but I don't think suspend-to-ram is more difficult than suspend-to-disk (probably quite the contrary), and there are a lot in common. I am sorry that you find some of the comments offensive, but there isn't really anything personal. It's merely a reflection of the frustration from linux users wrt suspend OVER THE YEARS. I know you guys are hard working, but in the end it's a top-quality suspend that counts, and people's patience has been wearing out, especially when there has been an out-of-tree implementation available for a long time already. Everyone wants to see the current maintainers take a more humble and proactive attitude when working with Nigal instead of just dismissing his "bad design" and bashing his patch quality. It's so easy to block someone's initiative than making it better. Hua ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-30 22:25 ` Hua Zhong @ 2006-07-30 23:07 ` Pavel Machek 2006-07-31 14:28 ` Bill Davidsen 2006-07-31 18:55 ` Hua Zhong 0 siblings, 2 replies; 639+ messages in thread From: Pavel Machek @ 2006-07-30 23:07 UTC (permalink / raw) To: Hua Zhong Cc: 'Rafael J. Wysocki', 'Bill Davidsen', 'Kernel Mailing List' On Sun 2006-07-30 15:25:49, Hua Zhong wrote: > > I don't _blame_ drivers. I only wanted to say this: "If > > Nigel knows that some drivers need to be fixed and he has > > working fixes for these drivers, he should have submitted > > these fixes for merging instead of just keeping them in > > suspend2". Period. > > > > If I know of a fix for a driver, I always do my best to make > > sure the fix will get considered for merging at least. The > > problem is I'm not a driver expert and I can't provide the > > fixes myself. > > Suspend2 patch is open source. You can always take a look. swsusp is open source. You can always take a look. And you can always submit a patch. > Moreover, if someone claims suspend2 isn't ready for merge, or the Moreover, if someone claims swsusp is broken, they should attach bugzilla id. > I'm not exactly an expert, but I don't think suspend-to-ram is more > difficult than suspend-to-disk (probably quite the contrary), and > there are a lot in common. As you said, you do not know what you are talking about. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-30 23:07 ` Pavel Machek @ 2006-07-31 14:28 ` Bill Davidsen 2006-07-31 16:23 ` Pavel Machek 2006-07-31 18:55 ` Hua Zhong 1 sibling, 1 reply; 639+ messages in thread From: Bill Davidsen @ 2006-07-31 14:28 UTC (permalink / raw) To: Pavel Machek Cc: Hua Zhong, 'Rafael J. Wysocki', 'Kernel Mailing List' Pavel Machek wrote: >On Sun 2006-07-30 15:25:49, Hua Zhong wrote: > > >>>I don't _blame_ drivers. I only wanted to say this: "If >>>Nigel knows that some drivers need to be fixed and he has >>>working fixes for these drivers, he should have submitted >>>these fixes for merging instead of just keeping them in >>>suspend2". Period. >>> >>>If I know of a fix for a driver, I always do my best to make >>>sure the fix will get considered for merging at least. The >>>problem is I'm not a driver expert and I can't provide the >>>fixes myself. >>> >>> >>Suspend2 patch is open source. You can always take a look. >> >> > >swsusp is open source. You can always take a look. And you can always >submit a patch. > > But you can't get the patch accepted, that's issue causing all this discussion. >>Moreover, if someone claims suspend2 isn't ready for merge, or the >> >> > >Moreover, if someone claims swsusp is broken, they should attach >bugzilla id. > > > >>I'm not exactly an expert, but I don't think suspend-to-ram is more >>difficult than suspend-to-disk (probably quite the contrary), and >>there are a lot in common. >> >> > >As you said, you do not know what you are talking about. > > > That's why people are frustrated. You blow off anyone who tells you the code doesn't work. Do you really think Linus doesn't know what he's talking about when he reported that it didn't work for him? Hua Zhong being "Not an expert" is not the same as not knowing what he's talking about. -- bill davidsen <davidsen@tmr.com> CTO TMR Associates, Inc Doing interesting things with small computers since 1979 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 14:28 ` Bill Davidsen @ 2006-07-31 16:23 ` Pavel Machek 0 siblings, 0 replies; 639+ messages in thread From: Pavel Machek @ 2006-07-31 16:23 UTC (permalink / raw) To: Bill Davidsen Cc: Hua Zhong, 'Rafael J. Wysocki', 'Kernel Mailing List' Hi! > >>Suspend2 patch is open source. You can always take a look. > > > >swsusp is open source. You can always take a look. And you can always > >submit a patch. > But you can't get the patch accepted, that's issue causing all this > discussion. If you send me a reasonably-sized patch, that actually fixes something, I'm likely to apply it. If you send me 14K line monster... > >>I'm not exactly an expert, but I don't think suspend-to-ram is more > >>difficult than suspend-to-disk (probably quite the contrary), and > >>there are a lot in common. > > > >As you said, you do not know what you are talking about. > > > That's why people are frustrated. You blow off anyone who tells you the > code doesn't work. Do you really think Linus doesn't know what he's > talking about when he reported that it didn't work for him? Hua Zhong > being "Not an expert" is not the same as not knowing what he's talking > about. He claims s-t-ram is easier than s-t-disk. That means that he did not do his homework, and did not check the archives on the subject. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ^ permalink raw reply [flat|nested] 639+ messages in thread
* RE: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-30 23:07 ` Pavel Machek 2006-07-31 14:28 ` Bill Davidsen @ 2006-07-31 18:55 ` Hua Zhong 2006-07-31 23:08 ` Rafael J. Wysocki 2006-08-01 1:09 ` Pavel Machek 1 sibling, 2 replies; 639+ messages in thread From: Hua Zhong @ 2006-07-31 18:55 UTC (permalink / raw) To: 'Pavel Machek' Cc: 'Rafael J. Wysocki', 'Bill Davidsen', 'Kernel Mailing List' > > Suspend2 patch is open source. You can always take a look. > > swsusp is open source. You can always take a look. And you > can always submit a patch. > > > Moreover, if someone claims suspend2 isn't ready for merge, or the > > Moreover, if someone claims swsusp is broken, they should > attach bugzilla id. Pavel, You can't blame me for not doing these things, because I am not a maintainer. However, you are, and you defend yourself so hard for that position, so if _you_ don't do these things, people complain. > As you said, you do not know what you are talking about. > > He claims s-t-ram is easier than s-t-disk. That means that he did not do his > homework, and did not check the archives on the subject. Oh yeah? Let's check the archives: "I seriously claim that STR _should_ be a lot simpler than suspend-to-disk, because it avoids all the memory management problems. The reason that we support suspend-to-disk but not STR is totally perverse - it's simply that it has been easier to debug, because unlike STR, we can do a "real boot" into a working system, and thus we don't have the debugging problems that the "easy" suspend/resume case has." http://thread.gmane.org/gmane.linux.power-management.general/1884/focus=2105 Maybe it's why he didn't like the STR design you had? Maybe I am still wrong, maybe Linus is wrong too, but you can't attack me not doing my homework. Hua ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 18:55 ` Hua Zhong @ 2006-07-31 23:08 ` Rafael J. Wysocki 2006-08-01 1:09 ` Pavel Machek 1 sibling, 0 replies; 639+ messages in thread From: Rafael J. Wysocki @ 2006-07-31 23:08 UTC (permalink / raw) To: Hua Zhong Cc: 'Pavel Machek', 'Bill Davidsen', 'Kernel Mailing List' On Monday 31 July 2006 20:55, Hua Zhong wrote: ]--snip--[ > > He claims s-t-ram is easier than s-t-disk. That means that he did not do his > > homework, and did not check the archives on the subject. > > Oh yeah? Let's check the archives: > > "I seriously claim that STR _should_ be a lot simpler than suspend-to-disk, > because it avoids all the memory management problems. The reason that > we support suspend-to-disk but not STR is totally perverse - it's simply that > it has been easier to debug, because unlike STR, we can do a "real boot" > into a working system, and thus we don't have the debugging problems that > the "easy" suspend/resume case has." > > http://thread.gmane.org/gmane.linux.power-management.general/1884/focus=2105 The "_should_" is important here, IMHO. I think the problems that we have with getting STR to work on many machines reflect the situation in which we are with respect to the hardware. From the programming point of view it's easy, because if you know how to handle the hardware, it doesn't take a lot to get it right. Still, you need to _know_, and we're talking of things that are hardly documented and require hours of trial-and-error to figure out. Usually, it can only be done by someone who (a) has access to the hardware in question, (b) has a lot of time, (c) is _very_ determined to make the suspend and resume work on this particular device, because these things are notoriously difficult to debug, and (d), ideally, knows how to write Linux device drivers. If you own an "exotic" notebook, there's practically no chance to find someone like that who owns one too. Moreover, even if there are some hardware-related fixes in the wild, we can't just throw them at Andrew to merge, because some kernel developers may think they are not the right fixes. Each fix, before it gets merged, if ever, has to be reviewed by the appropriate driver maintainers and people who know how the related subsystems work. If the fix gets rejected, we can't help it. For example, we've recently received a fix for a resume-related problem on some IDE chipsets (AMD and NVidia ones), but it has been vetoed by Alan Cox. The Alan's arguments are reasonable and everyone seems to agree that it should be done differently, but the net result, for now, is the problem remains - officially - unfixed (see http://www.ussg.iu.edu/hypermail/linux/kernel/0607.3/1607.html). However, Nigel maintains his patch in separation with the mainline kernel and he can include fixes like that into it just fine. He can include whatever he likes, as long as it works, but we just can't do that. You can say he has more freedom, because he doesn't have to take the other people's opinions into consideration, so he can make suspend2 work on a greater number of systems more easily. Yet, he faces the other people's opinions about the things that are in suspend2 whenever he submits it for merging. In other words, the same things that make suspend2 work on machines on which swsusp doesn't may also render it _very_ difficult to merge (eg. if Nigel had decided to include the patch vetoed by Alan into suspend2, the Alan's NAK would have applied to suspend2 as a whole). Generally speaking suspend2 is a collection of many different solutions, some of them being more or less questionable, in many different areas which should not be considered all at once. There should be a separate patch for each of them, submitted and discussed on its own. Moreover, some of these solutions may be considered as not the right ones and some patches may get rejected, which may affect the next patches etc. Still, this is the way in which the kernel is developed and we have no other way to follow. Greetings, Rafael ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-31 18:55 ` Hua Zhong 2006-07-31 23:08 ` Rafael J. Wysocki @ 2006-08-01 1:09 ` Pavel Machek 1 sibling, 0 replies; 639+ messages in thread From: Pavel Machek @ 2006-08-01 1:09 UTC (permalink / raw) To: Hua Zhong Cc: 'Rafael J. Wysocki', 'Bill Davidsen', 'Kernel Mailing List' On Mon 2006-07-31 11:55:58, Hua Zhong wrote: > > > Suspend2 patch is open source. You can always take a look. > > > > swsusp is open source. You can always take a look. And you > > can always submit a patch. > > > > > Moreover, if someone claims suspend2 isn't ready for merge, or the > > > > Moreover, if someone claims swsusp is broken, they should > > attach bugzilla id. > > Pavel, > > You can't blame me for not doing these things, because I am not a maintainer. > However, you are, and you defend yourself so hard for that position, so if _you_ > don't do these things, people complain. I have taken a look at suspend2 and decided I did not like it. (see archives). If you think parts of suspend2 are useful (== fix problem on hardware you have), separate them and submit them. If you want to claim that swsusp is broken, you should attach bugzilla ids. Anything else is unhelpful. I'm maintainer, but that does not mean that I have every possible notebook on earth. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-29 21:56 ` Hua Zhong 2006-07-29 22:54 ` Rafael J. Wysocki @ 2006-07-29 23:19 ` Bill Davidsen 2006-07-29 23:28 ` Pavel Machek 2006-07-29 23:47 ` Pavel Machek 2 siblings, 1 reply; 639+ messages in thread From: Bill Davidsen @ 2006-07-29 23:19 UTC (permalink / raw) To: Hua Zhong; +Cc: 'Pavel Machek', 'Kernel Mailing List' Hua Zhong wrote: >> Moreover, if the swsusp's resume doesn't work for you and >> suspend2's resume does, this probably means that suspend2 >> contains some driver fixes that haven't been submitted for merging. > > This statement worries me for several reasons. > > First, I've seen repeatedly blame for "drivers". People might buy it if there was not a working suspend2. I never saw Nigal blame > drivers; instead he makes sure to provide working code. In the end, users want a working suspend, and that's what counts. > > Second, if the current swsusp maintainers have genuine interest for the users (not just "it works on my machine"), I would think > they'd have taken a serious look at why suspend2 has a higher success rate. But from the above comment, you are not even sure why > that is, and could only speculate "this probably means (drivers)". I could hardly be convinced that the current maintainers have > been trying to work with Nigal to improve Linux suspend. > > At last, "maintainer" is not just a title or a feeling of superiority, it's a responsibility of providing a great system to the > users. I'll just quote Linus when he was flaming suspend-to-ram a couple of months ago. Replace it with suspend-to-disk at your own > will: > > "The fact that worries me is that suspend-to-ram DOES NOT WORK FOR PEOPLE. > I have never _ever_ met a laptop or machine of mine that "just worked". > I've always had to fix something, and people always end up having to do > something ridiculous like unlink all modules etc. > > If that isn't what worries you, you're on the wrong page." > > http://article.gmane.org/gmane.linux.power-management.general/2105 That's the way I feel, I'm glad someone who can't be ignored feels the same way. There is always a reason why the working version can't be put in mainline, there's always a reason why it doesn't work on my machines but I'm assured that it works on {someone}'s machine so I must have a bad {bios|setup|partition table|attitude}. -- Bill Davidsen <davidsen@tmr.com> Obscure bug of 2004: BASH BUFFER OVERFLOW - if bash is being run by a normal user and is setuid root, with the "vi" line edit mode selected, and the character set is "big5," an off-by-one errors occurs during wildcard (glob) expansion. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-29 23:19 ` Bill Davidsen @ 2006-07-29 23:28 ` Pavel Machek 0 siblings, 0 replies; 639+ messages in thread From: Pavel Machek @ 2006-07-29 23:28 UTC (permalink / raw) To: Bill Davidsen; +Cc: Hua Zhong, 'Kernel Mailing List' Hi! > That's the way I feel, I'm glad someone who can't be ignored feels the > same way. There is always a reason why the working version can't be put > in mainline, there's always a reason why it doesn't work on my machines > but I'm assured that it works on {someone}'s machine so I must have a > bad {bios|setup|partition table|attitude}. Do you have bugzilla # or are you just flaming? Please test 2.6.18-rc2. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-29 21:56 ` Hua Zhong 2006-07-29 22:54 ` Rafael J. Wysocki 2006-07-29 23:19 ` Bill Davidsen @ 2006-07-29 23:47 ` Pavel Machek 2 siblings, 0 replies; 639+ messages in thread From: Pavel Machek @ 2006-07-29 23:47 UTC (permalink / raw) To: Hua Zhong Cc: 'Rafael J. Wysocki', 'Bill Davidsen', 'Kernel Mailing List' Hi! > > Moreover, if the swsusp's resume doesn't work for you and > > suspend2's resume does, this probably means that suspend2 > > contains some driver fixes that haven't been submitted for merging. > > This statement worries me for several reasons. > > First, I've seen repeatedly blame for "drivers". People might buy it if there was not a working suspend2. I never saw Nigal blame > drivers; instead he makes sure to provide working code. In the end, users want a working suspend, and that's what counts. > You seem to be claiming that swsusp is broken, but I do not remember any bugreport from you. So... either provide bugzilla #, or stop spreading FUD. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-29 21:19 ` Rafael J. Wysocki 2006-07-29 21:56 ` Hua Zhong @ 2006-07-30 12:57 ` Bill Davidsen 2006-07-30 14:34 ` Rafael J. Wysocki 1 sibling, 1 reply; 639+ messages in thread From: Bill Davidsen @ 2006-07-30 12:57 UTC (permalink / raw) To: Rafael J. Wysocki; +Cc: Pavel Machek, Kernel Mailing List Rafael J. Wysocki wrote: >On Saturday 29 July 2006 21:23, Bill Davidsen wrote: > > >>Pavel Machek wrote: >> >> >>>On Fri 28-07-06 01:22:49, Olivier Galibert wrote: >>> >>> >>>>On Thu, Jul 27, 2006 at 11:42:25PM +0200, Pavel Machek wrote: >>>> >>>> >>>>>So we have 1 submission for review in 11/2004 and 1 submission for -mm >>>>>merge in 2006, right? >>>>> >>>>> >>>>Wrong. I gave a list of dates at the beginning of the month, do you >>>>think I threw dice to get them? >>>> >>>>And could you explain, as suspend maintainer for the linux kernel, how >>>>come code submitted for the first time two years ago and with a much >>>>better track record than the in-kernel one is still not in? >>>> >>>> >>>Because Nigel has too much of code to start with, and refuses to fix >>>his design because it would invalidate all the stabilization work. >>> >>> >>Why should he invalidate his stabilization work, and what's in need of >>fixing? The suspend in the kernel is great, but suspend2 includes both >>suspend and working resume code as well. >> >> >>>Plus Nigel did not do very good job with submitting those patches. >>> >>> >>They apply, they work. What's not very good about that? Is this being >>blocked because of a spelling error, or did he mess up the indenting on >>"signed off by" or what? I realize you may have something other than the >>download version, but it's been years now. >> >>I would like to see the working suspend (suspend2) in the kernel, and >>users wanting to debug the resume stuff currently in the kernel could >>get it under EXPERIMENTAL or some such. >> >> > >You probably don't realize how offensive this is. > >Actually some people have been working really hard to make the in-kernel >code work and you could just respect that. > > By respect I take it you mean "don't call attention to the fact that it doesn't work for many people?" >Now, please note the swsusp resume code works for me 100% of the time >and I know of many people who use it without any problems. Also I know of >some people for whom suspend2 doesn't work. Please take this into >consideration. > > That's exactly the answer I've been getting for saveral years, "it works for me and my friends," and "try x.y.z release." >Moreover, if the swsusp's resume doesn't work for you and suspend2's resume >does, this probably means that suspend2 contains some driver fixes that >haven't been submitted for merging. > That's been addressed by other people, but the suspend2 patch has been submitted multiple times, if there is some driver fix it certainly has been submitted, just not accepted. -- bill davidsen <davidsen@tmr.com> CTO TMR Associates, Inc Doing interesting things with small computers since 1979 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-30 12:57 ` Bill Davidsen @ 2006-07-30 14:34 ` Rafael J. Wysocki 0 siblings, 0 replies; 639+ messages in thread From: Rafael J. Wysocki @ 2006-07-30 14:34 UTC (permalink / raw) To: Bill Davidsen; +Cc: Pavel Machek, Kernel Mailing List On Sunday 30 July 2006 14:57, Bill Davidsen wrote: > Rafael J. Wysocki wrote: > > >On Saturday 29 July 2006 21:23, Bill Davidsen wrote: > > > > > >>Pavel Machek wrote: > >> > >> > >>>On Fri 28-07-06 01:22:49, Olivier Galibert wrote: > >>> > >>> > >>>>On Thu, Jul 27, 2006 at 11:42:25PM +0200, Pavel Machek wrote: > >>>> > >>>> > >>>>>So we have 1 submission for review in 11/2004 and 1 submission for -mm > >>>>>merge in 2006, right? > >>>>> > >>>>> > >>>>Wrong. I gave a list of dates at the beginning of the month, do you > >>>>think I threw dice to get them? > >>>> > >>>>And could you explain, as suspend maintainer for the linux kernel, how > >>>>come code submitted for the first time two years ago and with a much > >>>>better track record than the in-kernel one is still not in? > >>>> > >>>> > >>>Because Nigel has too much of code to start with, and refuses to fix > >>>his design because it would invalidate all the stabilization work. > >>> > >>> > >>Why should he invalidate his stabilization work, and what's in need of > >>fixing? The suspend in the kernel is great, but suspend2 includes both > >>suspend and working resume code as well. > >> > >> > >>>Plus Nigel did not do very good job with submitting those patches. > >>> > >>> > >>They apply, they work. What's not very good about that? Is this being > >>blocked because of a spelling error, or did he mess up the indenting on > >>"signed off by" or what? I realize you may have something other than the > >>download version, but it's been years now. > >> > >>I would like to see the working suspend (suspend2) in the kernel, and > >>users wanting to debug the resume stuff currently in the kernel could > >>get it under EXPERIMENTAL or some such. > >> > >You probably don't realize how offensive this is. > > > >Actually some people have been working really hard to make the in-kernel > >code work and you could just respect that. > > > By respect I take it you mean "don't call attention to the fact that it > doesn't work for many people?" Sorry, you won't get anywhere by insulting people. If you want to call attention to a problem, please file a bug report. Greetings, Rafael ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] 2006-07-29 19:23 ` Bill Davidsen 2006-07-29 21:19 ` Rafael J. Wysocki @ 2006-07-29 23:31 ` Pavel Machek 1 sibling, 0 replies; 639+ messages in thread From: Pavel Machek @ 2006-07-29 23:31 UTC (permalink / raw) To: Bill Davidsen; +Cc: Kernel Mailing List Hi! > >Plus Nigel did not do very good job with submitting those patches. > > They apply, they work. What's not very good about that? Is this being > blocked because of a spelling error, or did he mess up the indenting on > "signed off by" or what? I realize you may have something other than the > download version, but it's been years now. Ask Nigel or search the archives. I will not repeat myself again. > I would like to see the working suspend (suspend2) in the kernel, Great. Help with debugging. Alternatively, help Nigel with fixing design problems and separating patches for merge -- that should keep you busy and stop you from trolling on the lists. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-26 13:08 ` Pavel Machek 2006-07-27 15:52 ` Olivier Galibert @ 2006-07-27 16:43 ` Alan Cox 1 sibling, 0 replies; 639+ messages in thread From: Alan Cox @ 2006-07-27 16:43 UTC (permalink / raw) To: Pavel Machek Cc: Olivier Galibert, Theodore Tso, Linux Kernel Mailing List, Nikita Danilov, Steve Lord Ar Mer, 2006-07-26 am 13:08 +0000, ysgrifennodd Pavel Machek: > He did 1 (one) submission that looked like SubmittingPatches at the > first sight, and that was very recent. > > Stop spreading lies. Pavel, I think you might want to check your facts and then apologize. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 10:30 ` Theodore Tso 2006-07-24 11:35 ` Olivier Galibert @ 2006-07-25 21:44 ` Valdis.Kletnieks 1 sibling, 0 replies; 639+ messages in thread From: Valdis.Kletnieks @ 2006-07-25 21:44 UTC (permalink / raw) To: Theodore Tso; +Cc: Nikita Danilov, Steve Lord, Linux Kernel Mailing List [-- Attachment #1: Type: text/plain, Size: 346 bytes --] On Mon, 24 Jul 2006 06:30:23 EDT, Theodore Tso said: > both have to do a line-by-line review, and with a promise of on-disk > and ABI compatibility *forever* ---- that we do more commits in a week "forever"? Man, that's hard-core. Is that *really* the guideline, or is it some "might as well be forever in this industry" rule like "5 years"? [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-21 19:46 the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Hans Reiser ` (2 preceding siblings ...) 2006-07-22 13:02 ` Theodore Tso @ 2006-07-24 22:17 ` Paul Jackson 2006-07-25 8:05 ` Hans Reiser 3 siblings, 1 reply; 639+ messages in thread From: Paul Jackson @ 2006-07-24 22:17 UTC (permalink / raw) To: Hans Reiser; +Cc: linux-kernel Hans wrote: > Jim Grey Sounds like the name of Jim Gray, the famous database and transaction processing expert of IBM, Tandem, DEC and Microsoft. Neither SGI nor XFS has had the honor of working with Jim Gray. The original XFS developers and designers include Geoff Peck, Jeff Glover, Adam Sweeney and Doug Doucette, from what I can see from my notes dating back to 1993/1994 on this. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <pj@sgi.com> 1.925.600.0401 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion 2006-07-24 22:17 ` Paul Jackson @ 2006-07-25 8:05 ` Hans Reiser 0 siblings, 0 replies; 639+ messages in thread From: Hans Reiser @ 2006-07-25 8:05 UTC (permalink / raw) To: Paul Jackson; +Cc: linux-kernel No, it was not him, sigh, I'll have to ask Jim Mostek who it was if I can find a valid email for Jim. There was another guy who was Jim Mostek's peer. I might have the name wrong. ^ permalink raw reply [flat|nested] 639+ messages in thread
[parent not found: <matthias.andree@gmx.de>]
* #! incompatible -- binfmt_script.c broken? @ 2002-12-04 11:34 ` Matthias Andree 2002-12-04 14:26 ` Alex Riesen ` (2 more replies) 0 siblings, 3 replies; 639+ messages in thread From: Matthias Andree @ 2002-12-04 11:34 UTC (permalink / raw) To: Linux-Kernel mailing list Hi, I tried some of the Perl magic tricks shown in the perlrun man page with Linux 2.4.19; consider this Perl one-liner. It works on FreeBSD and Solaris, but fails on Linux. Looking at binfmt_script.c, I believe the "pass the rest of the line as the first argument to the interpreter" is the problem with Linux. Haven't yet figured if the other boxes just use the interpreter, ignoring the arguments or if they are doing argument splitting. ------------------------------------------------------------------------ #!/bin/sh -- # -*- perl -*- -T eval 'exec perl -wTS $0 ${1+"$@"}' if 0; print "Hello there.\n"; ------------------------------------------------------------------------ FreeBSD 4.7: $ /tmp/try.pl Hello there. Solaris 8: $ /tmp/try.pl Hello there. SuSE Linux 7.0, 7.3, 8.1 (2.4.19 kernel, binfmt_script.c identical to 2.4.20 BK): $ /tmp/try.pl /bin/sh: -- # -*- perl -*- -T: invalid option Usage: /bin/sh [GNU long option] [option] ... /bin/sh [GNU long option] [option] script-file ... [...] -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: #! incompatible -- binfmt_script.c broken? 2002-12-04 11:34 ` #! incompatible -- binfmt_script.c broken? Matthias Andree @ 2002-12-04 14:26 ` Alex Riesen 2002-12-04 15:23 ` Matthew Garrett ` (2 more replies) 2002-12-04 22:25 ` Andries Brouwer 2002-12-05 0:10 ` Horst von Brand 2 siblings, 3 replies; 639+ messages in thread From: Alex Riesen @ 2002-12-04 14:26 UTC (permalink / raw) To: Matthias Andree; +Cc: linux-kernel On Wed, Dec 04, 2002 at 12:34:19PM +0100, Matthias Andree wrote: > Hi, > > I tried some of the Perl magic tricks shown in the perlrun man page with > Linux 2.4.19; consider this Perl one-liner. It works on FreeBSD and > Solaris, but fails on Linux. Looking at binfmt_script.c, I believe the > "pass the rest of the line as the first argument to the interpreter" is > the problem with Linux. Haven't yet figured if the other boxes just use > the interpreter, ignoring the arguments or if they are doing argument > splitting. > > ------------------------------------------------------------------------ > #!/bin/sh -- # -*- perl -*- -T > eval 'exec perl -wTS $0 ${1+"$@"}' > if 0; > print "Hello there.\n"; > ------------------------------------------------------------------------ > > FreeBSD 4.7: > $ /tmp/try.pl > Hello there. > > Solaris 8: > $ /tmp/try.pl > Hello there. > > SuSE Linux 7.0, 7.3, 8.1 (2.4.19 kernel, binfmt_script.c identical to > 2.4.20 BK): > $ /tmp/try.pl > /bin/sh: -- # -*- perl -*- -T: invalid option looks correct. The interpreter (/bin/sh) has got everything after its name. IOW: "-- # -*- perl -*- -T" It's just solaris' shell (/bin/sh) just ignores options starting with "--". And freebsd's as well. Anyway - it's bash, not the bin_fmt. > Usage: /bin/sh [GNU long option] [option] ... > /bin/sh [GNU long option] [option] script-file ... > [...] ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: #! incompatible -- binfmt_script.c broken? 2002-12-04 14:26 ` Alex Riesen @ 2002-12-04 15:23 ` Matthew Garrett 2002-12-04 18:37 ` Matthias Andree 2002-12-05 0:42 ` Horst von Brand 2 siblings, 0 replies; 639+ messages in thread From: Matthew Garrett @ 2002-12-04 15:23 UTC (permalink / raw) To: linux-kernel Alexander Riesen wrote: >On Wed, Dec 04, 2002 at 12:34:19PM +0100, Matthias Andree wrote: >> SuSE Linux 7.0, 7.3, 8.1 (2.4.19 kernel, binfmt_script.c identical to >> 2.4.20 BK): >> $ /tmp/try.pl >> /bin/sh: -- # -*- perl -*- -T: invalid option > >looks correct. The interpreter (/bin/sh) has got everything after >its name. IOW: "-- # -*- perl -*- -T" >It's just solaris' shell (/bin/sh) just ignores options starting with >"--". And freebsd's as well. FreeBSD splits #! magic strings on whitespace and passes multiple arguments. Linux passes everything after the first whitespace as a single argument but strips trailing whitespace. NetBSD does the same as Linux but passes trailing whitespace as part of the argument. >Anyway - it's bash, not the bin_fmt. Bash is (correctly) complaining that it's been passed an invalid argument, but the reason for the different behaviour between it and FreeBSD is because of binfmt_script. There's no clearly defined standard for how this should behave. -- Matthew Garrett | mjg59-chiark.mail.linux-rutgers.kernel@srcf.ucam.org ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: #! incompatible -- binfmt_script.c broken? 2002-12-04 14:26 ` Alex Riesen 2002-12-04 15:23 ` Matthew Garrett @ 2002-12-04 18:37 ` Matthias Andree 2002-12-04 20:37 ` Matthew Garrett 2002-12-05 0:42 ` Horst von Brand 2 siblings, 1 reply; 639+ messages in thread From: Matthias Andree @ 2002-12-04 18:37 UTC (permalink / raw) To: linux-kernel; +Cc: Alan Cox On Wed, 04 Dec 2002, Alex Riesen wrote: > > SuSE Linux 7.0, 7.3, 8.1 (2.4.19 kernel, binfmt_script.c identical to > > 2.4.20 BK): > > $ /tmp/try.pl > > /bin/sh: -- # -*- perl -*- -T: invalid option > > looks correct. Nope. It cannot be correct if it breaks compatibility without giving us any advantage. > The interpreter (/bin/sh) has got everything after > its name. IOW: "-- # -*- perl -*- -T" Yes, as SINGLE argument. Therefore, Perl programs break if they use this procedure recommended by "man perlrun". I don't care WHY it works everywhere else, I want this incompatibility fixed and I'm not going through a flame war as with the 4.4BSD SIOCGIFNETMASK issue again. This is not negotiable. BTW, 2.2 is also affected. Think of someone using /usr/bin/env -i /path/to/program -- won't work on Linux, but works on FreeBSD. I cannot see technical reasons why this should remain unfixed. We have enough braindead frivulous incompatibilities in Linux. > It's just solaris' shell (/bin/sh) just ignores options starting with > "--". And freebsd's as well. > Anyway - it's bash, not the bin_fmt. Nope, zsh as /bin/sh complains as well: /bin/sh: no such option: # _*_ perl _*_ _T so does pdksh: /bin/sh: /bin/sh: --: unknown option ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: #! incompatible -- binfmt_script.c broken? 2002-12-04 18:37 ` Matthias Andree @ 2002-12-04 20:37 ` Matthew Garrett 2002-12-04 22:28 ` H. Peter Anvin 2002-12-05 11:55 ` Matthias Andree 0 siblings, 2 replies; 639+ messages in thread From: Matthew Garrett @ 2002-12-04 20:37 UTC (permalink / raw) To: linux-kernel Matthias Andre wrote: >On Wed, 04 Dec 2002, Alex Riesen wrote: >> The interpreter (/bin/sh) has got everything after >> its name. IOW: "-- # -*- perl -*- -T" > >Yes, as SINGLE argument. Therefore, Perl programs break if they use this >procedure recommended by "man perlrun". > >I don't care WHY it works everywhere else, I want this incompatibility >fixed and I'm not going through a flame war as with the 4.4BSD >SIOCGIFNETMASK issue again. This is not negotiable. See http://www.uni-ulm.de/~s_smasch/various/shebang/ . FreeBSD is the *only* OS to pass multiple arguments. SUS says nothing about it, and pretty much every single implementation varies. It does not work everywhere else. >We have enough braindead frivulous incompatibilities in Linux. The *only* thing you can reliably use in #! lines is an interpreter followed by a single argument with no trailing space. On NetBSD with bash as /bin/sh: mjg59@cysteine:/tmp$ cat foo.pl #!/bin/sh -- # -*- perl -*- -p mjg59@cysteine:/tmp$ ./foo.pl /bin/sh: -- # -*- perl -*- -p: unrecognized option File a bug against perlrun(1). -- Matthew Garrett | mjg59-chiark.mail.linux-rutgers.kernel@srcf.ucam.org ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: #! incompatible -- binfmt_script.c broken? 2002-12-04 20:37 ` Matthew Garrett @ 2002-12-04 22:28 ` H. Peter Anvin 2002-12-05 11:55 ` Matthias Andree 1 sibling, 0 replies; 639+ messages in thread From: H. Peter Anvin @ 2002-12-04 22:28 UTC (permalink / raw) To: linux-kernel Followup to: <E18JgHI-0006Gx-00@chiark.greenend.org.uk> By author: Matthew Garrett <mgarrett@chiark.greenend.org.uk> In newsgroup: linux.dev.kernel > > The *only* thing you can reliably use in #! lines is an interpreter > followed by a single argument with no trailing space. On NetBSD with > bash as /bin/sh: > > mjg59@cysteine:/tmp$ cat foo.pl > #!/bin/sh -- # -*- perl -*- -p > mjg59@cysteine:/tmp$ ./foo.pl > /bin/sh: -- # -*- perl -*- -p: unrecognized option > > File a bug against perlrun(1). > I personally think that it would be nice to split it by spaces (but yes, ' " and \ need to be handled for this to work.) It allows things like using env to spawn a binary where the location of the interpreter is to be extracted from PATH, for example. -hpa -- <hpa@transmeta.com> at work, <hpa@zytor.com> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt <amsp@zytor.com> ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: #! incompatible -- binfmt_script.c broken? 2002-12-04 20:37 ` Matthew Garrett 2002-12-04 22:28 ` H. Peter Anvin @ 2002-12-05 11:55 ` Matthias Andree 1 sibling, 0 replies; 639+ messages in thread From: Matthias Andree @ 2002-12-05 11:55 UTC (permalink / raw) To: linux-kernel On Wed, 04 Dec 2002, Matthew Garrett wrote: > See http://www.uni-ulm.de/~s_smasch/various/shebang/ . FreeBSD is the > *only* OS to pass multiple arguments. SUS says nothing about it, and > pretty much every single implementation varies. It does not work > everywhere else. Now that's a useful resource. > File a bug against perlrun(1). Hum, no. In fact, the offending line will work: - on systems that only pass on the first argument or none at all - on systems that split on whitespace - on /bin/sh that silently ignore options starting with "-- " So it's somewhat compatible already. ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: #! incompatible -- binfmt_script.c broken? 2002-12-04 14:26 ` Alex Riesen 2002-12-04 15:23 ` Matthew Garrett 2002-12-04 18:37 ` Matthias Andree @ 2002-12-05 0:42 ` Horst von Brand 2002-12-05 11:38 ` Matthias Andree 2 siblings, 1 reply; 639+ messages in thread From: Horst von Brand @ 2002-12-05 0:42 UTC (permalink / raw) To: Alexander.Riesen; +Cc: Matthias Andree, linux-kernel Alex Riesen <Alexander.Riesen@synopsys.com> said: [...] > looks correct. The interpreter (/bin/sh) has got everything after > its name. IOW: "-- # -*- perl -*- -T" > It's just solaris' shell (/bin/sh) just ignores options starting with > "--". And freebsd's as well. And Linux's too. Try it. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: #! incompatible -- binfmt_script.c broken? 2002-12-05 0:42 ` Horst von Brand @ 2002-12-05 11:38 ` Matthias Andree 0 siblings, 0 replies; 639+ messages in thread From: Matthias Andree @ 2002-12-05 11:38 UTC (permalink / raw) To: Horst von Brand; +Cc: Alexander.Riesen, Matthias Andree, linux-kernel On Wed, 04 Dec 2002, Horst von Brand wrote: > Alex Riesen <Alexander.Riesen@synopsys.com> said: > > [...] > > > looks correct. The interpreter (/bin/sh) has got everything after > > its name. IOW: "-- # -*- perl -*- -T" > > It's just solaris' shell (/bin/sh) just ignores options starting with > > "--". And freebsd's as well. > > And Linux's too. Try it. Is there the "Linux's /bin/sh"? I believe most distributions use GNU bash for /bin/sh, and that certainly does NOT ignore these, but parse. The problem is that binfmt_script.c does not split the remainder of the line at whitespace. Assuming your current working directory does not have files that match -*-: $ /bin/sh '-- # -*- perl -*- -T' /bin/sh: -- # -*- perl -*- -T: invalid option Usage: /bin/sh [GNU long option] [option] ... /bin/sh [GNU long option] [option] script-file ... GNU long options: --debug ... $ /bin/bash -- # -*- perl -*- -T $ $ /usr/bin/ksh '-- # -*- perl -*- -T' /usr/bin/ksh: /usr/bin/ksh: --: unknown option $ /usr/bin/ksh -- # -*- perl -*- -T $ $ /usr/bin/zsh '-- # -*- perl -*- -T' /usr/bin/zsh: no such option: # _*_ perl _*_ _T $ /usr/bin/zsh -- # -*- perl -*- -T $ -- Matthias Andree ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: #! incompatible -- binfmt_script.c broken? 2002-12-04 11:34 ` #! incompatible -- binfmt_script.c broken? Matthias Andree 2002-12-04 14:26 ` Alex Riesen @ 2002-12-04 22:25 ` Andries Brouwer 2002-12-05 0:10 ` Horst von Brand 2 siblings, 0 replies; 639+ messages in thread From: Andries Brouwer @ 2002-12-04 22:25 UTC (permalink / raw) To: Linux-Kernel mailing list On Wed, Dec 04, 2002 at 12:34:19PM +0100, Matthias Andree wrote: > I tried some of the Perl magic tricks shown in the perlrun man page with > Linux 2.4.19; consider this Perl one-liner. It works on FreeBSD and > Solaris, but fails on Linux. > > ------------------------------------------------------------------------ > #!/bin/sh -- # -*- perl -*- -T > eval 'exec perl -wTS $0 ${1+"$@"}' > if 0; > print "Hello there.\n"; > ------------------------------------------------------------------------ > > FreeBSD 4.7 and Solaris 8: > $ /tmp/try.pl > Hello there. > > SuSE Linux 7.0, 7.3, 8.1 > $ /tmp/try.pl > /bin/sh: -- # -*- perl -*- -T: invalid option See http://www.cwi.nl/~aeb/std/hashexclam-1.html There I wrote long ago: ... The interpreter is called with a parameter list consisting of four groups of arguments: arg0, argi, argn, args. The first group, arg0, consists of one argument. For SysVR4, SunOS, Solaris, IRIX, BSDI, DU, AIX, Unixware, Linux this argument is /path/interpreter. For FreeBSD and HPUX this argument is /scriptpath/script. The second group, argi, consists of the 0 or 1 or perhaps more arguments to the interpreter found in the #! line. Thus, this group is empty if there is no nonblank text following the interpreter name in the #! line. If there is such nonblank text then for SysVR4, SunOS, Solaris, IRIX, HPUX, AIX, Unixware, Linux, FreeBSD this group consists of precisely one argument, as in the example above where argi consists of the single argument "-a -b". BSDI splits the text following the interpreter name into zero or more arguments, and hence has an argi consisting of the two arguments "-a", "-b" in the above example. (Also Plan9 allows several arguments here.) ... Instead of trying this complicated perl example, you might try small scripts that test one thing at a time. I am interested in the results of such tests on various operating systems. But regarding your post: there is no reason to assume that Linux does something "wrong" here if it disagrees with Solaris and FreeBSD. There is no standard and no uniformity here. Andries ^ permalink raw reply [flat|nested] 639+ messages in thread
* Re: #! incompatible -- binfmt_script.c broken? 2002-12-04 11:34 ` #! incompatible -- binfmt_script.c broken? Matthias Andree 2002-12-04 14:26 ` Alex Riesen 2002-12-04 22:25 ` Andries Brouwer @ 2002-12-05 0:10 ` Horst von Brand 2 siblings, 0 replies; 639+ messages in thread From: Horst von Brand @ 2002-12-05 0:10 UTC (permalink / raw) To: Linux-Kernel mailing list Matthias Andree <matthias.andree@gmx.de> said: > I tried some of the Perl magic tricks shown in the perlrun man page with > Linux 2.4.19; consider this Perl one-liner. It works on FreeBSD and > Solaris, but fails on Linux. Looking at binfmt_script.c, I believe the > "pass the rest of the line as the first argument to the interpreter" is > the problem with Linux. Haven't yet figured if the other boxes just use > the interpreter, ignoring the arguments or if they are doing argument > splitting. > > ------------------------------------------------------------------------ > #!/bin/sh -- # -*- perl -*- -T > eval 'exec perl -wTS $0 ${1+"$@"}' > if 0; > print "Hello there.\n"; > ------------------------------------------------------------------------ > [...] > SuSE Linux 7.0, 7.3, 8.1 (2.4.19 kernel, binfmt_script.c identical to > 2.4.20 BK): > $ /tmp/try.pl > /bin/sh: -- # -*- perl -*- -T: invalid option > Usage: /bin/sh [GNU long option] [option] ... > /bin/sh [GNU long option] [option] script-file ... > [...] RH 8.0 with linux-2.5.50 (some early bk version, last CSET is 1.849) gives the same. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 639+ messages in thread
end of thread, other threads:[~2006-08-09 15:52 UTC | newest] Thread overview: 639+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2005-12-03 13:56 RFC: Starting a stable kernel series off the 2.6 kernel Adrian Bunk 2005-12-03 14:29 ` Jesper Juhl 2005-12-03 20:19 ` Greg KH 2005-12-03 21:04 ` M. 2005-12-03 21:37 ` James Courtier-Dutton [not found] ` <f0cc38560512031254j3b28d579s539be721c247c10a@mail.gmail.com> 2005-12-03 21:12 ` Greg KH 2005-12-03 21:31 ` M. 2005-12-03 21:38 ` Arjan van de Ven 2005-12-03 21:53 ` M. 2005-12-03 22:26 ` Greg KH 2005-12-04 7:56 ` Arjan van de Ven [not found] ` <f0cc38560512040657i58cc08efqa8596c357fcea82e@mail.gmail.com> 2005-12-04 15:10 ` Arjan van de Ven 2005-12-04 16:11 ` Matthias Andree 2005-12-04 16:41 ` Arjan van de Ven 2005-12-04 20:08 ` Paul Jackson [not found] ` <f0cc38560512040724re5114c2y76bb34d63c9c5ae0@mail.gmail.com> 2005-12-04 22:47 ` Greg KH [not found] ` <f0cc38560512041503y7abd1f12rbce8bdac0ebdf30d@mail.gmail.com> 2005-12-04 23:12 ` Greg KH 2005-12-05 1:03 ` Horst von Brand 2005-12-05 19:35 ` Bill Davidsen 2005-12-03 21:54 ` Greg KH 2005-12-06 1:19 ` Florian Weimer 2005-12-06 17:55 ` Greg KH 2005-12-03 22:51 ` Adrian Bunk 2005-12-03 23:28 ` Greg KH 2005-12-03 23:35 ` Chris Wright 2005-12-06 0:37 ` Rob Landley 2005-12-07 21:38 ` Nix 2005-12-04 8:07 ` Arjan van de Ven 2005-12-05 20:33 ` Florian Weimer 2005-12-06 1:10 ` Horst von Brand 2005-12-06 10:46 ` Matthias Andree 2005-12-06 14:01 ` Florian Weimer 2005-12-06 16:52 ` Gene Heskett 2005-12-04 3:48 ` Jesper Juhl 2005-12-04 11:56 ` Matthias Andree 2005-12-04 23:24 ` Greg KH 2005-12-05 6:26 ` Willy Tarreau 2005-12-05 10:00 ` Matthias Andree 2005-12-05 10:55 ` Lars Marowsky-Bree 2005-12-05 11:34 ` Willy Tarreau 2005-12-05 11:40 ` Lars Marowsky-Bree 2005-12-05 12:01 ` Willy Tarreau 2005-12-05 12:24 ` Bernd Eckenfels 2005-12-05 12:26 ` Arjan van de Ven 2005-12-06 17:54 ` Greg KH 2005-12-06 18:57 ` John Kelly 2005-12-06 21:55 ` Adrian Bunk 2005-12-06 22:40 ` John Kelly 2005-12-05 18:51 ` Adrian Bunk 2005-12-06 17:50 ` Greg KH 2005-12-06 22:50 ` Policy for reverting user ABI breaking patches was " Andi Kleen 2005-12-06 18:42 ` Dmitry Torokhov 2005-12-06 14:32 ` Florian Weimer [not found] ` <6f6293f10512060855p79fb5e91ke6fca33f96cb1750@mail.gmail.com> 2005-12-06 17:47 ` Greg KH 2005-12-06 23:27 ` David S. Miller 2005-12-06 19:01 ` Lee Revell 2005-12-04 17:00 ` Jakob Oestergaard 2005-12-04 22:39 ` Greg KH 2005-12-05 15:17 ` Jakob Oestergaard 2005-12-05 15:44 ` Pekka Enberg 2005-12-05 17:17 ` Jakob Oestergaard 2005-12-06 17:44 ` Greg KH 2005-12-06 21:16 ` Bill Davidsen 2005-12-07 14:38 ` Massimiliano Hofer 2005-12-07 16:05 ` Horst von Brand 2005-12-07 16:29 ` Massimiliano Hofer 2005-12-05 14:48 ` Florian Weimer 2005-12-06 17:46 ` Greg KH 2005-12-03 14:31 ` Ben Collins 2005-12-03 19:35 ` Adrian Bunk 2005-12-03 19:57 ` Lee Revell 2005-12-03 21:04 ` M. 2005-12-03 22:58 ` Matthias Andree 2005-12-03 23:49 ` Lee Revell 2005-12-05 21:05 ` Florian Weimer 2005-12-05 21:41 ` Lee Revell 2005-12-05 23:00 ` Florian Weimer 2005-12-05 23:06 ` Bernd Petrovitsch 2005-12-06 0:08 ` Florian Weimer 2005-12-05 21:00 ` Florian Weimer 2005-12-05 21:06 ` Arjan van de Ven 2005-12-06 0:43 ` Florian Weimer 2005-12-06 11:21 ` Matthias Andree 2005-12-06 15:10 ` Florian Weimer 2005-12-06 16:45 ` Dmitry Torokhov 2005-12-07 11:29 ` Matthias Andree 2005-12-07 13:54 ` Horst von Brand 2005-12-08 3:29 ` Dmitry Torokhov 2005-12-08 8:29 ` Matthias Andree 2005-12-10 0:22 ` Rob Landley 2005-12-06 20:35 ` Alan Cox 2005-12-05 23:03 ` Bill Davidsen 2005-12-06 1:48 ` Jeff Garzik 2005-12-06 11:23 ` Matthias Andree 2005-12-06 19:48 ` Bill Davidsen 2005-12-06 1:56 ` Horst von Brand 2005-12-03 14:36 ` Arjan van de Ven 2005-12-03 15:23 ` Adrian Bunk 2005-12-03 15:39 ` Arjan van de Ven 2005-12-04 13:53 ` Denis Vlasenko 2005-12-05 9:47 ` Michael Frank 2005-12-06 0:54 ` Horst von Brand 2005-12-06 17:08 ` Michael Frank 2005-12-03 16:27 ` Matthias Andree 2005-12-03 16:40 ` Otavio Salvador 2005-12-03 16:58 ` David Ranson 2005-12-03 17:13 ` Steven Rostedt 2005-12-03 17:17 ` David Ranson 2005-12-03 17:53 ` Adrian Bunk 2005-12-03 18:17 ` newbie - mdadm create raid1 mirrors on large drives Larry Bates 2005-12-03 22:23 ` Matthias Andree 2005-12-04 22:13 ` Neil Brown 2005-12-04 22:13 ` Neil Brown 2005-12-03 18:34 ` RFC: Starting a stable kernel series off the 2.6 kernel David Ranson 2005-12-03 22:27 ` Matthias Andree 2005-12-03 22:34 ` Lee Revell 2005-12-03 22:50 ` Matthias Andree 2005-12-04 0:20 ` Greg KH 2005-12-04 4:46 ` Luke-Jr 2005-12-04 15:06 ` Michael Frank 2005-12-04 23:22 ` Greg KH 2005-12-05 5:59 ` Luke-Jr 2005-12-06 0:34 ` Rob Landley 2005-12-06 10:34 ` Luke-Jr 2005-12-06 19:17 ` Rob Landley 2005-12-06 17:38 ` Greg KH 2005-12-06 14:58 ` Bill Davidsen 2005-12-06 17:59 ` Greg KH 2005-12-06 21:10 ` Bill Davidsen 2005-12-06 21:51 ` Kay Sievers 2005-12-10 2:16 ` Rob Landley 2005-12-10 4:04 ` Greg KH 2005-12-05 22:47 ` Rob Landley 2005-12-05 23:05 ` Benjamin LaHaise 2005-12-06 3:19 ` Rob Landley 2005-12-06 3:32 ` Benjamin LaHaise 2005-12-06 5:49 ` Rob Landley 2005-12-06 10:51 ` Matthias Andree 2005-12-06 3:15 ` Greg KH 2005-12-06 3:23 ` Rob Landley 2005-12-03 22:36 ` David Ranson 2005-12-03 22:50 ` Matthias Andree 2005-12-06 14:59 ` Bill Davidsen 2005-12-04 1:04 ` Horst von Brand 2005-12-04 12:07 ` Matthias Andree 2005-12-04 19:29 ` Horst von Brand 2005-12-06 20:01 ` Bill Davidsen 2005-12-05 20:43 ` Bill Davidsen 2005-12-05 3:31 ` Rob Landley 2005-12-05 16:17 ` Mark Lord 2005-12-05 16:28 ` Lee Revell 2005-12-05 16:44 ` Matthias Andree 2005-12-05 17:17 ` Lee Revell 2005-12-05 17:55 ` Matthias Andree 2005-12-05 20:52 ` Florian Weimer 2005-12-05 21:21 ` Steven Rostedt 2005-12-05 23:09 ` Rob Landley 2005-12-06 0:54 ` Steven Rostedt 2005-12-06 1:10 ` Florian Weimer 2005-12-06 1:26 ` Steven Rostedt 2005-12-06 18:06 ` Horst von Brand 2005-12-06 3:22 ` Rob Landley 2005-12-06 1:06 ` Florian Weimer 2005-12-06 11:09 ` Matthias Andree 2005-12-05 17:58 ` Rob Landley 2005-12-06 18:51 ` Bill Davidsen 2005-12-07 15:48 ` Arjan van de Ven 2005-12-07 18:40 ` Horst von Brand 2005-12-07 18:14 ` Rob Landley 2005-12-10 8:35 ` ipw2200 [was Re: RFC: Starting a stable kernel series off the 2.6 kernel] Pavel Machek 2005-12-11 5:30 ` Rob Landley 2005-12-11 8:37 ` Pavel Machek 2005-12-11 9:12 ` Rob Landley 2005-12-12 11:49 ` Pavel Machek 2005-12-14 12:26 ` Rob Landley 2004-12-14 16:01 ` Pavel Machek 2005-12-11 16:26 ` Horst von Brand 2005-12-12 17:34 ` Ben Slusky 2005-12-12 20:02 ` Rob Landley [not found] ` <439DE10E.4080901@tmr.com> 2005-12-12 21:52 ` Ben Slusky 2005-12-15 2:38 ` ipw2200 Miles Bader 2005-12-10 13:41 ` RFC: Starting a stable kernel series off the 2.6 kernel Bill Davidsen 2005-12-10 17:05 ` Douglas McNaught 2005-12-11 5:52 ` Rob Landley 2005-12-12 3:25 ` Bill Davidsen 2005-12-11 5:33 ` Rob Landley 2005-12-05 21:22 ` Bill Davidsen 2005-12-06 14:59 ` Bill Davidsen 2005-12-05 18:44 ` Adrian Bunk 2005-12-03 18:21 ` Mark Lord 2005-12-03 19:22 ` Linus Torvalds 2005-12-04 3:32 ` Mark Lord 2005-12-03 22:21 ` Matthias Andree 2005-12-03 22:29 ` Greg KH 2005-12-03 22:41 ` Matthias Andree 2005-12-03 22:48 ` Steven Rostedt 2005-12-03 17:22 ` Arjan van de Ven 2005-12-03 17:35 ` M. 2005-12-03 23:05 ` Matthias Andree 2005-12-03 23:37 ` Greg KH 2005-12-04 0:52 ` Jeff V. Merkey 2005-12-04 12:12 ` Matthias Andree 2005-12-04 12:32 ` Arjan van de Ven 2005-12-04 13:28 ` Matthias Andree 2005-12-04 13:35 ` Arjan van de Ven 2005-12-04 14:25 ` Matthias Andree 2005-12-04 14:50 ` Arjan van de Ven 2005-12-04 15:08 ` Matthias Andree 2005-12-04 15:11 ` Arjan van de Ven 2005-12-04 15:36 ` Andreas Schwab 2005-12-04 16:17 ` Matthias Andree 2005-12-05 3:09 ` Joel Becker 2005-12-06 20:13 ` Alan Cox 2005-12-04 15:20 ` Arjan van de Ven 2005-12-04 16:23 ` Matthias Andree 2005-12-04 15:25 ` Richard Knutsson 2005-12-04 15:23 ` Arjan van de Ven 2005-12-05 23:51 ` Rob Landley 2005-12-06 20:40 ` Matan Peled 2005-12-04 19:57 ` Horst von Brand 2005-12-04 21:35 ` Bernd Petrovitsch 2005-12-05 0:43 ` Jeff V. Merkey 2005-12-05 9:06 ` Bernd Petrovitsch 2005-12-06 0:41 ` Horst von Brand 2005-12-06 9:38 ` Bernd Petrovitsch 2005-12-03 18:43 ` Golden rule: don't break userland (was Re: RFC: Starting a stable kernel series off the 2.6 kernel) Jeff Garzik 2005-12-03 20:34 ` Greg KH 2005-12-04 1:40 ` Dmitry Torokhov 2005-12-04 23:29 ` Greg KH 2005-12-05 21:41 ` Dmitry Torokhov 2005-12-03 22:35 ` Matthias Andree 2005-12-04 15:37 ` Michael Frank 2005-12-05 3:23 ` RFC: Starting a stable kernel series off the 2.6 kernel Rob Landley 2005-12-10 19:48 ` Ryan Anderson 2005-12-03 18:39 ` Dr. David Alan Gilbert 2005-12-03 20:59 ` Lars Marowsky-Bree 2005-12-03 21:13 ` Dave Jones 2005-12-03 21:18 ` Lars Marowsky-Bree 2005-12-03 21:42 ` [OT] " Michael Buesch 2005-12-03 22:40 ` Adrian Bunk 2005-12-03 23:02 ` Matthias Andree 2005-12-03 23:09 ` Dave Jones 2005-12-06 0:14 ` Florian Weimer 2005-12-06 13:20 ` Lars Marowsky-Bree 2005-12-06 13:37 ` Florian Weimer 2005-12-04 12:56 ` Indrek Kruusa 2005-12-04 13:05 ` Arjan van de Ven 2005-12-04 15:26 ` Indrek Kruusa 2005-12-05 23:43 ` Rob Landley 2005-12-05 19:30 ` Bill Davidsen 2005-12-05 23:25 ` Adrian Bunk 2005-12-06 11:28 ` Matthias Andree 2005-12-06 13:25 ` Lars Marowsky-Bree 2005-12-06 20:46 ` Bill Davidsen 2005-12-12 14:45 ` Felix Oxley 2005-12-12 17:17 ` Horst von Brand 2005-12-12 18:53 ` Felix Oxley 2005-12-13 13:17 ` Horst von Brand 2005-12-14 0:09 ` Felix Oxley -- strict thread matches above, loose matches on Subject: below -- 2006-07-21 19:46 the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Hans Reiser 2006-07-22 0:11 ` Losing Technologists [was: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Neil Brown 2006-07-22 0:18 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Adrian Bunk 2006-07-22 19:19 ` Diego Calleja 2006-07-22 13:02 ` Theodore Tso 2006-07-22 14:30 ` Eric Sandeen 2006-07-22 18:33 ` Hans Reiser 2006-07-22 20:29 ` Jeff Garzik 2006-07-23 7:20 ` Hans Reiser 2006-07-23 9:12 ` Matt Heler 2006-07-24 4:01 ` Hans Reiser 2006-07-24 8:54 ` Matthias Andree 2006-07-24 8:13 ` Hans Reiser 2006-07-24 10:25 ` Matthias Andree 2006-07-24 11:34 ` Christian Iversen 2006-07-24 12:37 ` Erik Mouw 2006-07-24 12:37 ` Erik Mouw 2006-07-24 16:57 ` Mike Benoit 2006-07-24 17:35 ` Matthias Andree 2006-07-24 18:28 ` Valdis.Kletnieks 2006-07-24 18:06 ` Horst H. von Brand 2006-07-24 20:37 ` Mike Benoit 2006-07-24 21:22 ` Jan-Benedict Glaw 2006-07-24 21:51 ` Horst H. von Brand 2006-07-24 22:51 ` Mike Benoit 2006-07-25 15:08 ` Denis Vlasenko 2006-07-25 20:49 ` Matthias Andree 2006-07-25 23:04 ` David Masover 2006-07-26 11:20 ` Matthias Andree 2006-07-26 11:26 ` Matthias Andree 2006-07-26 13:02 ` Bernd Eckenfels 2006-07-26 18:54 ` Buddy Lucas 2006-07-27 1:29 ` David Masover 2006-07-26 0:29 ` David Masover 2006-07-26 0:36 ` David Lang 2006-07-26 0:47 ` David Masover 2006-07-31 10:58 ` Adrian Ulrich 2006-07-31 14:47 ` Matthias Andree 2006-07-31 15:59 ` Adrian Ulrich 2006-07-31 15:59 ` Adrian Ulrich 2006-07-31 16:22 ` Jan-Benedict Glaw 2006-07-31 16:44 ` David Masover 2006-07-31 17:34 ` Bernd Eckenfels 2006-07-31 18:36 ` Jan-Benedict Glaw 2006-07-31 16:44 ` Rudy Zijlstra 2006-07-31 17:20 ` Jan-Benedict Glaw 2006-07-31 17:32 ` Jan-Benedict Glaw 2006-07-31 17:46 ` Dan Oglesby 2006-07-31 18:11 ` Matthias Andree 2006-07-31 18:43 ` Jan-Benedict Glaw 2006-07-31 19:17 ` Clay Barnes 2006-07-31 19:29 ` Jan-Benedict Glaw 2006-07-31 20:00 ` David Masover 2006-07-31 20:53 ` the " 'official' point of view" expressed by kernelnewbies.orgregarding " David Lang 2006-07-31 21:16 ` David Masover 2006-07-31 21:14 ` the " 'official' point of view" expressed by kernelnewbies.org regarding " Bernd Schubert 2006-08-01 14:28 ` Horst H. von Brand 2006-08-01 14:52 ` Adrian Ulrich 2006-08-01 14:52 ` Adrian Ulrich 2006-08-01 15:29 ` Alan Cox 2006-08-01 16:44 ` David Masover 2006-08-01 17:04 ` Gregory Maxwell 2006-08-01 12:01 ` Hans Reiser 2006-08-01 17:41 ` David Masover 2006-08-01 18:14 ` Adrian Ulrich 2006-08-01 17:19 ` Alan Cox 2006-08-01 11:36 ` Hans Reiser 2006-08-01 17:40 ` David Masover 2006-08-01 19:27 ` Krzysztof Halasa 2006-08-03 13:58 ` Matthias Andree 2006-08-03 18:48 ` Gregory Maxwell 2006-08-03 23:25 ` Checksumming blocks? [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Russell Leighton 2006-08-03 23:35 ` Matthias Andree 2006-08-04 0:01 ` Russell Leighton 2006-08-04 7:56 ` Toby Thain 2006-08-04 11:55 ` Russell Leighton 2006-08-04 11:41 ` Tomasz Torcz 2006-08-04 20:42 ` Horst H. von Brand 2006-08-04 20:51 ` David Masover 2006-08-01 18:11 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Adrian Ulrich 2006-08-01 18:21 ` Ric Wheeler 2006-08-01 11:41 ` Hans Reiser 2006-08-03 14:03 ` Matthias Andree 2006-08-03 15:44 ` Edward Shishkin 2006-08-03 17:26 ` Hans Reiser 2006-08-04 17:04 ` Edward Shishkin 2006-08-04 18:57 ` Antonio Vargas 2006-08-05 1:02 ` Hans Reiser 2006-08-05 0:56 ` Hans Reiser 2006-08-06 22:19 ` Edward Shishkin 2006-08-09 8:40 ` Hans Reiser 2006-08-09 11:53 ` Jan Engelhardt 2006-08-09 15:48 ` David Masover 2006-08-07 7:57 ` Matthias Andree 2006-08-08 11:06 ` Edward Shishkin 2006-08-01 19:11 ` David Masover 2006-08-03 14:03 ` Matthias Andree 2006-08-03 16:50 ` Theodore Tso 2006-08-01 16:57 ` David Masover 2006-08-06 22:59 ` Pavel Machek 2006-08-06 23:36 ` David Masover 2006-08-09 8:37 ` Hans Reiser 2006-08-09 9:48 ` Pavel Machek 2006-08-09 9:15 ` Hans Reiser 2006-08-09 15:52 ` David Masover 2006-07-31 19:42 ` Alan Cox 2006-07-31 19:34 ` Clay Barnes 2006-07-31 21:00 ` Gregory Maxwell 2006-07-31 21:40 ` Alan Cox 2006-07-31 21:43 ` David Masover 2006-07-31 21:54 ` Jeff V. Merkey 2006-07-31 22:02 ` Jeff V. Merkey 2006-07-31 22:21 ` Jeff V. Merkey 2006-07-31 22:56 ` Nate Diller 2006-07-31 23:52 ` Jeff V. Merkey 2006-07-31 23:43 ` Nate Diller 2006-08-01 0:15 ` Jeffrey V. Merkey 2006-08-01 6:28 ` Toby Thain 2006-07-31 22:17 ` Matthias Andree 2006-07-31 21:21 ` Łukasz Mierzwa 2006-07-31 21:21 ` Łukasz Mierzwa 2006-07-31 16:47 ` Dan Oglesby 2006-07-31 17:16 ` Jan-Benedict Glaw 2006-07-31 17:34 ` Matthias Andree 2006-07-31 17:44 ` Dan Oglesby 2006-07-31 16:54 ` Matthias Andree 2006-07-31 17:56 ` Adrian Ulrich 2006-07-31 20:07 ` Matthias Andree 2006-07-31 20:32 ` Adrian Ulrich 2006-07-31 20:32 ` Adrian Ulrich 2006-07-31 19:41 ` Theodore Tso 2006-07-31 22:53 ` Matthias Andree 2006-08-01 2:33 ` Hans Reiser 2006-07-31 19:18 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Horst H. von Brand 2006-07-31 20:31 ` Toby Thain 2006-07-31 20:57 ` Adrian Ulrich 2006-07-31 20:57 ` Adrian Ulrich 2006-07-31 21:54 ` Matthias Andree 2006-07-31 23:21 ` Nate Diller 2006-07-31 23:27 ` David Lang 2006-07-31 23:50 ` Nate Diller 2006-07-31 23:55 ` David Lang 2006-08-01 0:05 ` Nate Diller 2006-08-01 1:02 ` Matthias Andree 2006-08-01 1:25 ` Nate Diller 2006-08-01 1:31 ` David Masover 2006-08-01 3:00 ` Theodore Tso 2006-08-01 3:38 ` David Masover 2006-08-01 3:47 ` Timothy Webster 2006-08-01 4:24 ` David Masover 2006-08-01 4:41 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed " David Lang 2006-08-01 4:57 ` David Masover 2006-08-01 6:48 ` Theodore Tso 2006-08-01 2:40 ` Hans Reiser 2006-08-01 10:59 ` Jan Engelhardt 2006-08-01 10:56 ` Hans Reiser 2006-08-01 7:24 ` Avi Kivity 2006-08-01 9:25 ` Matthias Andree 2006-08-01 9:38 ` Avi Kivity 2006-08-01 17:03 ` David Masover 2006-08-01 4:38 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed " Mike Benoit 2006-08-01 4:21 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed " David Lang 2006-08-01 4:32 ` David Masover 2006-08-01 4:53 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressedby " David Lang 2006-08-01 5:59 ` David Masover 2006-08-01 23:50 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by " Ian Stirling 2006-08-02 2:29 ` Kyle Moffett 2006-08-02 14:28 ` Solaris ZFS on Linux Krzysztof Halasa 2006-08-02 18:12 ` Ian Stirling 2006-08-03 2:20 ` Wil Reichert 2006-08-03 9:32 ` Helge Hafting 2006-08-02 3:52 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view"expressed by kernelnewbies.org regarding reiser4 inclusion] David Masover 2006-08-01 7:51 ` Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed " Adrian Ulrich 2006-08-01 7:51 ` Adrian Ulrich 2006-08-01 9:09 ` Matthias Andree 2006-08-01 9:57 ` Avi Kivity 2006-08-01 10:57 ` Jan Engelhardt 2006-08-01 13:15 ` Matthias Andree 2006-08-01 13:40 ` Jan Engelhardt 2006-07-31 22:08 ` Horst H. von Brand 2006-07-31 23:25 ` Nate Diller 2006-08-01 7:47 ` Adrian Ulrich 2006-08-01 7:47 ` Adrian Ulrich 2006-08-01 10:40 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Helge Hafting 2006-08-01 10:59 ` venom 2006-07-31 16:52 ` David Masover 2006-08-01 6:22 ` Jan Engelhardt 2006-07-24 20:28 ` Lex Lyamin 2006-07-26 13:17 ` Pavel Machek 2006-07-27 15:39 ` Grzegorz Kulewski 2006-07-27 17:28 ` Matthias Andree 2006-07-28 2:40 ` Hans Reiser 2006-07-27 17:56 ` Jeff Garzik 2006-07-27 19:06 ` David Masover 2006-07-28 2:26 ` Hans Reiser 2006-07-28 11:15 ` metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) Jeff Garzik 2006-07-28 14:02 ` Horst H. von Brand 2006-07-28 15:48 ` David Masover 2006-07-28 16:33 ` Linus Torvalds 2006-07-28 11:57 ` Hans Reiser 2006-07-28 19:44 ` David Masover 2006-07-28 13:34 ` Hans Reiser 2006-07-28 20:58 ` Łukasz Mierzwa 2006-07-28 21:32 ` Mike Benoit 2006-07-29 3:21 ` Hans Reiser 2006-07-28 21:53 ` David Masover 2006-07-29 1:36 ` Hans Reiser 2006-07-29 9:12 ` Arjan van de Ven 2006-07-29 18:11 ` David Masover 2006-07-29 20:02 ` Sarath Menon 2006-07-29 20:30 ` David Masover 2006-07-30 11:55 ` Christian Trefzer 2006-07-30 21:10 ` Wil Reichert 2006-07-30 21:39 ` Christian Trefzer 2006-07-30 21:42 ` Christian Trefzer 2006-07-30 22:38 ` David Masover 2006-07-31 15:21 ` Wil Reichert 2006-07-31 15:57 ` David Masover 2006-07-31 16:05 ` Łukasz Mierzwa 2006-07-31 22:05 ` Sarath Menon 2006-08-01 12:27 ` Christian Trefzer 2006-08-01 8:50 ` Christian Trefzer 2006-08-01 16:59 ` David Masover 2006-07-30 0:45 ` Maciej Sołtysiak 2006-07-30 1:08 ` Hans Reiser 2006-07-30 10:30 ` Łukasz Mierzwa 2006-07-30 10:30 ` Łukasz Mierzwa 2006-08-07 22:00 ` Pavel Machek 2006-07-29 12:28 ` Nikita Danilov 2006-07-29 18:31 ` David Masover 2006-07-29 18:57 ` Nikita Danilov 2006-07-29 23:01 ` Hans Reiser 2006-07-30 10:13 ` Łukasz Mierzwa 2006-07-30 10:13 ` Łukasz Mierzwa 2006-07-30 15:33 ` David Masover 2006-07-30 15:33 ` David Masover 2006-07-29 22:58 ` Hans Reiser 2006-07-31 11:58 ` Nikita Danilov 2006-07-29 18:25 ` David Masover 2006-07-31 13:28 ` Horst H. von Brand 2006-07-28 23:08 ` Jeff Garzik 2006-07-29 3:20 ` Hans Reiser 2006-08-07 21:56 ` ext3 vs reiserfs speed (was Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)) Pavel Machek 2006-07-29 10:30 ` metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) Łukasz Mierzwa 2006-07-29 10:31 ` Łukasz Mierzwa 2006-07-30 18:41 ` Horst H. von Brand 2006-07-30 21:15 ` Toby Thain 2006-07-28 20:14 ` Hua Zhong 2006-07-28 13:40 ` Hans Reiser 2006-08-01 15:32 ` Łukasz Mierzwa 2006-08-01 15:32 ` Łukasz Mierzwa 2006-08-01 16:43 ` Vladimir V. Saveliev 2006-08-01 16:43 ` Vladimir V. Saveliev 2006-08-02 18:45 ` Horst H. von Brand 2006-08-02 17:07 ` Łukasz Mierzwa 2006-08-02 17:07 ` Łukasz Mierzwa 2006-08-05 1:01 ` David Masover 2006-07-23 11:48 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Jan-Benedict Glaw 2006-07-23 17:43 ` Toby Thain 2006-07-23 17:43 ` Toby Thain 2006-07-23 20:46 ` Jeff Mahoney 2006-07-23 21:15 ` Hans Reiser 2006-07-23 23:22 ` Jeff Mahoney 2006-07-23 22:35 ` Hans Reiser 2006-07-25 19:13 ` Russell Cattelan 2006-07-25 23:51 ` David Masover 2006-07-26 14:26 ` Hans Reiser 2006-07-26 18:16 ` Russell Cattelan 2006-07-27 1:19 ` David Masover 2006-07-27 20:11 ` Maciej Sołtysiak 2006-07-27 20:23 ` David Masover 2006-07-26 0:44 ` David Masover 2006-07-26 7:59 ` the ' 'official' point of view' " Luigi Genoni 2006-07-26 14:33 ` the " 'official' point of view" " Hans Reiser 2006-07-26 10:38 ` David Weinehall 2006-07-24 8:41 ` Matthias Andree 2006-07-24 8:09 ` Hans Reiser 2006-07-24 10:32 ` Matthias Andree 2006-07-24 13:49 ` Horst H. von Brand 2006-07-23 7:20 ` Rene Rebe 2006-07-24 7:49 ` Nikita Danilov 2006-07-25 12:35 ` Andrea Arcangeli 2006-07-25 12:51 ` Rene Rebe 2006-07-25 17:47 ` Hans Reiser 2006-07-25 19:20 ` Francesco Biscani 2006-07-25 21:17 ` Horst H. von Brand 2006-07-25 22:48 ` Jim Crilly 2006-07-25 23:48 ` andrea 2006-07-25 23:45 ` Luigi Genoni 2006-07-26 12:45 ` Adrian Bunk 2006-07-26 13:29 ` andrea 2006-07-26 13:43 ` Adrian Bunk 2006-07-26 14:28 ` andrea 2006-07-26 14:50 ` Adrian Bunk 2006-07-26 16:06 ` andrea 2006-07-26 16:53 ` Adrian Bunk 2006-07-26 17:02 ` J. Bruce Fields 2006-07-26 17:20 ` andrea 2006-07-26 19:01 ` Jerome Pinot 2006-07-26 20:50 ` andrea 2006-07-26 20:50 ` Adrian Bunk 2006-07-26 21:17 ` andrea 2006-07-26 21:37 ` J. Bruce Fields 2006-07-26 22:17 ` andrea 2006-07-27 6:56 ` Adrian Bunk 2006-07-27 8:33 ` the ' 'official' point of view' " Luigi Genoni 2006-07-27 10:04 ` Adrian Bunk 2006-07-27 11:07 ` Luigi Genoni 2006-07-27 11:35 ` Adrian Bunk 2006-07-27 11:43 ` Luigi Genoni 2006-07-27 11:56 ` Adrian Bunk 2006-07-27 13:30 ` Horst H. von Brand 2006-07-27 13:42 ` gmu 2k6 2006-07-27 13:50 ` andrea 2006-07-27 14:31 ` Luigi Genoni 2006-07-27 18:37 ` Jim Crilly 2006-07-27 19:34 ` andrea 2006-07-27 18:43 ` Horst H. von Brand 2006-07-27 20:11 ` andrea 2006-07-27 12:21 ` CaT 2006-07-28 2:25 ` Hans Reiser 2006-07-28 10:01 ` Adrian Bunk 2006-07-28 14:05 ` Horst H. von Brand 2006-07-27 11:52 ` the " 'official' point of view" " andrea 2006-07-27 12:18 ` Adrian Bunk 2006-07-27 13:10 ` andrea 2006-07-27 13:58 ` Adrian Bunk 2006-07-27 14:45 ` andrea 2006-07-27 15:05 ` Adrian Bunk 2006-07-27 16:11 ` andrea 2006-07-28 2:25 ` Hans Reiser 2006-07-28 14:31 ` andrea 2006-07-28 1:47 ` Hans Reiser 2006-07-27 4:35 ` Hans Reiser 2006-07-27 13:26 ` Horst H. von Brand 2006-07-24 2:08 ` Steve Lord 2006-07-24 7:53 ` Nikita Danilov 2006-07-24 10:30 ` Theodore Tso 2006-07-24 11:35 ` Olivier Galibert 2006-07-24 13:39 ` Theodore Tso 2006-07-24 15:38 ` Olivier Galibert 2006-07-24 16:17 ` Theodore Tso 2006-07-24 17:50 ` Olivier Galibert 2006-07-24 18:29 ` Jeff Garzik 2006-07-26 13:08 ` Pavel Machek 2006-07-27 15:52 ` Olivier Galibert 2006-07-27 21:42 ` suspend2 merge history [was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion] Pavel Machek 2006-07-27 23:22 ` Olivier Galibert 2006-07-28 14:01 ` Pavel Machek 2006-07-29 19:23 ` Bill Davidsen 2006-07-29 21:19 ` Rafael J. Wysocki 2006-07-29 21:56 ` Hua Zhong 2006-07-29 22:54 ` Rafael J. Wysocki 2006-07-30 22:25 ` Hua Zhong 2006-07-30 23:07 ` Pavel Machek 2006-07-31 14:28 ` Bill Davidsen 2006-07-31 16:23 ` Pavel Machek 2006-07-31 18:55 ` Hua Zhong 2006-07-31 23:08 ` Rafael J. Wysocki 2006-08-01 1:09 ` Pavel Machek 2006-07-29 23:19 ` Bill Davidsen 2006-07-29 23:28 ` Pavel Machek 2006-07-29 23:47 ` Pavel Machek 2006-07-30 12:57 ` Bill Davidsen 2006-07-30 14:34 ` Rafael J. Wysocki 2006-07-29 23:31 ` Pavel Machek 2006-07-27 16:43 ` the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion Alan Cox 2006-07-25 21:44 ` Valdis.Kletnieks 2006-07-24 22:17 ` Paul Jackson 2006-07-25 8:05 ` Hans Reiser [not found] <matthias.andree@gmx.de> 2002-12-04 11:34 ` #! incompatible -- binfmt_script.c broken? Matthias Andree 2002-12-04 14:26 ` Alex Riesen 2002-12-04 15:23 ` Matthew Garrett 2002-12-04 18:37 ` Matthias Andree 2002-12-04 20:37 ` Matthew Garrett 2002-12-04 22:28 ` H. Peter Anvin 2002-12-05 11:55 ` Matthias Andree 2002-12-05 0:42 ` Horst von Brand 2002-12-05 11:38 ` Matthias Andree 2002-12-04 22:25 ` Andries Brouwer 2002-12-05 0:10 ` Horst von Brand
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.