* REISER4: fix for reiser4_write_extent @ 2007-04-05 22:42 Ignatich 2007-04-06 0:05 ` johnrobertbanks 2007-04-07 12:51 ` Laurent Riffard 0 siblings, 2 replies; 64+ messages in thread From: Ignatich @ 2007-04-05 22:42 UTC (permalink / raw) To: reiserfs-list, linux-kernel [-- Attachment #1: Type: text/plain, Size: 342 bytes --] While trying to find the cause of problems with reiser4 in recent kernels I came across this. Incomplete write handling seem to be missing from reiser4_write_extent() thanks to reiser4-temp-fix.patch. Strangely, there is a patch by Edward Shishkin that should address that issue, but it is missing from -mm tree. Please check. Max [-- Attachment #2: reiser4-fix-write_extent.patch --] [-- Type: text/plain, Size: 6591 bytes --] ------------------------------------------------------ Subject: reiser4: fix write_extent From: Edward Shishkin <[EMAIL PROTECTED]> . Fix reiser4_write_extent(): 1) handling incomplete writes missed in reiser4-temp-fix.patch 2) bugs in the case of returned errors Signed-off-by: Andrew Morton <[EMAIL PROTECTED]> --- fs/reiser4/plugin/item/extent_file_ops.c | 64 ++++++++++++--------- 1 file changed, 37 insertions(+), 27 deletions(-) diff -puN fs/reiser4/plugin/item/extent_file_ops.c~reiser4-fix-write_extent fs/reiser4/plugin/item/extent_file_ops.c --- a/fs/reiser4/plugin/item/extent_file_ops.c~reiser4-fix-write_extent +++ a/fs/reiser4/plugin/item/extent_file_ops.c @@ -941,15 +941,15 @@ static int write_extent_reserve_space(st * reiser4_write_extent - write method of extent item plugin * @file: file to write to * @buf: address of user-space buffer - * @write_amount: number of bytes to write - * @off: position in file to write to + * @count: number of bytes to write + * @pos: position in file to write to * */ ssize_t reiser4_write_extent(struct file *file, const char __user *buf, size_t count, loff_t *pos) { int have_to_update_extent; - int nr_pages; + int nr_pages, nr_dirty; struct page *page; jnode *jnodes[WRITE_GRANULARITY + 1]; struct inode *inode; @@ -958,7 +958,7 @@ ssize_t reiser4_write_extent(struct file int i; int to_page, page_off; size_t left, written; - int result; + int result = 0; inode = file->f_dentry->d_inode; if (write_extent_reserve_space(inode)) @@ -972,10 +972,12 @@ ssize_t reiser4_write_extent(struct file BUG_ON(get_current_context()->trans->atom != NULL); + left = count; index = *pos >> PAGE_CACHE_SHIFT; /* calculate number of pages which are to be written */ end = ((*pos + count - 1) >> PAGE_CACHE_SHIFT); nr_pages = end - index + 1; + nr_dirty = 0; assert("", nr_pages <= WRITE_GRANULARITY + 1); /* get pages and jnodes */ @@ -983,22 +985,17 @@ ssize_t reiser4_write_extent(struct file page = find_or_create_page(inode->i_mapping, index + i, reiser4_ctx_gfp_mask_get()); if (page == NULL) { - while(i --) { - unlock_page(jnode_page(jnodes[i])); - page_cache_release(jnode_page(jnodes[i])); - } - return RETERR(-ENOMEM); + nr_pages = i; + result = RETERR(-ENOMEM); + goto out; } - jnodes[i] = jnode_of_page(page); if (IS_ERR(jnodes[i])) { unlock_page(page); page_cache_release(page); - while (i --) { - jput(jnodes[i]); - page_cache_release(jnode_page(jnodes[i])); - } - return RETERR(-ENOMEM); + nr_pages = i; + result = RETERR(-ENOMEM); + goto out; } /* prevent jnode and page from disconnecting */ JF_SET(jnodes[i], JNODE_WRITE_PREPARED); @@ -1009,7 +1006,6 @@ ssize_t reiser4_write_extent(struct file have_to_update_extent = 0; - left = count; page_off = (*pos & (PAGE_CACHE_SIZE - 1)); for (i = 0; i < nr_pages; i ++) { to_page = PAGE_CACHE_SIZE - page_off; @@ -1050,14 +1046,26 @@ ssize_t reiser4_write_extent(struct file flush_dcache_page(page); kunmap_atomic(kaddr, KM_USER0); } - - written = filemap_copy_from_user(page, page_off, buf, to_page); + written = filemap_copy_from_user_atomic(page, page_off, buf, + to_page); + if (written != to_page) + /* Do it the slow way */ + written = filemap_copy_from_user_nonatomic(page, + page_off, + buf, + to_page); + if (unlikely(written != to_page)) { + unlock_page(page); + result = RETERR(-EFAULT); + break; + } flush_dcache_page(page); reiser4_set_page_dirty_internal(page); unlock_page(page); + nr_dirty ++; + mark_page_accessed(page); SetPageUptodate(page); - page_cache_release(page); if (jnodes[i]->blocknr == 0) have_to_update_extent ++; @@ -1067,27 +1075,29 @@ ssize_t reiser4_write_extent(struct file left -= to_page; BUG_ON(get_current_context()->trans->atom != NULL); } - if (have_to_update_extent) { - update_extents(file, jnodes, nr_pages, *pos); + update_extents(file, jnodes, nr_dirty, *pos); } else { - for (i = 0; i < nr_pages; i ++) { + for (i = 0; i < nr_dirty; i ++) { + int ret; spin_lock_jnode(jnodes[i]); - result = reiser4_try_capture(jnodes[i], + ret = reiser4_try_capture(jnodes[i], ZNODE_WRITE_LOCK, 0); - BUG_ON(result != 0); + BUG_ON(ret != 0); jnode_make_dirty_locked(jnodes[i]); spin_unlock_jnode(jnodes[i]); } } - + out: for (i = 0; i < nr_pages; i ++) { + page_cache_release(jnode_page(jnodes[i])); JF_CLR(jnodes[i], JNODE_WRITE_PREPARED); jput(jnodes[i]); } + /* the only errors handled so far is ENOMEM and + EFAULT on copy_from_user */ - /* the only error handled so far is EFAULT on copy_from_user */ - return (count - left) ? (count - left) : -EFAULT; + return (count - left) ? (count - left) : result; } static inline void zero_page(struct page *page) ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER? I need help. 2007-04-05 22:42 REISER4: fix for reiser4_write_extent Ignatich @ 2007-04-06 0:05 ` johnrobertbanks 2007-04-07 12:51 ` Laurent Riffard 1 sibling, 0 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-06 0:05 UTC (permalink / raw) To: Ignatich, reiserfs-list, linux-kernel Hi Ignatich, After seeing the following benchmarks at http://linuxhelp.150m.com/resources/fs-benchmarks.htm and http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm The Reiser4 benchmarks are so good, I have decided to try the Reiser4 filesystem. .-------------------------. | FILESYSTEM | TIME |DISK | | TYPE |(secs)|USAGE| .-------------------------. |REISER4 lzo | 1938 | 278 | |REISER4 gzip| 2295 | 213 | |REISER4 | 3462 | 692 | |EXT2 | 4092 | 816 | |JFS | 4225 | 806 | |EXT4 | 4408 | 816 | |EXT3 | 4421 | 816 | |XFS | 4625 | 779 | |REISER3 | 6178 | 793 | |FAT32 |12342 | 988 | |NTFS-3g |10414 | 772 | .-------------------------. Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0) Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources). Anyway, I have patched the 2.6.20 kernel and have a partition formatted with Reiser4. However, I am having trouble getting LILO or GRUB working (with Reiser4). Could you guys who know all about this, help me, or point me to some help. Thanks a lot, John. On Fri, 06 Apr 2007 02:42:35 +0400, "Ignatich" <ignatich@gmail.com> said: > While trying to find the cause of problems with reiser4 in recent > kernels I came across this. > > Incomplete write handling seem to be missing from reiser4_write_extent() > thanks to reiser4-temp-fix.patch. Strangely, there is a patch by Edward > Shishkin that should address that issue, but it is missing from -mm > tree. Please check. > > Max > -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - And now for something completely different ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER? I need help. @ 2007-04-06 0:05 ` johnrobertbanks 0 siblings, 0 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-06 0:05 UTC (permalink / raw) To: Ignatich, reiserfs-list, linux-kernel Hi Ignatich, After seeing the following benchmarks at http://linuxhelp.150m.com/resources/fs-benchmarks.htm and http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm The Reiser4 benchmarks are so good, I have decided to try the Reiser4 filesystem. .-------------------------. | FILESYSTEM | TIME |DISK | | TYPE |(secs)|USAGE| .-------------------------. |REISER4 lzo | 1938 | 278 | |REISER4 gzip| 2295 | 213 | |REISER4 | 3462 | 692 | |EXT2 | 4092 | 816 | |JFS | 4225 | 806 | |EXT4 | 4408 | 816 | |EXT3 | 4421 | 816 | |XFS | 4625 | 779 | |REISER3 | 6178 | 793 | |FAT32 |12342 | 988 | |NTFS-3g |10414 | 772 | .-------------------------. Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0) Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources). Anyway, I have patched the 2.6.20 kernel and have a partition formatted with Reiser4. However, I am having trouble getting LILO or GRUB working (with Reiser4). Could you guys who know all about this, help me, or point me to some help. Thanks a lot, John. On Fri, 06 Apr 2007 02:42:35 +0400, "Ignatich" <ignatich@gmail.com> said: > While trying to find the cause of problems with reiser4 in recent > kernels I came across this. > > Incomplete write handling seem to be missing from reiser4_write_extent() > thanks to reiser4-temp-fix.patch. Strangely, there is a patch by Edward > Shishkin that should address that issue, but it is missing from -mm > tree. Please check. > > Max > -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - And now for something completely different… ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER? I need help. 2007-04-06 0:05 ` johnrobertbanks (?) @ 2007-04-06 0:23 ` H. Peter Anvin 2007-04-06 0:34 ` johnrobertbanks -1 siblings, 1 reply; 64+ messages in thread From: H. Peter Anvin @ 2007-04-06 0:23 UTC (permalink / raw) To: johnrobertbanks; +Cc: Ignatich, reiserfs-list, linux-kernel johnrobertbanks@fastmail.fm wrote: > > Anyway, I have patched the 2.6.20 kernel and have a partition formatted > with Reiser4. > > However, I am having trouble getting LILO or GRUB working (with > Reiser4). > > Could you guys who know all about this, help me, or point me to some > help. > Make your /boot a separate partition and format it as conservatively as possible (e.g. ext3, or even ext2.) Problem solved. -hpa ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER? I need help. 2007-04-06 0:23 ` H. Peter Anvin @ 2007-04-06 0:34 ` johnrobertbanks 2007-04-06 0:39 ` H. Peter Anvin 0 siblings, 1 reply; 64+ messages in thread From: johnrobertbanks @ 2007-04-06 0:34 UTC (permalink / raw) To: H. Peter Anvin; +Cc: Ignatich, reiserfs-list, linux-kernel Yeap, I guess that will probably work. And here I was trying to compile old versions of GRUB from namesys.com. By the way, do you think the benchmarks from: http://linuxhelp.150m.com/resources/fs-benchmarks.htm and http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm are accurate? .-------------------------. | FILESYSTEM | TIME |DISK | | TYPE |(secs)|USAGE| .-------------------------. |REISER4 lzo | 1938 | 278 | |REISER4 gzip| 2295 | 213 | |REISER4 | 3462 | 692 | |EXT2 | 4092 | 816 | |JFS | 4225 | 806 | |EXT4 | 4408 | 816 | |EXT3 | 4421 | 816 | |XFS | 4625 | 779 | |REISER3 | 6178 | 793 | |FAT32 |12342 | 988 | |NTFS-3g |10414 | 772 | .-------------------------. Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0) Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources). Thanks for that, John. On Thu, 05 Apr 2007 17:23:23 -0700, "H. Peter Anvin" <hpa@zytor.com> said: > johnrobertbanks@fastmail.fm wrote: > > > > Anyway, I have patched the 2.6.20 kernel and have a partition formatted > > with Reiser4. > > > > However, I am having trouble getting LILO or GRUB working (with > > Reiser4). > > > > Could you guys who know all about this, help me, or point me to some > > help. > > > > Make your /boot a separate partition and format it as conservatively as > possible (e.g. ext3, or even ext2.) > > Problem solved. > > -hpa -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - Send your email first class ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER? I need help. 2007-04-06 0:34 ` johnrobertbanks @ 2007-04-06 0:39 ` H. Peter Anvin 2007-04-06 1:34 ` Reiser4. BEST FILESYSTEM EVER johnrobertbanks 0 siblings, 1 reply; 64+ messages in thread From: H. Peter Anvin @ 2007-04-06 0:39 UTC (permalink / raw) To: johnrobertbanks; +Cc: Ignatich, reiserfs-list, linux-kernel johnrobertbanks@fastmail.fm wrote: > Yeap, I guess that will probably work. > > And here I was trying to compile old versions of GRUB from namesys.com. > > By the way, do you think the benchmarks from: > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > are accurate? > Accurate, probably. Whether or not they're *relevant* is a totally different ball of wax. -hpa ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-06 0:39 ` H. Peter Anvin @ 2007-04-06 1:34 ` johnrobertbanks 2007-04-06 3:12 ` Valdis.Kletnieks 2007-04-06 4:07 ` H. Peter Anvin 0 siblings, 2 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-06 1:34 UTC (permalink / raw) To: H. Peter Anvin; +Cc: Ignatich, reiserfs-list, linux-kernel Hi Peter, You say that the results may be accurate, but not relevant. .-------------------------. | FILESYSTEM | TIME |DISK | | TYPE |(secs)|USAGE| .-------------------------. |REISER4 lzo | 1938 | 278 | |REISER4 gzip| 2295 | 213 | |REISER4 | 3462 | 692 | |EXT2 | 4092 | 816 | |JFS | 4225 | 806 | |EXT4 | 4408 | 816 | |EXT3 | 4421 | 816 | |XFS | 4625 | 779 | |REISER3 | 6178 | 793 | |FAT32 |12342 | 988 | |NTFS-3g |10414 | 772 | .-------------------------. If they are accurate,.... THEN they are obviously very relevant. Trying to follow http://linuxhelp.150m.com/resources/fs-benchmarks.htm I have set up a Reiser4 partition with gzip compression, here is the difference in disk usage of a typical Debian installation on two 10GB partitions, one with Reiser3 and the other with Reiser4. debian:/# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda3 10490104 6379164 4110940 61% /3 /dev/sda7 9967960 2632488 7335472 27% /7 Partitions 3 and 7 have exactly the same data on them (the typical Debian install). The partitions are exactly the same size (although df records different sizes). Partition 3 is Reiser3 -- uses 6.4 GB. Partition 7 is Reiser4 -- uses 2.6 GB. So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same info). This seems very relevant to me. John. On Thu, 05 Apr 2007 17:39:58 -0700, "H. Peter Anvin" <hpa@zytor.com> said: > johnrobertbanks@fastmail.fm wrote: > > Yeap, I guess that will probably work. > > > > And here I was trying to compile old versions of GRUB from namesys.com. > > > > By the way, do you think the benchmarks from: > > > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > > > are accurate? > > > > Accurate, probably. Whether or not they're *relevant* is a totally > different ball of wax. > > -hpa -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - mmm... Fastmail... ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-06 1:34 ` Reiser4. BEST FILESYSTEM EVER johnrobertbanks @ 2007-04-06 3:12 ` Valdis.Kletnieks 2007-04-06 4:07 ` H. Peter Anvin 1 sibling, 0 replies; 64+ messages in thread From: Valdis.Kletnieks @ 2007-04-06 3:12 UTC (permalink / raw) To: johnrobertbanks; +Cc: H. Peter Anvin, Ignatich, reiserfs-list, linux-kernel [-- Attachment #1: Type: text/plain, Size: 1550 bytes --] On Thu, 05 Apr 2007 18:34:48 PDT, johnrobertbanks@fastmail.fm said: > If they are accurate,.... THEN they are obviously very relevant. Erm. No. They're not "obviously" very relevant. I could hypothetically create a benchmark, that's accurate and repeatable, that shows that reiser4 is able to wash a herd of elephants exactly 11.458% faster than ext3. And you would, of course, say "But elephants have nothing to do with file systems", Because they aren't relevant to file systems. Similarly, we've seen benchmarks that show some patch improves NUMA performance by 5% - and those aren't relevant on my laptop because my laptop doesn't do NUMA. And a benchmark of file system performance is only as relevant as it reflects *your* application's use of the filesystem - how fast it can create and remove tiny files isn't relevant if your use of the filesystem is to store large files with long sequential read/write patterns. And the level of compression isn't very relevant if you're using the partition to store already-compressed audio or video. I know somebody who defines a "relevance index" for things, and the measure is "how many cubicles do I have to go to find somebody who actually cares about ABC?" - and for him, that's itself a relevant index, because if it's 0, *he* cares, and if it's 1, his immediate neighbors care and will cause him grief if ABC is a problem. People who are 5 or 6 cubicles away are less likely to give him a hard time, and the people who are 15 to 20 cubicles away are in an entirely separate building. :) [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-06 1:34 ` Reiser4. BEST FILESYSTEM EVER johnrobertbanks 2007-04-06 3:12 ` Valdis.Kletnieks @ 2007-04-06 4:07 ` H. Peter Anvin 2007-04-06 4:32 ` johnrobertbanks 1 sibling, 1 reply; 64+ messages in thread From: H. Peter Anvin @ 2007-04-06 4:07 UTC (permalink / raw) To: johnrobertbanks; +Cc: Ignatich, reiserfs-list, linux-kernel johnrobertbanks@fastmail.fm wrote: > Hi Peter, > > You say that the results may be accurate, but not relevant. > NO, I said that whether they're accurate is another matter. > If they are accurate,.... THEN they are obviously very relevant. Crap-o-la. Whether or not they're relevant depends on how well they happen to reflect your particular usage pattern. There are NO benchmarks which are relevant to all users. Understanding whether or not a benchmark is relevant to one's particular application is one of the trickiest things about benchmarks. -hpa ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-06 4:07 ` H. Peter Anvin @ 2007-04-06 4:32 ` johnrobertbanks [not found] ` <20070406152119.GC4228@delft.aura.cs.cmu.edu> 2007-04-07 19:17 ` Lennart Sorensen 0 siblings, 2 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-06 4:32 UTC (permalink / raw) To: H. Peter Anvin; +Cc: Ignatich, reiserfs-list, linux-kernel, linux-fsdevel Hi Peter, You say that the results may be accurate, but "Whether or not they're *relevant* is a totally different ball of wax." and "Whether or not they're relevant depends on how well they happen to reflect your particular usage pattern." Well, surprise, surprise,.. everyone knows that. Have a look at the (summary) of the results: .-------------------------. | FILESYSTEM | TIME |DISK | | TYPE |(secs)|USAGE| .-------------------------. |REISER4 lzo | 1938 | 278 | |REISER4 gzip| 2295 | 213 | |REISER4 | 3462 | 692 | |EXT2 | 4092 | 816 | |JFS | 4225 | 806 | |EXT4 | 4408 | 816 | |EXT3 | 4421 | 816 | |XFS | 4625 | 779 | |REISER3 | 6178 | 793 | |FAT32 |12342 | 988 | |NTFS-3g |10414 | 772 | .-------------------------. for the full results see: http://linuxhelp.150m.com/resources/fs-benchmarks.htm Don't you agree, that "If they are accurate,.... THEN they are obviously very relevant." I have set up a Reiser4 partition with gzip compression, here is the difference in disk usage of a typical Debian installation on two 10GB partitions, one with Reiser3 and the other with Reiser4. debian:/# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda3 10490104 6379164 4110940 61% /3 /dev/sda7 9967960 2632488 7335472 27% /7 Partitions 3 and 7 have exactly the same data on them (the typical Debian install). The partitions are exactly the same size (although df records different sizes). Partition 3 is Reiser3 -- uses 6.4 GB. Partition 7 is Reiser4 -- uses 2.6 GB. So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same info). Don't you think this result is significant in itself? Following your hint I have booted /dev/sda7 and all the programs seem to work fine. They do not seem to be any faster than when using Reiser3. The whole system seems about as responsive as always. For fun, I ran bonnie++. Here are the results: debian:/# ./bonnie++ -u root Using uid:0, gid:0. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP debian 1G 121 99 86524 21 63297 41 920 99 187762 80 1782 233 Latency 82484us 386ms 438ms 26758us 110ms 398ms Version 1.93c ------Sequential Create------ --------Random Create-------- debian -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 +++++ +++ +++++ +++ 18509 92 17776 86 +++++ +++ 19495 91 Latency 210us 5475us 5525us 5777us 5522us 5839us I particularly liked the 233%CP for Random-Seeks. John. On Thu, 05 Apr 2007 21:07:28 -0700, "H. Peter Anvin" <hpa@zytor.com> said: > johnrobertbanks@fastmail.fm wrote: > > Hi Peter, > > > > You say that the results may be accurate, but not relevant. > > > > NO, I said that whether they're accurate is another matter. > > > If they are accurate,.... THEN they are obviously very relevant. > > Crap-o-la. Whether or not they're relevant depends on how well they > happen to reflect your particular usage pattern. > > There are NO benchmarks which are relevant to all users. Understanding > whether or not a benchmark is relevant to one's particular application > is one of the trickiest things about benchmarks. > > -hpa -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - Email service worth paying for. Try it for free ^ permalink raw reply [flat|nested] 64+ messages in thread
[parent not found: <20070406152119.GC4228@delft.aura.cs.cmu.edu>]
* Re: Reiser4. BEST FILESYSTEM EVER. [not found] ` <20070406152119.GC4228@delft.aura.cs.cmu.edu> @ 2007-04-07 2:47 ` johnrobertbanks 2007-04-07 3:30 ` Jan Harkes 2007-04-07 17:39 ` Valdis.Kletnieks 0 siblings, 2 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-07 2:47 UTC (permalink / raw) To: Jan Harkes; +Cc: linux-kernel, linux-fsdevel On Fri, 6 Apr 2007 11:21:19 -0400, "Jan Harkes" <jaharkes@cs.cmu.edu> said: > Do you really have to repeat the results in every email you sent? The following benchmarks are from http://linuxhelp.150m.com/resources/fs-benchmarks.htm or, http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm .-------------------------. | FILESYSTEM | TIME |DISK | | TYPE |(secs)|USAGE| .-------------------------. |REISER4 lzo | 1938 | 278 | |REISER4 gzip| 2295 | 213 | |REISER4 | 3462 | 692 | |EXT2 | 4092 | 816 | |JFS | 4225 | 806 | |EXT4 | 4408 | 816 | |EXT3 | 4421 | 816 | |XFS | 4625 | 779 | |REISER3 | 6178 | 793 | |FAT32 |12342 | 988 | |NTFS-3g |10414 | 772 | .-------------------------. Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0) Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources). > Do you really have to repeat the results in every email you sent? Damn, I did it again. WHY DO YOU CARE? Look, its simple, I am (among other things) discussing these results, so people need to see them. > > Don't you agree, that "If they are accurate,.... THEN they are obviously > > very relevant." > > Not everyone does. I care mostly about reliability and availability > neither of which are shown by your results. Actually, to some extent, bonnie++ tests the reliability of the filesystem, eg, NTFS-3g usually fails. By the way, I have pulled the plug on my REISER4 system, a number of times now, and it recovers without problem. > With compression there is a pretty high probability that one corrupted > byte or disk block will result in loss of a considerably larger amount > of data. Bad blocks are NOT dealt with by the filesystem,... so your comment is irrelevant, or just plain wrong. If your filesystem is writing to bad blocks, then throw away your operating system. > > I have set up a Reiser4 partition with gzip compression, here is the > > difference in disk usage of a typical Debian installation on two 10GB > > partitions, one with Reiser3 and the other with Reiser4. > > > > debian:/# df > > Filesystem 1K-blocks Used Available Use% Mounted on > > /dev/sda3 10490104 6379164 4110940 61% /3 > > /dev/sda7 9967960 2632488 7335472 27% /7 > ... > > Partition 3 is Reiser3 -- uses 6.4 GB. > > Partition 7 is Reiser4 -- uses 2.6 GB. > > > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 > > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same > > info). > > Wow, consider me totally and completely, unimpressed. > Wow, consider me totally impressed by your AMAZING BIAS. Would you like to tell me why you are SO BIASED against REISER4. John. -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - A fast, anti-spam email service. ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 2:47 ` johnrobertbanks @ 2007-04-07 3:30 ` Jan Harkes 2007-04-07 5:58 ` johnrobertbanks 2007-04-07 17:39 ` Valdis.Kletnieks 1 sibling, 1 reply; 64+ messages in thread From: Jan Harkes @ 2007-04-07 3:30 UTC (permalink / raw) To: johnrobertbanks; +Cc: linux-kernel, linux-fsdevel Since you decide to publically respond to a private email, but not only you did not 'discuss' anything I wrote and in fact cut out most of the useful information in my reply I guess I will have to repeat my observations. Once I send out this email, I'll just add you to my friendly killfile (as well as this thread's subject/msgids) so don't bother continuing you one-sided 'discussions' on this topic. On Fri, Apr 06, 2007 at 07:47:36PM -0700, johnrobertbanks@fastmail.fm wrote: > > Do you really have to repeat the results in every email you sent? > > Damn, I did it again. WHY DO YOU CARE? Because I saw them the first time around. And although the performance difference on those micro benchmarks seems quite impressive, I'm not convinced. > Look, its simple, I am (among other things) discussing these results, so > people need to see them. However, you do not discuss, you just repeat, and repeat, and repeat. But for what reason. Do you want an actual discussion, or do you hate the reiserfs developers so much that you want to antagonize any and all other Linux file systems developers? > > > Don't you agree, that "If they are accurate,.... THEN they are obviously > > > very relevant." > > > > Not everyone does. I care mostly about reliability and availability > > neither of which are shown by your results. > > Actually, to some extent, bonnie++ tests the reliability of the > filesystem, eg, NTFS-3g usually fails. > > By the way, I have pulled the plug on my REISER4 system, a number of > times now, and it recovers without problem. Very nice for you. I guess you have also not been hit by out of memory conditions or failing partial writes. So I guess we can ignore the patch that was just sent a day or two ago to the mailing list because you succesfully pulled the plug, a number of times at that. > > > I have set up a Reiser4 partition with gzip compression, here is the > > > difference in disk usage of a typical Debian installation on two 10GB > > > partitions, one with Reiser3 and the other with Reiser4. > > > > > > debian:/# df > > > Filesystem 1K-blocks Used Available Use% Mounted on > > > /dev/sda3 10490104 6379164 4110940 61% /3 > > > /dev/sda7 9967960 2632488 7335472 27% /7 > > ... > > > Partition 3 is Reiser3 -- uses 6.4 GB. > > > Partition 7 is Reiser4 -- uses 2.6 GB. > > > > > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 > > > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same > > > info). > > > > Wow, consider me totally and completely, unimpressed. > > Here is the part of my email that you seemed to totally ignore, You've just saved yourself $3.80, now go get yourself a latte. (see. http://tomayko.com/weblog/2006/09/11/that-dilbert-cartoon) Seriously, disk storage is getting less expensive all the time, you can already buy a 250GB SATA drive for $70. Also, compression doesn't help once you store already compressed data such as jpeg images, mp3 files, or mpeg2/4/divx video files. Not only are the savings non-existant, but we still end up with the disadvantage that corruption propagates to multiple files because of the compression in the file system. And if it doesn't propagate across multiple files, the compression can't be all that good since it can't benefit as much from similarity between files. So if that is the case and you really want to save diskspace you almost have to look at read-only compressed filesystems such as cramfs, squashfs, zisofs, cloop and various other variants in combination with a unionfs overlay to get read/write functionality. But in the end everything is a tradeoff. You can save diskspace, but increase the cost of corruption. Use a better compression algorithm, but that uses more CPU or which is visible in performance of the application. This can be offset by caching more, and being lazier with writebacks, but that hurts on-disk consistency, creates more memory pressure (more swapout/paging) and generally slows down other applications that aren't actually accessing the disk. Having a fast multi-core cpu and lots of memory helps a lot, but at some point what tradeoff did we just make, we saved a couple of dollars not having to buy a larger disk, but we spend considerably more on the more expensive cpu and memory. > Wow, consider me totally impressed by your AMAZING BIAS. > > Would you like to tell me why you are SO BIASED against REISER4. And that is the reponse I get, I thought you wanted discussion, but clearly you don't care about any meaningful discussion. Your goal seems to be to make sure that other developers end up ignoring any thread that has reiser in the subject. And even if they are not biased and welcome a discussion, you will just call them out on it, because clearly if they want to discuss something they aren't totally with it, so they have to be totally BIASED against REISER4. At least it looks like we agree on something, I think. Jan ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 3:30 ` Jan Harkes @ 2007-04-07 5:58 ` johnrobertbanks 2007-04-07 7:15 ` Willy Tarreau 2007-04-07 14:11 ` Krzysztof Halasa 0 siblings, 2 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-07 5:58 UTC (permalink / raw) To: Jan Harkes; +Cc: linux-kernel, linux-fsdevel On Fri, 6 Apr 2007 23:30:49 -0400, "Jan Harkes" <jaharkes@cs.cmu.edu> said: > > Since you decide to publically respond to a private email, but not only > you did not 'discuss' anything I wrote and in fact cut out most of the > useful information in my reply I guess I will have to repeat my > observations. You are a funny guy Jan. Here you are, once again, cutting out my most useful information, ie, the data I was discussing, while complaining that I cut out your most useful information. You know,... you cut out this bit: ----------------------------------------------------------------- > The following benchmarks are from > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm or, > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > .-------------------------. > | FILESYSTEM | TIME |DISK | > | TYPE |(secs)|USAGE| > .-------------------------. > |REISER4 lzo | 1938 | 278 | > |REISER4 gzip| 2295 | 213 | > |REISER4 | 3462 | 692 | > |EXT2 | 4092 | 816 | > |JFS | 4225 | 806 | > |EXT4 | 4408 | 816 | > |EXT3 | 4421 | 816 | > |XFS | 4625 | 779 | > |REISER3 | 6178 | 793 | > |FAT32 |12342 | 988 | > |NTFS-3g |10414 | 772 | > .-------------------------. > > Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0) > > Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources). ----------------------------------------------------------------- And this bit: Jan,... Here is another section, you conveniently cut out. Maybe you should explain why you cut out this section? Was it embarrassing to you? I mean, your statement is sort of correct,... but it shows a basic misunderstanding of filesystems. > > With compression there is a pretty high probability that one corrupted > > byte or disk block will result in loss of a considerably larger amount > > of data. > > Bad blocks are NOT dealt with by the filesystem,... so your comment is > irrelevant, or just plain wrong. > > If your filesystem is writing to bad blocks, then throw away your > operating system. ----------------------------------------------------------------- It is true that I considered your "most useful information," an irrelevant section, which is why it was cut out (ignored). I did not see my doing this, any worse than you doing it. I did not realize that you were be so impolite. As to your email being private, I had thought I had joined a mailing list. I had not idea your email was meant to be private and just considered it like all the others. Now you mention it, I wondered why the email did not automatically list the mailing lists, as recipients, and I had to add them. If I had realized this I may have added the mailing lists as recipients, anyway. It would be like me to do such. However, you should understand that I am new to mailing lists. > > > > Do you really have to repeat the results in every email you sent? > > > > Damn, I did it again. WHY DO YOU CARE? > > Because I saw them the first time around. And although the performance > difference on those micro benchmarks seems quite impressive, I'm not > convinced. So, likewise, I saw your comments (you know the ones you miss so much) the first time around, as I was not convinced of their worth. The benchmarks measure certain data. Its fine you do not read into them, stuff that isn't there, like reliability, for example. > > Look, its simple, I am (among other things) discussing these results, so > > people need to see them. > > However, you do not discuss, you just repeat, and repeat, and repeat. I never said I wanted discussion, you just said I did. You just repeat, and repeat, and repeat. In reality, I quite appreciate reasonable discussion. But, I doubt I will get much from you. > But for what reason. Do you want an actual discussion, or do you hate > the reiserfs developers so much that you want to antagonize any and all > other Linux file systems developers? Why do you think I hate reiserfs developers? That is an insane claim. Why would I hate reiser3 developers? Why would I hate reiser4 developers? Why would I even dislike them? I think Hans Reiser is a genius. Is that what you mean by hate? Answer this question. Why do YOU think I am antagonizing reiserfs developers? You must have a reason for stating what you have. > > By the way, I have pulled the plug on my REISER4 system, a number of > > times now, and it recovers without problem. > > Very nice for you. I guess you have also not been hit by out of memory > conditions or failing partial writes. So I guess we can ignore the patch > that was just sent a day or two ago to the mailing list because you > succesfully pulled the plug, a number of times at that. Why are you attacking me with sarcasm, when I have just stated a simple fact? > > > > I have set up a Reiser4 partition with gzip compression, here is the > > > > difference in disk usage of a typical Debian installation on two 10GB > > > > partitions, one with Reiser3 and the other with Reiser4. > > > > > > > > debian:/# df > > > > Filesystem 1K-blocks Used Available Use% Mounted on > > > > /dev/sda3 10490104 6379164 4110940 61% /3 > > > > /dev/sda7 9967960 2632488 7335472 27% /7 > > > ... > > > > Partition 3 is Reiser3 -- uses 6.4 GB. > > > > Partition 7 is Reiser4 -- uses 2.6 GB. > > > > > > > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 > > > > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same > > > > info). > > > > > > Wow, consider me totally and completely, unimpressed. > > > > > Here is the part of my email that you seemed to totally ignore, Yes, I ignored it,.... is that a crime? > You've just saved yourself $3.80, now go get yourself a latte. > (see. http://tomayko.com/weblog/2006/09/11/that-dilbert-cartoon) > > Seriously, disk storage is getting less expensive all the time, you can > already buy a 250GB SATA drive for $70. Also, compression doesn't help > once you store already compressed data such as jpeg images, mp3 files, > or mpeg2/4/divx video files. Not only are the savings non-existant, but > we still end up with the disadvantage that corruption propagates to > multiple files because of the compression in the file system. I believe that compression in REISER4 is just a wrapper for gzip or lzo. So each file is compressed and your "corruption propagation" is just in your imagination. > And if it doesn't propagate across multiple files, the compression can't > be all that good It can be as good as gzipping every file, ie, pretty damn good. > since it can't benefit as much from similarity between > files. So if that is the case and you really want to save diskspace you > almost have to look at read-only compressed filesystems such as cramfs, > squashfs, zisofs, cloop and various other variants in combination with > a unionfs overlay to get read/write functionality. > > But in the end everything is a tradeoff. You can save diskspace, but > increase the cost of corruption. You deliberately ignored the fact that bad blocks are NOT dealt with by the filesystem,... but by the operating system. Like I said: If your filesystem is writing to bad blocks, then throw away your operating system. > Use a better compression algorithm, but > that uses more CPU or which is visible in performance of the > application. Not necessarily, just look at the bonnie++ tests. The tests show that using compression (with todays hardware) leads to read speeds that are some FOUR times the physical disk read rate,... but you ignore this as you have some strange anti-REISER religion. Think about it,... read speeds that are some FOUR times the physical disk read rate,... impossible without the use of compression (or something similar). > This can be offset by caching more, and being lazier with > writebacks, but that hurts on-disk consistency, creates more memory > pressure (more swapout/paging) and generally slows down other > applications that aren't actually accessing the disk. Having a fast > multi-core cpu and lots of memory helps a lot, but at some point what > tradeoff did we just make, we saved a couple of dollars not having to > buy a larger disk, but we spend considerably more on the more > expensive cpu and memory. > Wow, consider me STILL totally impressed by your AMAZING BIAS. Would you like to tell me why you are SO BIASED against REISER4? John. -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - Choose from over 50 domains or use your own ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 5:58 ` johnrobertbanks @ 2007-04-07 7:15 ` Willy Tarreau 2007-04-07 13:47 ` johnrobertbanks 2007-04-07 14:11 ` Krzysztof Halasa 1 sibling, 1 reply; 64+ messages in thread From: Willy Tarreau @ 2007-04-07 7:15 UTC (permalink / raw) To: johnrobertbanks; +Cc: Jan Harkes, linux-kernel, linux-fsdevel On Fri, Apr 06, 2007 at 10:58:45PM -0700, johnrobertbanks@fastmail.fm wrote: > You know,... you cut out this bit: > > ----------------------------------------------------------------- > > > The following benchmarks are from > > > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm or, > > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm ... Hey John, please change your disk, it's scratched and you're repeating yourself again and again. At first I thought "Oh cool, some good news about reiser4", now when I see "reiserfs" in a thread, I think "oh no, not this boring guy who escaped from the asylum again !". I hope this thread will be cut shortly so that you stop doing bad publicity to reiserfs and its developers, because when a product is indicated as good by stupid people, it's really doing harm. Also, about this part : [Jan] > > But in the end everything is a tradeoff. You can save diskspace, but > > increase the cost of corruption. I don't 100% agree with Jan, because for some usages (temporary space), light compression can increase speed. For instance, when processing logs, I get better speed by compressing intermediate files with LZO on the fly. [John] > You deliberately ignored the fact that bad blocks are NOT dealt with by > the filesystem,... but by the operating system. Like I said: If your > filesystem is writing to bad blocks, then throw away your operating > system. But what you write here is complete crap. The filesystem relies on a linear block device. The operating system is responsible for doing read retries or reporting errors on bad blocks, but the FS and only the FS can decide how not to use some known defective areas, for instance not putting any metadata on them nor any useful data. Now if you want to stop writing stupid things again and again, take your bag, don't miss the bus to school, and listen to the teachers instead of playing games on your calculator. Willy PS: non need to reply either, I'll kill this thread and your address here. ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 7:15 ` Willy Tarreau @ 2007-04-07 13:47 ` johnrobertbanks 0 siblings, 0 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-07 13:47 UTC (permalink / raw) To: Willy Tarreau; +Cc: Jan Harkes, linux-kernel, linux-fsdevel Hi Willy,... > With decent CPU, you can reach higher read/write data rates than what a > single off-the-shelf disk can achieve. For this reason, I think that > reiser4 would be worth trying for this particular usage. Glad to see you are willing to give Reiser4 a go. Good man. ---------------------------------------------------------------------- On Sat, 7 Apr 2007 09:15:35 +0200, "Willy Tarreau" <w@1wt.eu> said: > On Fri, Apr 06, 2007 at 10:58:45PM -0700, johnrobertbanks@fastmail.fm > wrote: > > You know,... you cut out this bit: > > > > ----------------------------------------------------------------- > > > > > The following benchmarks are from > > > > > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm or, > > > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > ... > > Hey John, please change your disk, it's scratched and you're repeating > yourself again and again. At first I thought "Oh cool, some good news > about reiser4", now when I see "reiserfs" in a thread, I think "oh no, > not this boring guy who escaped from the asylum again !". I hope this > thread will be cut shortly so that you stop doing bad publicity to > reiserfs and its developers, because when a product is indicated as > good by stupid people, it's really doing harm. > > Also, about this part : > [Jan] > > > But in the end everything is a tradeoff. You can save diskspace, but > > > increase the cost of corruption. > > I don't 100% agree with Jan, because for some usages (temporary space), > light compression can increase speed. For instance, when processing logs, > I get better speed by compressing intermediate files with LZO on the fly. > > [John] > > You deliberately ignored the fact that bad blocks are NOT dealt with by > > the filesystem,... but by the operating system. Like I said: If your > > filesystem is writing to bad blocks, then throw away your operating > > system. > > But what you write here is complete crap. The filesystem relies on a > linear block device. The operating system is responsible for doing > read retries or reporting errors on bad blocks, but the FS and only > the FS can decide how not to use some known defective areas, for > instance not putting any metadata on them nor any useful data. > > Now if you want to stop writing stupid things again and again, take > your bag, don't miss the bus to school, and listen to the teachers > instead of playing games on your calculator. > > Willy > PS: non need to reply either, I'll kill this thread and your address > here. > -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - One of many happy users: http://www.fastmail.fm/docs/quotes.html ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 5:58 ` johnrobertbanks 2007-04-07 7:15 ` Willy Tarreau @ 2007-04-07 14:11 ` Krzysztof Halasa 2007-04-07 15:07 ` johnrobertbanks ` (2 more replies) 1 sibling, 3 replies; 64+ messages in thread From: Krzysztof Halasa @ 2007-04-07 14:11 UTC (permalink / raw) To: johnrobertbanks; +Cc: Jan Harkes, linux-kernel, linux-fsdevel johnrobertbanks@fastmail.fm writes: > Why do you think I hate reiserfs developers? That is an insane claim. > Why would I hate reiser3 developers? > Why would I hate reiser4 developers? > Why would I even dislike them? > > I think Hans Reiser is a genius. Is that what you mean by hate? I think they could hire a person with a bit better marketing skills, though. People on a technical mailing list don't buy things just because something on TV told them they have to. > Answer this question. Why do YOU think I am antagonizing reiserfs > developers? That might be just a side effect. > Think about it,... read speeds that are some FOUR times the physical > disk read rate,... impossible without the use of compression (or > something similar). It's really impossible with compression only unless you're writing only zeros or stuff alike. I don't know what bonnie uses for testing but real life data doesn't compress 4 times. Two times, sometimes, but then it will be typically slower than disk access (I mean read, as write will be much slower). You can get faster I/O (both linear speed and access times) using multiple disks (mirrors etc). Perhaps some ZFS ideas would do us some good? Gzip - 3 files (zeros only, raw DV data from video camera, x86_64 kernel rpm file), 10 MB of data (10*1024*1024), done on tmpfs so no real disk speed factor. The CPU is AMD64 with 1 MB cache per core, 2600 MHz clock (clock scaling disabled). That's my typical usage pattern (well, not counting these zeros). $ l -Ggh zeros dv bin -rw-r--r-- 1 10M Apr 7 15:30 bin -rw-r--r-- 1 10M Apr 7 15:31 dv -rw-r--r-- 1 10M Apr 7 15:31 zeros $ for f in zeros dv bin; do time gzip $f; done real 0m0.112s real 0m0.686s real 0m0.559s Dealing with pure zeros gzip can get almost 90 MB/s compressing, but with DV and rpm it only does 14.5 and almost 18 MB/s respectively... $ l -Ggh zeros.gz dv.gz bin.gz -rw-r--r-- 1 10K Apr 7 15:31 zeros.gz -rw-r--r-- 1 9.1M Apr 7 15:31 dv.gz -rw-r--r-- 1 9.3M Apr 7 15:30 bin.gz ... and though the numbers may still sound impressive, space savings are less than 10%. $ for f in zeros dv bin; do time gunzip $f.gz; done real 0m0.067s real 0m0.131s real 0m0.120s Decompression gives 150 MB/s for zeros and ~ 80 MB/s for DV and rpm. $ for f in zeros dv bin; do time gzip -1 $f; done real 0m0.079s real 0m0.572s real 0m0.530s Supposed to be "fastest gzip". 126 MB/s for zeros but still less than 19 MB/s for DV and rpm. $ l -Ggh zeros.gz dv.gz bin.gz -rw-r--r-- 1 45K Apr 7 15:31 zeros.gz -rw-r--r-- 1 9.2M Apr 7 15:31 dv.gz -rw-r--r-- 1 9.3M Apr 7 15:30 bin.gz $ for f in zeros dv bin; do time gunzip $f.gz; done real 0m0.044s real 0m0.135s real 0m0.120s It seems gzip can decompress zeros with 227 MB/s rate. I assume the "4x read speed" claim comes from something like this. $ /sbin/hdparm -t /dev/sda /dev/sda: Timing buffered disk reads: 210 MB in 3.02 seconds = 69.59 MB/sec $ echo "69.59*4" | bc 278.36 Seems you'd need a faster algorithm, faster machine or slower disk - slower than this cheap SATA with disabled NCQ (NV SATA) at least: $ cat /sys/block/sda/device/model Maxtor 6V250F0 Please note that aplication-level compression usually gives way better results - the application knows much more. -- Krzysztof Halasa ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 14:11 ` Krzysztof Halasa @ 2007-04-07 15:07 ` johnrobertbanks 2007-04-07 16:05 ` Pekka Enberg 2007-04-07 17:10 ` Valdis.Kletnieks 2007-04-07 17:21 ` Valdis.Kletnieks 2 siblings, 1 reply; 64+ messages in thread From: johnrobertbanks @ 2007-04-07 15:07 UTC (permalink / raw) To: Krzysztof Halasa; +Cc: Jan Harkes, linux-kernel, linux-fsdevel Krzysztof -- Aren't you missing the point? Twice the speed would be great,... even a 50% increase,... even a 0% increase. I checked what bonnie++ actually writes to its test files, for you. It is about 98-99% zeros. Still, the results record sequential reads, of 232,729 K/sec, nearly four times the physical disk read rate, 63,160 K/sec, of the hard drive. The sequential writes are about three times the physical disk write rate. Even if the speed increase was zero, the more efficient use of disk space means that Reiser4 is worth investigating. People use RAID arrays to achieve speed increases. The people who developed RAID clearly thought that increases in speed were worth investigating. > > > Why do you think I hate reiserfs developers? That is an insane claim. > > Why would I hate reiser3 developers? > > Why would I hate reiser4 developers? > > Why would I even dislike them? > > > > I think Hans Reiser is a genius. Is that what you mean by hate? > > I think they could hire a person with a bit better marketing skills, > though. People on a technical mailing list don't buy things just > because something on TV told them they have to. I don't work for Reiser if that is what you are suggesting. And people buy all sorts of lies because someone on TV told them it was true. Did you believe Iraq had WMD (weapons of mass destruction) because a bunch of American liars told you this on TV? Millions of Americans did. > > Answer this question. Why do YOU think I am antagonizing reiserfs > > developers? > > That might be just a side effect. > -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - Send your email first class ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 15:07 ` johnrobertbanks @ 2007-04-07 16:05 ` Pekka Enberg 0 siblings, 0 replies; 64+ messages in thread From: Pekka Enberg @ 2007-04-07 16:05 UTC (permalink / raw) To: johnrobertbanks; +Cc: Krzysztof Halasa, Jan Harkes, linux-kernel, linux-fsdevel On 4/7/07, johnrobertbanks@fastmail.fm <johnrobertbanks@fastmail.fm> wrote: > I checked what bonnie++ actually writes to its test files, for you. It > is about 98-99% zeros. > > Still, the results record sequential reads, of 232,729 K/sec, nearly > four times the physical disk read rate, 63,160 K/sec, of the hard drive. Excellent! You've established the undeniable hard cold fact that reiser4 beats the crap out of all other filesystems, when the files are 98-99% filled with zeros. You've proven your point, so can we stop this thread now? ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 14:11 ` Krzysztof Halasa 2007-04-07 15:07 ` johnrobertbanks @ 2007-04-07 17:10 ` Valdis.Kletnieks 2007-04-08 16:31 ` Adrian Bunk 2007-04-07 17:21 ` Valdis.Kletnieks 2 siblings, 1 reply; 64+ messages in thread From: Valdis.Kletnieks @ 2007-04-07 17:10 UTC (permalink / raw) To: Krzysztof Halasa; +Cc: johnrobertbanks, Jan Harkes, linux-kernel, linux-fsdevel [-- Attachment #1: Type: text/plain, Size: 1622 bytes --] On Sat, 07 Apr 2007 16:11:46 +0200, Krzysztof Halasa said: > > Think about it,... read speeds that are some FOUR times the physical > > disk read rate,... impossible without the use of compression (or > > something similar). > > It's really impossible with compression only unless you're writing > only zeros or stuff alike. I don't know what bonnie uses for testing > but real life data doesn't compress 4 times. Two times, sometimes, All depends on your data. From a recent "compress the old logs" job on our syslog server: /logs/lennier.cc.vt.edu/2007/03/maillog-2007-0308: 85.4% -- replaced with /logs/lennier.cc.vt.edu/2007/03/maillog-2007-0308.gz And it wasn't a tiny file either - it's a busy mailserver, the logs run to several hundred megabytes a day. Syslogs *often* compress 90% or more, meaning a 10X compression. > but then it will be typically slower than disk access (I mean read, > as write will be much slower). Actually, as far back as 1998 or so, I was able to document 20% *speedups* on an AIX system that supported compressed file systems - and that was from when a 133mz PowerPC 604e was a *fast* machine. Since then, CPUs have gotten faster at a faster rate than disks have, even increasing the speedup. The basic theory is that unless you're sitting close to 100%CPU, it is *faster* to burn some CPU to compress/decompress a 4K chunk of data down to 2K, and then move 2K to the disk drive, than it is to move 4K. It's particularly noticable for larger files - if you can apply the compression to remove the need to move 2M of data faster than you can move 2M of data, you win. [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 17:10 ` Valdis.Kletnieks @ 2007-04-08 16:31 ` Adrian Bunk 0 siblings, 0 replies; 64+ messages in thread From: Adrian Bunk @ 2007-04-08 16:31 UTC (permalink / raw) To: Valdis.Kletnieks Cc: Krzysztof Halasa, johnrobertbanks, Jan Harkes, linux-kernel, linux-fsdevel On Sat, Apr 07, 2007 at 01:10:31PM -0400, Valdis.Kletnieks@vt.edu wrote: > On Sat, 07 Apr 2007 16:11:46 +0200, Krzysztof Halasa said: > > > > Think about it,... read speeds that are some FOUR times the physical > > > disk read rate,... impossible without the use of compression (or > > > something similar). > > > > It's really impossible with compression only unless you're writing > > only zeros or stuff alike. I don't know what bonnie uses for testing > > but real life data doesn't compress 4 times. Two times, sometimes, > > All depends on your data. From a recent "compress the old logs" job on > our syslog server: > > /logs/lennier.cc.vt.edu/2007/03/maillog-2007-0308: 85.4% -- replaced with /logs/lennier.cc.vt.edu/2007/03/maillog-2007-0308.gz > > And it wasn't a tiny file either - it's a busy mailserver, the logs run to > several hundred megabytes a day. Syslogs *often* compress 90% or more, > meaning a 10X compression. > > > but then it will be typically slower than disk access (I mean read, > > as write will be much slower). > > Actually, as far back as 1998 or so, I was able to document 20% *speedups* > on an AIX system that supported compressed file systems - and that was from > when a 133mz PowerPC 604e was a *fast* machine. Since then, CPUs have gotten > faster at a faster rate than disks have, even increasing the speedup. > > The basic theory is that unless you're sitting close to 100%CPU, it is *faster* > to burn some CPU to compress/decompress a 4K chunk of data down to 2K, and then > move 2K to the disk drive, than it is to move 4K. It's particularly noticable > for larger files - if you can apply the compression to remove the need to move > 2M of data faster than you can move 2M of data, you win. Counterpoints: - not only CPUs have became faster, RAM has become faster, too a kernel tree after an allyesconfig build is at about 1 GB which is less than half the size of RAM in my desktop computer if all disk accesses are asynchronous write accesses without any pressure of being done quickly, compression can't improve performance - today, much of the bigger data is already compressed data like mp3s or movies - for cases like logfiles or databases, application specific compression should give best results There might be special cases where compressed filesystems make sense, but my impression is that filesystem compresssion is not important and suited for current average systems. cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 14:11 ` Krzysztof Halasa 2007-04-07 15:07 ` johnrobertbanks 2007-04-07 17:10 ` Valdis.Kletnieks @ 2007-04-07 17:21 ` Valdis.Kletnieks 2007-04-08 0:41 ` Krzysztof Halasa 2 siblings, 1 reply; 64+ messages in thread From: Valdis.Kletnieks @ 2007-04-07 17:21 UTC (permalink / raw) To: Krzysztof Halasa; +Cc: johnrobertbanks, Jan Harkes, linux-kernel, linux-fsdevel [-- Attachment #1: Type: text/plain, Size: 1717 bytes --] On Sat, 07 Apr 2007 16:11:46 +0200, Krzysztof Halasa said: > > Gzip - 3 files (zeros only, raw DV data from video camera, x86_64 > kernel rpm file), 10 MB of data (10*1024*1024), > $ l -Ggh zeros dv bin > -rw-r--r-- 1 10M Apr 7 15:30 bin > -rw-r--r-- 1 10M Apr 7 15:31 dv > -rw-r--r-- 1 10M Apr 7 15:31 zeros > $ l -Ggh zeros.gz dv.gz bin.gz > -rw-r--r-- 1 10K Apr 7 15:31 zeros.gz > -rw-r--r-- 1 9.1M Apr 7 15:31 dv.gz > -rw-r--r-- 1 9.3M Apr 7 15:30 bin.gz > > ... and though the numbers may still sound impressive, space savings > are less than 10%. I am quite sure that the kernel RPM file is *already* compressed, at least somewhat. Otherwise, it's hard to explain this: -rw-r--r-- 1 529 263 17835757 Apr 5 00:19 kernel-2.6.20-1.3045.fc7.x86_64.rpm % du -s /lib/modules/2.6.20-1.3038.fc7/ 76436 /lib/modules/2.6.20-1.3038.fc7/ and it can't all be slack space at ends of files: % find /lib/modules/2.6.20-1.3038.fc7/ -type f | wc -l 1482 Even on a 4K filesystem, the *max* wasted slack would be about 4M. And what do you know - if you tar.gz that /lib/modules: % tar czf /tmp/kern.tar.gz /lib/modules/2.6.20-1.3038.fc7/ tar: Removing leading `/' from member names % ls -l /tmp/kern.tar.gz -rw-r--r-- 1 valdis valdis 15506359 2007-04-07 13:19 /tmp/kern.tar.gz The *compressed* tar is about 15M (remember the .rpm contained a 2M vmlinuz as well - that;s compressed too). So we're right up to the 17M of the .rpm, which indicates that the RPM is compressed at a factor close to tar.gz. I'd not be surprised to find out that your digital-video also contains at least some light compression - if it's mpeg or similar, that's already had some *heavy* compression done to it.... [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 17:21 ` Valdis.Kletnieks @ 2007-04-08 0:41 ` Krzysztof Halasa 0 siblings, 0 replies; 64+ messages in thread From: Krzysztof Halasa @ 2007-04-08 0:41 UTC (permalink / raw) To: Valdis.Kletnieks; +Cc: johnrobertbanks, Jan Harkes, linux-kernel, linux-fsdevel Valdis.Kletnieks@vt.edu writes: > I am quite sure that the kernel RPM file is *already* compressed, at least > somewhat. Sure - that's the point - it's better to have the tool compress data when it makes sense. OTOH I think Reiser4 fs is not about transparent compression, it's rather about the plugins etc. There are other filesystems with transparent compression, that's nothing new. -- Krzysztof Halasa ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 2:47 ` johnrobertbanks 2007-04-07 3:30 ` Jan Harkes @ 2007-04-07 17:39 ` Valdis.Kletnieks 2007-04-08 4:32 ` David H. Lynch Jr 1 sibling, 1 reply; 64+ messages in thread From: Valdis.Kletnieks @ 2007-04-07 17:39 UTC (permalink / raw) To: johnrobertbanks; +Cc: Jan Harkes, linux-kernel, linux-fsdevel [-- Attachment #1: Type: text/plain, Size: 1869 bytes --] On Fri, 06 Apr 2007 19:47:36 PDT, johnrobertbanks@fastmail.fm said: > On Fri, 6 Apr 2007 11:21:19 -0400, "Jan Harkes" <jaharkes@cs.cmu.edu> > > With compression there is a pretty high probability that one corrupted > > byte or disk block will result in loss of a considerably larger amount > > of data. > > Bad blocks are NOT dealt with by the filesystem,... so your comment is > irrelevant, or just plain wrong. > > If your filesystem is writing to bad blocks, then throw away your > operating system. You know... occasionally, blocks go bad *after* you write to them. If you have an uncompressed filesystem, it's often possible to recover most of the file , and just have a few 512-byte blocks of zeros, simply by doing something like 'dd if=bad.file of=bad.file bs=512 conv=noerror' or careful applications of 'skip=N'. If it's compressed, you usually can't recover the rest of a compression group if a previous block is lost. (And for those who talk about backups - yes, taking backups is good. However, it's the rare laptop or desktop machine that can afford the luxury of RAID disks, and backups usually happen once a night, if that often. This means that if you've been working hard on something important all day, and the disk blows chunks at 4:30PM, you *will* be suddenly very concerned over exactly how much you can recover off the failing drive.... And yes, I'd *love* to have all my users connected to nice SAN systems that do snapshotting and remote replication to DR sites and all that - but have you ever *priced* a petabyte of SAN storage, the NAS gateways to serve it to users, and upgrading several tens of thousands of network ports to Gig-E? Hint - US$1M would get us through a pilot, and probably $5M and up to *start* deployment. Anybody wanna buy us an EMC DMX-3? :) http://www.emc.com/products/systems/symmetrix/DMX_series/DMX3.jsp [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 17:39 ` Valdis.Kletnieks @ 2007-04-08 4:32 ` David H. Lynch Jr 0 siblings, 0 replies; 64+ messages in thread From: David H. Lynch Jr @ 2007-04-08 4:32 UTC (permalink / raw) To: linux-fsdevel Valdis.Kletnieks@vt.edu wrote: > > (And for those who talk about backups - yes, taking backups is good. > However, it's the rare laptop or desktop machine that can afford the > luxury of RAID disks, and backups usually happen once a night, if that > often. This means that if you've been working hard on something important > all day, and the disk blows chunks at 4:30PM, you *will* be suddenly very > concerned over exactly how much you can recover off the failing drive.... Been there done, that. I am not sure that RAID is physically practical for a laptop, but it may be substantially more practical and economical for a desktop or home server, than the hope that a home user might backup a system. I am sure that there are plenty of people with true stories of backup systems saving their lives. But my personal history involves systems with no backup, systems with poor backup, systems with wishful backup, and finally systems with failed backup. My first experience with tape drivers was disassembling and patching a QIC driver (not Linux) because small portions of the backup tape were unreadable (the client smoked while backing the system up) and the tape software thought recovery was an all or nothing proposition. My first personal rule of data disaster recovery is start with the data on the disk. I have only had rare instances where 99% of the data on a disk was not readable and recoverable - though for some reason the most critical data is always in that 1% that puts up a fight. Backup systems are what you go to after you have recovered everything possible from the disk. -- Dave Lynch DLA Systems Software Development: Embedded Linux 717.627.3770 dhlii@dlasys.net http://www.dlasys.net fax: 1.253.369.9244 Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-06 4:32 ` johnrobertbanks [not found] ` <20070406152119.GC4228@delft.aura.cs.cmu.edu> @ 2007-04-07 19:17 ` Lennart Sorensen 2007-04-08 0:44 ` johnrobertbanks 2007-04-08 4:06 ` David H. Lynch Jr 1 sibling, 2 replies; 64+ messages in thread From: Lennart Sorensen @ 2007-04-07 19:17 UTC (permalink / raw) To: johnrobertbanks Cc: H. Peter Anvin, Ignatich, reiserfs-list, linux-kernel, linux-fsdevel On Thu, Apr 05, 2007 at 09:32:11PM -0700, johnrobertbanks@fastmail.fm wrote: > Don't you agree, that "If they are accurate,.... THEN they are obviously > very relevant." Nope, if they are accurate and they have something to do with your particular usage and applications, then they are relevant. But it requires both to make them relevant. Although it may be possible for a benchmark to be relevant even if not particularly accurate. > I have set up a Reiser4 partition with gzip compression, here is the > difference in disk usage of a typical Debian installation on two 10GB > partitions, one with Reiser3 and the other with Reiser4. > > debian:/# df > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda3 10490104 6379164 4110940 61% /3 > /dev/sda7 9967960 2632488 7335472 27% /7 > > Partitions 3 and 7 have exactly the same data on them (the typical > Debian install). > > The partitions are exactly the same size (although df records different > sizes). > > Partition 3 is Reiser3 -- uses 6.4 GB. > Partition 7 is Reiser4 -- uses 2.6 GB. > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same > info). > > Don't you think this result is significant in itself? Only if you think disk space is so valuable that trading cpu time to compress and decompress the data is a good trade off. It is not one I would want to make. So you saved 3GB, what is that? About $1 worth? maybe $2 if you have raid. How much extra time and cpu will it take to access the data that way? How much extra electricity will the cpu use? What is your time worth? There are so many variables. Do you _trust_ reiserfs4 to not loose your data any more or less than some other filesystem? -- Len Sorensen ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 19:17 ` Lennart Sorensen @ 2007-04-08 0:44 ` johnrobertbanks 2007-04-08 1:27 ` Lennart Sorensen ` (2 more replies) 2007-04-08 4:06 ` David H. Lynch Jr 1 sibling, 3 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-08 0:44 UTC (permalink / raw) To: Lennart Sorensen Cc: H. Peter Anvin, Ignatich, reiserfs-list, linux-kernel, linux-fsdevel Lennart. Tell me again that these results from http://linuxhelp.150m.com/resources/fs-benchmarks.htm and http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm are not of interest to you. I still don't understand why you have your head in the sand. .-------------------------. | FILESYSTEM | TIME |DISK | | TYPE |(secs)|USAGE| .-------------------------. |REISER4 lzo | 1938 | 278 | |REISER4 gzip| 2295 | 213 | |REISER4 | 3462 | 692 | |EXT2 | 4092 | 816 | |JFS | 4225 | 806 | |EXT4 | 4408 | 816 | |EXT3 | 4421 | 816 | |XFS | 4625 | 779 | |REISER3 | 6178 | 793 | |FAT32 |12342 | 988 | |NTFS-3g |10414 | 772 | .-------------------------. Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0) Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources). OR LOOK AT THE FULL RESULTS: .-------------------------------------------------. |File |Disk |Copy |Copy |Tar |Unzip| Del | |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | |Type | (MB)| (1) | (2) |655MB|655MB| Gig | .-------------------------------------------------. |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | |NTFS | 779 | 781 | 173 | X | X | X | |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | |XFS | 799 | 220 | 173 | 119 | 90 | 106 | |JFS | 806 | 228 | 202 | 95 | 97 | 127 | |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | |FAT32 | 988 | 253 | 158 | 118 | 81 | 95 | .-------------------------------------------------. Each test was preformed 5 times and the average value recorded. Disk Usage: The amount of disk used to store the data (which was 3 different copies of the Linux kernel sources). The raw data (without filesystem meta-data, block alignment wastage, etc) was 655MB. Copy 655MB (1): Copy the data over a partition boundary. Copy 655MB (2): Copy the data within a partition. Tar Gzip 655MB: Tar and Gzip the data. Unzip UnTar 655MB: UnGzip and UnTar the data. Del 2.5 Gig: Delete everything just written (about 2.5 Gig). To get a feel for the performance increases that can be achieved by using compression, we look at the total time (in seconds) to run the test: bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c) .-------------------. | FILESYSTEM | TIME | .-------------------. |REISER4 lzo | 1938| |REISER4 gzip| 2295| |REISER4 | 3462| |EXT4 | 4408| |EXT2 | 4092| |JFS | 4225| |EXT3 | 4421| |XFS | 4625| |REISER3 | 6178| |FAT32 | 12342| |NTFS-3g |>10414| .-------------------. -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - Access all of your messages and folders wherever you are ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-08 0:44 ` johnrobertbanks @ 2007-04-08 1:27 ` Lennart Sorensen 2007-04-08 2:56 ` Theodore Tso 2007-04-08 4:32 ` Christer Weinigel 2 siblings, 0 replies; 64+ messages in thread From: Lennart Sorensen @ 2007-04-08 1:27 UTC (permalink / raw) To: johnrobertbanks Cc: H. Peter Anvin, Ignatich, reiserfs-list, linux-kernel, linux-fsdevel On Sat, Apr 07, 2007 at 05:44:57PM -0700, johnrobertbanks@fastmail.fm wrote: > Lennart. Tell me again that these results from > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm Hmm, copying kernel sources around. Not that interesting. How does it handle mpeg2 files (the majority of my personal data files is on a mythtv system). So a few large files, with mostly linear access, and the occational file deletion. Compression would gain nothing. > are not of interest to you. I still don't understand why you have your > head in the sand. Well I find it hard to get excited about new filesystems. I had sufficiently nasty data loses due to reiserfs 3 back in the early 2.4 kernel days, that I no longer get excited about new filesystems. now I want something I trust that hasn't destroyed any of my data. I tried XFS for a while, ut the early 2.6 kernels had some nasty bugs in XFS too that made that pretty much unusable. Now I just stick with ext3. Screw performance, give me something that works all the time. > .-------------------------. > | FILESYSTEM | TIME |DISK | > | TYPE |(secs)|USAGE| > .-------------------------. > |REISER4 lzo | 1938 | 278 | > |REISER4 gzip| 2295 | 213 | > |REISER4 | 3462 | 692 | > |EXT2 | 4092 | 816 | > |JFS | 4225 | 806 | > |EXT4 | 4408 | 816 | > |EXT3 | 4421 | 816 | > |XFS | 4625 | 779 | > |REISER3 | 6178 | 793 | > |FAT32 |12342 | 988 | > |NTFS-3g |10414 | 772 | > .-------------------------. > > > Column one measures the time taken to complete the bonnie++ benchmarking > test (run with the parameters bonnie++ -n128:128k:0) Time without cpu usage is not interesting. If you can increase filesystem speed by 10% by doubling cpu load, then I don't want the increase. It is all relative. Wall clock time by itself just doesn't contain enough data to be useful. > Column two, Disk Usage: measures the amount of disk used to store 655MB > of raw data (which was 3 different copies of the Linux kernel sources). I remember disk compression from the DOS days. Disk space is too cheap to bother with that crap anymore. I don't care if it can theoretically turn idle cpu cycles into improved disk speed. Sometimes I don't have idle cpu cycles to waste on that. > OR LOOK AT THE FULL RESULTS: > > .-------------------------------------------------. > |File |Disk |Copy |Copy |Tar |Unzip| Del | > |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | > |Type | (MB)| (1) | (2) |655MB|655MB| Gig | > .-------------------------------------------------. > |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | > |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | > |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | > |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | > |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | > |NTFS | 779 | 781 | 173 | X | X | X | > |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | > |XFS | 799 | 220 | 173 | 119 | 90 | 106 | > |JFS | 806 | 228 | 202 | 95 | 97 | 127 | > |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | > |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | > |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | > |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | > |FAT32 | 988 | 253 | 158 | 118 | 81 | 95 | > .-------------------------------------------------. > > > Each test was preformed 5 times and the average value recorded. > Disk Usage: The amount of disk used to store the data (which was 3 > different copies of the Linux kernel sources). > The raw data (without filesystem meta-data, block alignment wastage, > etc) was 655MB. > Copy 655MB (1): Copy the data over a partition boundary. > Copy 655MB (2): Copy the data within a partition. > Tar Gzip 655MB: Tar and Gzip the data. > Unzip UnTar 655MB: UnGzip and UnTar the data. > Del 2.5 Gig: Delete everything just written (about 2.5 Gig). > > To get a feel for the performance increases that can be achieved by > using compression, we look at the total time (in seconds) to run the > test: kernel sources are some of the most compressable data files around. Try with some interesting data instead, like something with larger files, mostly binary, which isn't likely to compess very much. > bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c) > > .-------------------. > | FILESYSTEM | TIME | > .-------------------. > |REISER4 lzo | 1938| > |REISER4 gzip| 2295| > |REISER4 | 3462| > |EXT4 | 4408| > |EXT2 | 4092| > |JFS | 4225| > |EXT3 | 4421| > |XFS | 4625| > |REISER3 | 6178| > |FAT32 | 12342| > |NTFS-3g |>10414| > .-------------------. > -- Well Reiser4 certainly looks impresive, but I still want to know what the cpu load is like, what the repair tools are like, how well it handles power failures in the middle of a write (I didn't like the way reiser3 would claim to have the filesystem back to a sane state, but the file it had been in the middle of writing now contained garbage in many parts of it, with no warning what so ever that this had taken place). Performance looks impresive, but based on the history of reiser3 there is a lot of historical damage to undo before I will even consider testing it out. -- Len Sorensen ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-08 0:44 ` johnrobertbanks 2007-04-08 1:27 ` Lennart Sorensen @ 2007-04-08 2:56 ` Theodore Tso 2007-04-08 4:13 ` johnrobertbanks 2007-04-08 4:32 ` Christer Weinigel 2 siblings, 1 reply; 64+ messages in thread From: Theodore Tso @ 2007-04-08 2:56 UTC (permalink / raw) To: johnrobertbanks Cc: Lennart Sorensen, H. Peter Anvin, Ignatich, reiserfs-list, linux-kernel, linux-fsdevel On Sat, Apr 07, 2007 at 05:44:57PM -0700, johnrobertbanks@fastmail.fm wrote: > To get a feel for the performance increases that can be achieved by > using compression, we look at the total time (in seconds) to run the > test: You mean the performance increases of writing a file which is mostly all zero's? Yawn..... - Ted ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-08 2:56 ` Theodore Tso @ 2007-04-08 4:13 ` johnrobertbanks 2007-04-08 12:48 ` Jose Celestino 2007-04-08 17:03 ` Theodore Tso 0 siblings, 2 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-08 4:13 UTC (permalink / raw) To: Theodore Tso Cc: Lennart Sorensen, H. Peter Anvin, Ignatich, reiserfs-list, linux-kernel, linux-fsdevel Teddy, It is a pity you don't address the full set of results, when you make your snide comments. Now since you have them,... why don't you make reasoned comment about them. You can read more here: http://linuxhelp.150m.com/resources/fs-benchmarks.htm and http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm .-------------------------. | FILESYSTEM | TIME |DISK | | TYPE |(secs)|USAGE| .-------------------------. |REISER4 lzo | 1938 | 278 | |REISER4 gzip| 2295 | 213 | |REISER4 | 3462 | 692 | |EXT2 | 4092 | 816 | |JFS | 4225 | 806 | |EXT4 | 4408 | 816 | |EXT3 | 4421 | 816 | |XFS | 4625 | 779 | |REISER3 | 6178 | 793 | |FAT32 |12342 | 988 | |NTFS-3g |10414 | 772 | .-------------------------. Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0) Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources). OR LOOK AT THE FULL RESULTS: .-------------------------------------------------. |File |Disk |Copy |Copy |Tar |Unzip| Del | |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | |Type | (MB)| (1) | (2) |655MB|655MB| Gig | .-------------------------------------------------. |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | |NTFS | 779 | 781 | 173 | X | X | X | |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | |XFS | 799 | 220 | 173 | 119 | 90 | 106 | |JFS | 806 | 228 | 202 | 95 | 97 | 127 | |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | |FAT32 | 988 | 253 | 158 | 118 | 81 | 95 | .-------------------------------------------------. Each test was preformed 5 times and the average value recorded. Disk Usage: The amount of disk used to store the data (which was 3 different copies of the Linux kernel sources). The raw data (without filesystem meta-data, block alignment wastage, etc) was 655MB. Copy 655MB (1): Copy the data over a partition boundary. Copy 655MB (2): Copy the data within a partition. Tar Gzip 655MB: Tar and Gzip the data. Unzip UnTar 655MB: UnGzip and UnTar the data. Del 2.5 Gig: Delete everything just written (about 2.5 Gig). To get a feel for the performance increases that can be achieved by using compression, we look at the total time (in seconds) to run the test: bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c) .-------------------. | FILESYSTEM | TIME | .-------------------. |REISER4 lzo | 1938| |REISER4 gzip| 2295| |REISER4 | 3462| |EXT4 | 4408| |EXT2 | 4092| |JFS | 4225| |EXT3 | 4421| |XFS | 4625| |REISER3 | 6178| |FAT32 | 12342| |NTFS-3g |>10414| .-------------------. On Sat, 7 Apr 2007 22:56:32 -0400, "Theodore Tso" <tytso@mit.edu> said: > On Sat, Apr 07, 2007 at 05:44:57PM -0700, johnrobertbanks@fastmail.fm > wrote: > > To get a feel for the performance increases that can be achieved by > > using compression, we look at the total time (in seconds) to run the > > test: > > You mean the performance increases of writing a file which is mostly > all zero's? Yawn..... > > - Ted -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - IMAP accessible web-mail ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-08 4:13 ` johnrobertbanks @ 2007-04-08 12:48 ` Jose Celestino 2007-04-08 13:21 ` johnrobertbanks 2007-04-08 17:03 ` Theodore Tso 1 sibling, 1 reply; 64+ messages in thread From: Jose Celestino @ 2007-04-08 12:48 UTC (permalink / raw) To: johnrobertbanks; +Cc: reiserfs-list, linux-kernel, linux-fsdevel Words by johnrobertbanks@fastmail.fm [Sat, Apr 07, 2007 at 09:13:32PM -0700]: > Teddy, > > It is a pity you don't address the full set of results, when you make > your snide comments. > > Now since you have them,... why don't you make reasoned comment about > them. > > You can read more here: > John, it is not because you keep posting the same numbers over and over again (or is this your new signature?) that they will be more substantiated (hint: cpu usage). Just more annoying each time. I'll remember to use reiser4 for my 90-percent-zero-files-no-need-for-proven-robustness-and-plenty-of-cpu-to-spare boxes. Thank you. > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > .-------------------------. > | FILESYSTEM | TIME |DISK | > | TYPE |(secs)|USAGE| > .-------------------------. > |REISER4 lzo | 1938 | 278 | > |REISER4 gzip| 2295 | 213 | > |REISER4 | 3462 | 692 | > |EXT2 | 4092 | 816 | > |JFS | 4225 | 806 | > |EXT4 | 4408 | 816 | > |EXT3 | 4421 | 816 | > |XFS | 4625 | 779 | > |REISER3 | 6178 | 793 | > |FAT32 |12342 | 988 | > |NTFS-3g |10414 | 772 | > .-------------------------. > > > Column one measures the time taken to complete the bonnie++ benchmarking > test (run with the parameters bonnie++ -n128:128k:0) > > Column two, Disk Usage: measures the amount of disk used to store 655MB > of raw data (which was 3 different copies of the Linux kernel sources). > > OR LOOK AT THE FULL RESULTS: > > .-------------------------------------------------. > |File |Disk |Copy |Copy |Tar |Unzip| Del | > |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | > |Type | (MB)| (1) | (2) |655MB|655MB| Gig | > .-------------------------------------------------. > |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | > |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | > |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | > |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | > |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | > |NTFS | 779 | 781 | 173 | X | X | X | > |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | > |XFS | 799 | 220 | 173 | 119 | 90 | 106 | > |JFS | 806 | 228 | 202 | 95 | 97 | 127 | > |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | > |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | > |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | > |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | > |FAT32 | 988 | 253 | 158 | 118 | 81 | 95 | > .-------------------------------------------------. > > > Each test was preformed 5 times and the average value recorded. > Disk Usage: The amount of disk used to store the data (which was 3 > different copies of the Linux kernel sources). > The raw data (without filesystem meta-data, block alignment wastage, > etc) was 655MB. > Copy 655MB (1): Copy the data over a partition boundary. > Copy 655MB (2): Copy the data within a partition. > Tar Gzip 655MB: Tar and Gzip the data. > Unzip UnTar 655MB: UnGzip and UnTar the data. > Del 2.5 Gig: Delete everything just written (about 2.5 Gig). > > > To get a feel for the performance increases that can be achieved by > using compression, we look at the total time (in seconds) to run the > test: > > bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c) > > .-------------------. > | FILESYSTEM | TIME | > .-------------------. > |REISER4 lzo | 1938| > |REISER4 gzip| 2295| > |REISER4 | 3462| > |EXT4 | 4408| > |EXT2 | 4092| > |JFS | 4225| > |EXT3 | 4421| > |XFS | 4625| > |REISER3 | 6178| > |FAT32 | 12342| > |NTFS-3g |>10414| > .-------------------. > > > > On Sat, 7 Apr 2007 22:56:32 -0400, "Theodore Tso" <tytso@mit.edu> said: > > On Sat, Apr 07, 2007 at 05:44:57PM -0700, johnrobertbanks@fastmail.fm > > wrote: > > > To get a feel for the performance increases that can be achieved by > > > using compression, we look at the total time (in seconds) to run the > > > test: > > > > You mean the performance increases of writing a file which is mostly > > all zero's? Yawn..... > > > > - Ted > -- > > johnrobertbanks@fastmail.fm > > -- > http://www.fastmail.fm - IMAP accessible web-mail > > - > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Jose Celestino ---------------------------------------------------------------- http://www.msversus.org/ ; http://techp.org/petition/show/1 http://www.vinc17.org/noswpat.en.html ---------------------------------------------------------------- "And on the trillionth day, Man created Gods." -- Thomas D. Pate ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-08 12:48 ` Jose Celestino @ 2007-04-08 13:21 ` johnrobertbanks 2007-04-08 14:14 ` Willy Tarreau 0 siblings, 1 reply; 64+ messages in thread From: johnrobertbanks @ 2007-04-08 13:21 UTC (permalink / raw) To: Jose Celestino; +Cc: reiserfs-list, linux-kernel, linux-fsdevel Jose, since you clearly have nothing useful to say. Why don't you let Teddy talk for himself. On Sun, 8 Apr 2007 13:48:11 +0100, "Jose Celestino" <japc@co.sapo.pt> said: > Words by johnrobertbanks@fastmail.fm [Sat, Apr 07, 2007 at 09:13:32PM > -0700]: > > Teddy, > > > > It is a pity you don't address the full set of results, when you make > > your snide comments. > > > > Now since you have them,... why don't you make reasoned comment about > > them. > > > > You can read more here: > > > > John, > > it is not because you keep posting the same numbers over and over again > (or is this your new signature?) that they will be more substantiated > (hint: cpu usage). > Just more annoying each time. > > I'll remember to use reiser4 for my > 90-percent-zero-files-no-need-for-proven-robustness-and-plenty-of-cpu-to-spare > boxes. Thank you. > > > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > > > .-------------------------. > > | FILESYSTEM | TIME |DISK | > > | TYPE |(secs)|USAGE| > > .-------------------------. > > |REISER4 lzo | 1938 | 278 | > > |REISER4 gzip| 2295 | 213 | > > |REISER4 | 3462 | 692 | > > |EXT2 | 4092 | 816 | > > |JFS | 4225 | 806 | > > |EXT4 | 4408 | 816 | > > |EXT3 | 4421 | 816 | > > |XFS | 4625 | 779 | > > |REISER3 | 6178 | 793 | > > |FAT32 |12342 | 988 | > > |NTFS-3g |10414 | 772 | > > .-------------------------. > > > > > > Column one measures the time taken to complete the bonnie++ benchmarking > > test (run with the parameters bonnie++ -n128:128k:0) > > > > Column two, Disk Usage: measures the amount of disk used to store 655MB > > of raw data (which was 3 different copies of the Linux kernel sources). > > > > OR LOOK AT THE FULL RESULTS: > > > > .-------------------------------------------------. > > |File |Disk |Copy |Copy |Tar |Unzip| Del | > > |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | > > |Type | (MB)| (1) | (2) |655MB|655MB| Gig | > > .-------------------------------------------------. > > |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | > > |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | > > |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | > > |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | > > |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | > > |NTFS | 779 | 781 | 173 | X | X | X | > > |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | > > |XFS | 799 | 220 | 173 | 119 | 90 | 106 | > > |JFS | 806 | 228 | 202 | 95 | 97 | 127 | > > |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | > > |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | > > |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | > > |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | > > |FAT32 | 988 | 253 | 158 | 118 | 81 | 95 | > > .-------------------------------------------------. > > > > > > Each test was preformed 5 times and the average value recorded. > > Disk Usage: The amount of disk used to store the data (which was 3 > > different copies of the Linux kernel sources). > > The raw data (without filesystem meta-data, block alignment wastage, > > etc) was 655MB. > > Copy 655MB (1): Copy the data over a partition boundary. > > Copy 655MB (2): Copy the data within a partition. > > Tar Gzip 655MB: Tar and Gzip the data. > > Unzip UnTar 655MB: UnGzip and UnTar the data. > > Del 2.5 Gig: Delete everything just written (about 2.5 Gig). > > > > > > To get a feel for the performance increases that can be achieved by > > using compression, we look at the total time (in seconds) to run the > > test: > > > > bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c) > > > > .-------------------. > > | FILESYSTEM | TIME | > > .-------------------. > > |REISER4 lzo | 1938| > > |REISER4 gzip| 2295| > > |REISER4 | 3462| > > |EXT4 | 4408| > > |EXT2 | 4092| > > |JFS | 4225| > > |EXT3 | 4421| > > |XFS | 4625| > > |REISER3 | 6178| > > |FAT32 | 12342| > > |NTFS-3g |>10414| > > .-------------------. > > > > > > > > On Sat, 7 Apr 2007 22:56:32 -0400, "Theodore Tso" <tytso@mit.edu> said: > > > On Sat, Apr 07, 2007 at 05:44:57PM -0700, johnrobertbanks@fastmail.fm > > > wrote: > > > > To get a feel for the performance increases that can be achieved by > > > > using compression, we look at the total time (in seconds) to run the > > > > test: > > > > > > You mean the performance increases of writing a file which is mostly > > > all zero's? Yawn..... > > > > > > - Ted > > -- > > > > johnrobertbanks@fastmail.fm > > > > -- > > http://www.fastmail.fm - IMAP accessible web-mail > > > > - > > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- > Jose Celestino > ---------------------------------------------------------------- > http://www.msversus.org/ ; http://techp.org/petition/show/1 > http://www.vinc17.org/noswpat.en.html > ---------------------------------------------------------------- > "And on the trillionth day, Man created Gods." -- Thomas D. Pate -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - Does exactly what it says on the tin ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-08 13:21 ` johnrobertbanks @ 2007-04-08 14:14 ` Willy Tarreau 0 siblings, 0 replies; 64+ messages in thread From: Willy Tarreau @ 2007-04-08 14:14 UTC (permalink / raw) To: johnrobertbanks Cc: Jose Celestino, reiserfs-list, linux-kernel, linux-fsdevel On Sun, Apr 08, 2007 at 06:21:29AM -0700, johnrobertbanks@fastmail.fm wrote: > Jose, > since you clearly have nothing useful to say. Why don't you let Teddy > talk for himself. John, You should first apply your own advice to yourself. Annoying everyone with the exact same mail 10 times a day is really disserving to the cause you pretend to defend. Please stop tainting reiser4's reputation, because I suspect that it can do far more things than what you make it look like. Its developers certainly need useful reports instead of a mentally deficient's rant. Now please call the nurse for your injection and go back to bed. Thank you in advance, Willy ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-08 4:13 ` johnrobertbanks 2007-04-08 12:48 ` Jose Celestino @ 2007-04-08 17:03 ` Theodore Tso 2007-04-08 18:18 ` Jeff Mahoney 1 sibling, 1 reply; 64+ messages in thread From: Theodore Tso @ 2007-04-08 17:03 UTC (permalink / raw) To: johnrobertbanks Cc: Lennart Sorensen, H. Peter Anvin, Ignatich, reiserfs-list, linux-kernel, linux-fsdevel The reason why I ignore the tar+gzip tests is that in the past Hans has rigged the test by using a tar ball which was generated by unpacking a set of kernel sources on a reiser4 filesystem, and then repacking them using tar+gzip. The result was a tar file whose files were optimally laid out so that reiser4 could insert them into the filesystem b-tree without doing any extra work. I can't say for sure whether or not this set of benchmarks has done this (there's not enough information describing the benchmark setup), but the sad fact of the matter is that people trying to pitch Reiser4 have generated for themselves a reputation for using rigged benchmarks. Hans's used of a carefully stacked and ordered tar file (which is the same as stacking a deck of cards), and your repeated use of the bonnee++ benchmarks despite being told that it is a meaningless result given the fact that well, zero's compress very well and most people are interested in storing a file of all zeros, has caused me to look at any benchmarks cited by Reiser4 partisans with a very jaundiced and skeptical eye. Fortunately for you, it's not up to me whether or not Reiser4 makes it into the kernel. And if it works for you, hey, go wild. You can always patch it into your own kernel and encourage others to do the same with respect to getting it tested and adopted. My personal take on it is that Reiser3, Reiser4 and JFS suffer the same problems, which is to say they have a very small and limited development community, and this was referenced in Novell's decision to drop Reiser3: http://linux.wordpress.com/2006/09/27/suse-102-ditching-reiserfs-as-it-default-fs/ SuSE has deprecated Reiser3 *and* JFS, and I believe quite strongly it is the failure of the organizations to attract a diverse development community is ultimately what doomed them in the long term, both in terms of support as the kernel migrated and new feature support. It is for that reason that Hans' personality traits that tend to drive away those developers who would help them, beyond those that he hires, is what has been so self-destructive to Reiser4. Read the announcement Jeff Mahoney from SUSE Labs again; he pointed out was that reiser3 was getting dropped even though it performs better than ext3 in some scenarios. There are many other considerations, such as a filesystem's robustness in case on-disk corruption, long term maintenance as the kernel maintains, availability of developers to provide bug fixes, how well the system performs on systems with multiple cores/CPU's, etc. - Ted ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-08 17:03 ` Theodore Tso @ 2007-04-08 18:18 ` Jeff Mahoney 0 siblings, 0 replies; 64+ messages in thread From: Jeff Mahoney @ 2007-04-08 18:18 UTC (permalink / raw) To: Theodore Tso, johnrobertbanks, Lennart Sorensen, H. Peter Anvin, Ignatich, reiserfs-list, linux-kernel, linux-fsdevel -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Theodore Tso wrote: > The reason why I ignore the tar+gzip tests is that in the past Hans > has rigged the test by using a tar ball which was generated by > unpacking a set of kernel sources on a reiser4 filesystem, and then > repacking them using tar+gzip. The result was a tar file whose files > were optimally laid out so that reiser4 could insert them into the > filesystem b-tree without doing any extra work. > > I can't say for sure whether or not this set of benchmarks has done > this (there's not enough information describing the benchmark setup), > but the sad fact of the matter is that people trying to pitch Reiser4 > have generated for themselves a reputation for using rigged > benchmarks. Hans's used of a carefully stacked and ordered tar file > (which is the same as stacking a deck of cards), and your repeated use > of the bonnee++ benchmarks despite being told that it is a meaningless > result given the fact that well, zero's compress very well and most > people are interested in storing a file of all zeros, has caused me to > look at any benchmarks cited by Reiser4 partisans with a very > jaundiced and skeptical eye. > > Fortunately for you, it's not up to me whether or not Reiser4 makes it > into the kernel. And if it works for you, hey, go wild. You can > always patch it into your own kernel and encourage others to do the > same with respect to getting it tested and adopted. My personal take > on it is that Reiser3, Reiser4 and JFS suffer the same problems, which > is to say they have a very small and limited development community, > and this was referenced in Novell's decision to drop Reiser3: > > http://linux.wordpress.com/2006/09/27/suse-102-ditching-reiserfs-as-it-default-fs/ > > SuSE has deprecated Reiser3 *and* JFS, and I believe quite strongly it > is the failure of the organizations to attract a diverse development > community is ultimately what doomed them in the long term, both in > terms of support as the kernel migrated and new feature support. It > is for that reason that Hans' personality traits that tend to drive > away those developers who would help them, beyond those that he hires, > is what has been so self-destructive to Reiser4. Read the > announcement Jeff Mahoney from SUSE Labs again; he pointed out was > that reiser3 was getting dropped even though it performs better than > ext3 in some scenarios. There are many other considerations, such as > a filesystem's robustness in case on-disk corruption, long term > maintenance as the kernel maintains, availability of developers to > provide bug fixes, how well the system performs on systems with > multiple cores/CPU's, etc. Those are all arguments I've made and still stand by, but I should address one point that has been repeated fairly often. Novell _isn't_ dropping support for Reiser3 in any of our products. The change only refers to the choice of a default file system. Most users don't care about which file system they use, and those that do are still free to choose reiser3 if they want it. We'll support it and I still have patches under development to improve it. - -Jeff - -- Jeff Mahoney SUSE Labs -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFGGTHYLPWxlyuTD7IRAj0SAJ4txD5NoStOA4GFgkzcXDdE/Xf9ngCZATNL QtyNTGbi6YFbNF71T5C9hTA= =Emwr -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-08 0:44 ` johnrobertbanks 2007-04-08 1:27 ` Lennart Sorensen 2007-04-08 2:56 ` Theodore Tso @ 2007-04-08 4:32 ` Christer Weinigel 2007-04-08 21:50 ` johnrobertbanks 2007-04-09 18:35 ` Reiser4. BEST FILESYSTEM EVER Nate Diller 2 siblings, 2 replies; 64+ messages in thread From: Christer Weinigel @ 2007-04-08 4:32 UTC (permalink / raw) To: johnrobertbanks Cc: Lennart Sorensen, H. Peter Anvin, Ignatich, reiserfs-list, linux-kernel, linux-fsdevel johnrobertbanks@fastmail.fm writes: > Lennart. Tell me again that these results from > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > are not of interest to you. I still don't understand why you have your > head in the sand. Oh, for fucks sake, stop sounding like a broken record. You have repeated the same totally meaningless statistics more times than I care to count. Please shut the fuck up. As you discovered yourself (even though you seem to fail to understand the significance of your discovery), bonnie writes files that consist of mostly zeroes. If your normal use cases consist of creating a bunch of files containing zeroes, reiser4 with compression will do great. Just lovely. Except that nobody sane would store a lot of files containing zeroes, except an an excercize in mental masturbation. So the two bonnie benchmarks with lzo and gzip are totally meaningless for any real life usages. As for the amount of disk needed to store three kernel trees, the figures you quote show that Reiser4 does tail combining where the tail of multiple files are stored in one disk block. A nice trick that seems save you about 15% disk space compared to ext3. Now you have to realise what that means, it means that if the disk block containing those tails (or any metadata pointing at that block) gets corrupted, instead of just losing one disk block for one file, you will have lost the tail for all the files sharing that disk block. Depending on your personal prioritites, saving 15% of the space may be worth the risk to you, or maybe not. Personally, for the only disk I'm short on space on, I mostly store flac encoded images of my CD collection, and saving 2kByte out of every 300MByte disk simply doesn't make any difference, and I much prefer a stable file system that I can trust not to lose my data. You might make different choices. The same goes for just about every feature that you tout, it has its advantages, and it has its disadvantages. Doing compression on data is great if the data you store is compressible, and sucks if it isn't. Doing compression on each disk block and then packing multiple compressed blocks into each physical disk block will probably save some space if the data is compressible, but at the same time it means that you will spend a lot of CPU (and cache footprint) compressing and uncompressing that data. On a single user system where the CPU is mostly idle it might not make much of a difference, on a heavily loaded multiuser system it might do. Logs can be compressed quite well using a block based compression scheme, but the logs can be compressed even better by doing compression on the whole file with gzip. So what's the best choice, to do transparent compression on the fly giving ok compression or teaching the userspace tools to do compression of old logs and get really good compression? Or maybe disk space really isn't that important anyway and the best thing is to just leave the logs uncompressed. Another example: one of the things Reiser3 is supposed to be really good for is storing an INN news spool, doing tail merging of lots of individual files containing articles gives a great space saving, and since it's just a news spool, reliability in face of a system crashes really don't matter all that much. On the other hand, INN's Cyclic News File System running on top of ext2 is probably an even better choice in that case. What do you want to use? What I want to get at is that you can troll the mailing lists (and crossposting stupid inflammatory material with an inane subject to a bunch of mailing lists the way you have done definitely is trolling) trying to say that whatever you're trying to sell is the best, but at the end, if a file system is better or not is a lot more complex than quoting just one benchmark (which, once again, is meaningless, compressing a lot of zeroes is simple and really does not tell you anything about real world usages). And there are other considerations too, even if Reiser4 would be the best thing since sliced breadd, can I trust Hans Reiser to support Reiser4 for the next five years? Or will he drop support for Reiser4 the same way he dropped support for the old Reiser3 when Reiser4 came along? Or will he drop Reiser4 when the grant to do Reiser 4 development expires? /Christer -- "Just how much can I get away with and still go to heaven?" Christer Weinigel <christer@weinigel.se> http://www.weinigel.se ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel 2007-04-08 4:32 ` Christer Weinigel @ 2007-04-08 21:50 ` johnrobertbanks 2007-04-09 18:35 ` Reiser4. BEST FILESYSTEM EVER Nate Diller 1 sibling, 0 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-08 21:50 UTC (permalink / raw) To: Christer Weinigel; +Cc: linux-kernel, linux-fsdevel, reiserfs-list Christer Weinigel: Until YOU, have actually used the REISER4 filesystem yourself, I think YOU OWE IT to the people on the linux-kernel mailing list, to, AS YOU SAY, shut the fuck up. Even reading up on the REISER4 filesystem would help. Applying a little intelligence would undoubtedly help too. > johnrobertbanks@fastmail.fm writes: > > > Lennart. Tell me again that these results from > > > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > > > are not of interest to you. I still don't understand why you > > have your head in the sand. > > Oh, for fucks sake, stop sounding like a broken record. Oh, for fucks sake, would you, and your religious anti-REISER cohorts, stop sounding like a broken record. > You have repeated the same totally meaningless statistics more > times than I care to count. Please shut the fuck up. You, and your religious anti-REISER cohorts, have indeed repeated the same broken arguments (if you can call them such) more times than I care to count. NO statistics, NO real facts, just selective MANIPULATION of facts. > Please shut the fuck up. Yes, why don't you politely, shut the fuck up. Until YOU, have actually used the REISER4 filesystem yourself, I think YOU OWE IT to the people on the linux-kernel mailing list, to shut the fuck up, as YOU say. I guess, the fact that you are a TOTAL HYPOCRITE, has completely escaped you. By the way: Did I thank you "delightful" people for the "pleasant" welcome to the linux-kernel mailing list? ----------------------------------------------------------------- > So the two bonnie benchmarks with lzo and gzip are > totally meaningless for any real life usages. YOU (yes, the one with no experience and next to NO knowledge on the subject) claim that because bonnie++ writes files that are mostly zeros, the results are meaningless. It should be mentioned that bonnie++ writes files that are mostly zero for all the filesystems compared. So the results are meaningful, contrary to would you claim. And hopefully all will notice that you just ignore these tests: .-------------------------------------------------. |File |Disk |Copy |Copy |Tar |Unzip| Del | |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | |Type | (MB)| (1) | (2) |655MB|655MB| Gig | .-------------------------------------------------. |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | |NTFS | 779 | 781 | 173 | X | X | X | |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | |XFS | 799 | 220 | 173 | 119 | 90 | 106 | |JFS | 806 | 228 | 202 | 95 | 97 | 127 | |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | |FAT32 | 988 | 253 | 158 | 118 | 81 | 95 | .-------------------------------------------------. where the files are definitely NOT mostly zeros. Your negligence has to be deliberate,... but why? Are you manipulating the facts just to try and win an argument? Most sane people will realize, that what you say is simply wrong. ALSO YOU IGNORE examples offered by others, on lkml, which contradict your assertion: FOR EXAMPLE: > I see the same thing with my nightly scripts that do syslog analysis, last year > I trimmed 2 hours from the nightly run by processing compressed files instead of > uncompressed ones (after I did this I configured it to compress the files as > they are rolled, but rolling every 5 min the compression takes <20 seconds, so > the compression is < 30 min) >From David Lang http://lkml.org/lkml/2007/4/7/196 Willy Tarreau also mentions this situation in a couple of articles. Let me spoon feed you: David has said that compressing the logs takes 24 x 12 x 20 secs = 5,760 secs = 1.6 hours of CPU time (over the day) but he saves 2 hours of CPU time on the daily syslog analysis. For a total (minimum) saving of 24 minutes. The actual saving is probably much greater. It depends on the CPU utilization when not compressing, ie, whether you are using ide CPU cycles or not. I guess it also depends on whether you can go home one and a half hours earlier by using compression, or if your boss makes you stick around anyway. NOTE THAT THE FILES IN THIS EXAMPLE ARE ALSO NOT MAINLY ZEROS. MAYBE you just lacked the knowledge to understand what David was saying, or maybe your desire to denigrate REISER4 is so strong, that you simply don't care what other people say about similar circumstances. I am not sure why you have to be spoon feed on these matters, or why you adamantly refuse to find the facts of the matter, for yourself. ----------------------------------------------------------------- ----------------------------------------------------------------- On Fri, 6 Apr 2007 23:30:49 -0400, "Jan Harkes" <jaharkes@cs.cmu.edu> said: > > Since you decide to publically respond to a private email, but not only > you did not 'discuss' anything I wrote and in fact cut out most of the > useful information in my reply I guess I will have to repeat my > observations. You are a funny guy Jan. Here you are, once again, cutting out my most useful information, ie, the data I was discussing, while complaining that I cut out your most useful information. You know,... you cut out this bit: ----------------------------------------------------------------- > The following benchmarks are from > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm or, > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > .-------------------------. > | FILESYSTEM | TIME |DISK | > | TYPE |(secs)|USAGE| > .-------------------------. > |REISER4 lzo | 1938 | 278 | > |REISER4 gzip| 2295 | 213 | > |REISER4 | 3462 | 692 | > |EXT2 | 4092 | 816 | > |JFS | 4225 | 806 | > |EXT4 | 4408 | 816 | > |EXT3 | 4421 | 816 | > |XFS | 4625 | 779 | > |REISER3 | 6178 | 793 | > |FAT32 |12342 | 988 | > |NTFS-3g |10414 | 772 | > .-------------------------. > > Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0) > > Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources). ----------------------------------------------------------------- And this bit: Jan,... Here is another section, you conveniently cut out. Maybe you should explain why you cut out this section? Was it embarrassing to you? I mean, your statement is sort of correct,... but it shows a basic misunderstanding of filesystems. > > With compression there is a pretty high probability that one corrupted > > byte or disk block will result in loss of a considerably larger amount > > of data. > > Bad blocks are NOT dealt with by the filesystem,... so your comment is > irrelevant, or just plain wrong. > > If your filesystem is writing to bad blocks, then throw away your > operating system. ----------------------------------------------------------------- It is true that I considered your "most useful information," an irrelevant section, which is why it was cut out (ignored). I did not see my doing this, any worse than you doing it. I did not realize that you were being so impolite. As to your email being private, I had thought I had joined a mailing list. I had no idea your email was meant to be private and just considered it like all the others. Now you mention it, I wondered why the email did not automatically list the mailing lists, as recipients, and I had to add them. If I had realized this I may have added the mailing lists as recipients, anyway. It would be like me to do such. However, you should understand that I am new to mailing lists. > > > > Do you really have to repeat the results in every email you sent? > > > > Damn, I did it again. WHY DO YOU CARE? > > Because I saw them the first time around. And although the performance > difference on those micro benchmarks seems quite impressive, I'm not > convinced. So, likewise, I saw your comments (you know the ones you miss so much) the first time around, and I ignored them, as I was not convinced of their worth. The benchmarks measure certain data. Its fine you do not read into them, stuff that isn't there, like reliability, for example. > > Look, its simple, I am (among other things) discussing these results, so > > people need to see them. > > However, you do not discuss, you just repeat, and repeat, and repeat. I never said I wanted discussion, you just said I did. You just repeat, and repeat, and repeat. In reality, I quite appreciate reasonable discussion. But, I doubt I will get much from you. > But for what reason. Do you want an actual discussion, or do you hate > the reiserfs developers so much that you want to antagonize any and all > other Linux file systems developers? Why do you think I hate reiserfs developers? That is an insane claim. Why would I hate reiser3 developers? Why would I hate reiser4 developers? Why would I even dislike them? I think Hans Reiser is a genius. Is that what you mean by hate? Answer this question. Why do YOU think I am antagonizing reiserfs developers? You must have a reason for stating what you have. > > By the way, I have pulled the plug on my REISER4 system, a number of > > times now, and it recovers without problem. > > Very nice for you. I guess you have also not been hit by out of memory > conditions or failing partial writes. So I guess we can ignore the patch > that was just sent a day or two ago to the mailing list because you > succesfully pulled the plug, a number of times at that. Why are you attacking me with sarcasm, when I have just stated a simple fact? > > > > I have set up a Reiser4 partition with gzip compression, here is the > > > > difference in disk usage of a typical Debian installation on two 10GB > > > > partitions, one with Reiser3 and the other with Reiser4. > > > > > > > > debian:/# df > > > > Filesystem 1K-blocks Used Available Use% Mounted on > > > > /dev/sda3 10490104 6379164 4110940 61% /3 > > > > /dev/sda7 9967960 2632488 7335472 27% /7 > > > ... > > > > Partition 3 is Reiser3 -- uses 6.4 GB. > > > > Partition 7 is Reiser4 -- uses 2.6 GB. > > > > > > > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 > > > > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same > > > > info). > > > > > > Wow, consider me totally and completely, unimpressed. > > > > > Here is the part of my email that you seemed to totally ignore, Yes, I ignored it,.... is that a crime? > You've just saved yourself $3.80, now go get yourself a latte. > (see. http://tomayko.com/weblog/2006/09/11/that-dilbert-cartoon) > > Seriously, disk storage is getting less expensive all the time, you can > already buy a 250GB SATA drive for $70. Also, compression doesn't help > once you store already compressed data such as jpeg images, mp3 files, > or mpeg2/4/divx video files. Not only are the savings non-existant, but > we still end up with the disadvantage that corruption propagates to > multiple files because of the compression in the file system. I believe that compression in REISER4 is just a wrapper for gzip or lzo. So each file is compressed and your "corruption propagation" is just in your imagination. > And if it doesn't propagate across multiple files, the compression can't > be all that good It can be as good as gzipping every file, ie, pretty damn good. > since it can't benefit as much from similarity between > files. So if that is the case and you really want to save diskspace you > almost have to look at read-only compressed filesystems such as cramfs, > squashfs, zisofs, cloop and various other variants in combination with > a unionfs overlay to get read/write functionality. > > But in the end everything is a tradeoff. You can save diskspace, but > increase the cost of corruption. You deliberately ignored the fact that bad blocks are NOT dealt with by the filesystem,... but by the operating system. Like I said: If your filesystem is writing to bad blocks, then throw away your operating system. > Use a better compression algorithm, but > that uses more CPU or which is visible in performance of the > application. Not necessarily, just look at the bonnie++ tests. The tests show that using compression (with todays hardware) leads to read speeds that are some FOUR times the physical disk read rate,... but you ignore this as you have some strange anti-REISER religion. Think about it,... read speeds that are some FOUR times the physical disk read rate,... impossible without the use of compression (or something similar). > This can be offset by caching more, and being lazier with > writebacks, but that hurts on-disk consistency, creates more memory > pressure (more swapout/paging) and generally slows down other > applications that aren't actually accessing the disk. Having a fast > multi-core cpu and lots of memory helps a lot, but at some point what > tradeoff did we just make, we saved a couple of dollars not having to > buy a larger disk, but we spend considerably more on the more > expensive cpu and memory. > Wow, consider me STILL totally impressed by your AMAZING BIAS. Would you like to tell me why you are SO BIASED against REISER4? John. -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - The professional email service ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel @ 2007-04-08 21:50 ` johnrobertbanks 0 siblings, 0 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-08 21:50 UTC (permalink / raw) To: Christer Weinigel; +Cc: linux-kernel, linux-fsdevel, reiserfs-list Christer Weinigel: Until YOU, have actually used the REISER4 filesystem yourself, I think YOU OWE IT to the people on the linux-kernel mailing list, to, AS YOU SAY, shut the fuck up. Even reading up on the REISER4 filesystem would help. Applying a little intelligence would undoubtedly help too. > johnrobertbanks@fastmail.fm writes: > > > Lennart. Tell me again that these results from > > > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > > > are not of interest to you. I still don't understand why you > > have your head in the sand. > > Oh, for fucks sake, stop sounding like a broken record. Oh, for fucks sake, would you, and your religious anti-REISER cohorts, stop sounding like a broken record. > You have repeated the same totally meaningless statistics more > times than I care to count. Please shut the fuck up. You, and your religious anti-REISER cohorts, have indeed repeated the same broken arguments (if you can call them such) more times than I care to count. NO statistics, NO real facts, just selective MANIPULATION of facts. > Please shut the fuck up. Yes, why don't you politely, shut the fuck up. Until YOU, have actually used the REISER4 filesystem yourself, I think YOU OWE IT to the people on the linux-kernel mailing list, to shut the fuck up, as YOU say. I guess, the fact that you are a TOTAL HYPOCRITE, has completely escaped you. By the way: Did I thank you "delightful" people for the "pleasant" welcome to the linux-kernel mailing list? ----------------------------------------------------------------- > So the two bonnie benchmarks with lzo and gzip are > totally meaningless for any real life usages. YOU (yes, the one with no experience and next to NO knowledge on the subject) claim that because bonnie++ writes files that are mostly zeros, the results are meaningless. It should be mentioned that bonnie++ writes files that are mostly zero for all the filesystems compared. So the results are meaningful, contrary to would you claim. And hopefully all will notice that you just ignore these tests: .-------------------------------------------------. |File |Disk |Copy |Copy |Tar |Unzip| Del | |System |Usage|655MB|655MB|Gzip |UnTar| 2.5 | |Type | (MB)| (1) | (2) |655MB|655MB| Gig | .-------------------------------------------------. |REISER4 gzip | 213 | 148 | 68 | 83 | 48 | 70 | |REISER4 lzo | 278 | 138 | 56 | 80 | 34 | 84 | |REISER4 tails| 673 | 148 | 63 | 78 | 33 | 65 | |REISER4 | 692 | 148 | 55 | 67 | 25 | 56 | |NTFS3g | 772 |1333 |1426 | 585 | 767 | 194 | |NTFS | 779 | 781 | 173 | X | X | X | |REISER3 | 793 | 184 | 98 | 85 | 63 | 22 | |XFS | 799 | 220 | 173 | 119 | 90 | 106 | |JFS | 806 | 228 | 202 | 95 | 97 | 127 | |EXT4 extents | 806 | 162 | 55 | 69 | 36 | 32 | |EXT4 default | 816 | 174 | 70 | 74 | 42 | 50 | |EXT3 | 816 | 182 | 74 | 73 | 43 | 51 | |EXT2 | 816 | 201 | 82 | 73 | 39 | 67 | |FAT32 | 988 | 253 | 158 | 118 | 81 | 95 | .-------------------------------------------------. where the files are definitely NOT mostly zeros. Your negligence has to be deliberate,... but why? Are you manipulating the facts just to try and win an argument? Most sane people will realize, that what you say is simply wrong. ALSO YOU IGNORE examples offered by others, on lkml, which contradict your assertion: FOR EXAMPLE: > I see the same thing with my nightly scripts that do syslog analysis, last year > I trimmed 2 hours from the nightly run by processing compressed files instead of > uncompressed ones (after I did this I configured it to compress the files as > they are rolled, but rolling every 5 min the compression takes <20 seconds, so > the compression is < 30 min) From David Lang http://lkml.org/lkml/2007/4/7/196 Willy Tarreau also mentions this situation in a couple of articles. Let me spoon feed you: David has said that compressing the logs takes 24 x 12 x 20 secs = 5,760 secs = 1.6 hours of CPU time (over the day) but he saves 2 hours of CPU time on the daily syslog analysis. For a total (minimum) saving of 24 minutes. The actual saving is probably much greater. It depends on the CPU utilization when not compressing, ie, whether you are using ide CPU cycles or not. I guess it also depends on whether you can go home one and a half hours earlier by using compression, or if your boss makes you stick around anyway. NOTE THAT THE FILES IN THIS EXAMPLE ARE ALSO NOT MAINLY ZEROS. MAYBE you just lacked the knowledge to understand what David was saying, or maybe your desire to denigrate REISER4 is so strong, that you simply don't care what other people say about similar circumstances. I am not sure why you have to be spoon feed on these matters, or why you adamantly refuse to find the facts of the matter, for yourself. ----------------------------------------------------------------- ----------------------------------------------------------------- On Fri, 6 Apr 2007 23:30:49 -0400, "Jan Harkes" <jaharkes@cs.cmu.edu> said: > > Since you decide to publically respond to a private email, but not only > you did not 'discuss' anything I wrote and in fact cut out most of the > useful information in my reply I guess I will have to repeat my > observations. You are a funny guy Jan. Here you are, once again, cutting out my most useful information, ie, the data I was discussing, while complaining that I cut out your most useful information. You know,... you cut out this bit: ----------------------------------------------------------------- > The following benchmarks are from > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm or, > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > .-------------------------. > | FILESYSTEM | TIME |DISK | > | TYPE |(secs)|USAGE| > .-------------------------. > |REISER4 lzo | 1938 | 278 | > |REISER4 gzip| 2295 | 213 | > |REISER4 | 3462 | 692 | > |EXT2 | 4092 | 816 | > |JFS | 4225 | 806 | > |EXT4 | 4408 | 816 | > |EXT3 | 4421 | 816 | > |XFS | 4625 | 779 | > |REISER3 | 6178 | 793 | > |FAT32 |12342 | 988 | > |NTFS-3g |10414 | 772 | > .-------------------------. > > Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0) > > Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources). ----------------------------------------------------------------- And this bit: Jan,... Here is another section, you conveniently cut out. Maybe you should explain why you cut out this section? Was it embarrassing to you? I mean, your statement is sort of correct,... but it shows a basic misunderstanding of filesystems. > > With compression there is a pretty high probability that one corrupted > > byte or disk block will result in loss of a considerably larger amount > > of data. > > Bad blocks are NOT dealt with by the filesystem,... so your comment is > irrelevant, or just plain wrong. > > If your filesystem is writing to bad blocks, then throw away your > operating system. ----------------------------------------------------------------- It is true that I considered your "most useful information," an irrelevant section, which is why it was cut out (ignored). I did not see my doing this, any worse than you doing it. I did not realize that you were being so impolite. As to your email being private, I had thought I had joined a mailing list. I had no idea your email was meant to be private and just considered it like all the others. Now you mention it, I wondered why the email did not automatically list the mailing lists, as recipients, and I had to add them. If I had realized this I may have added the mailing lists as recipients, anyway. It would be like me to do such. However, you should understand that I am new to mailing lists. > > > > Do you really have to repeat the results in every email you sent? > > > > Damn, I did it again. WHY DO YOU CARE? > > Because I saw them the first time around. And although the performance > difference on those micro benchmarks seems quite impressive, I'm not > convinced. So, likewise, I saw your comments (you know the ones you miss so much) the first time around, and I ignored them, as I was not convinced of their worth. The benchmarks measure certain data. Its fine you do not read into them, stuff that isn't there, like reliability, for example. > > Look, its simple, I am (among other things) discussing these results, so > > people need to see them. > > However, you do not discuss, you just repeat, and repeat, and repeat. I never said I wanted discussion, you just said I did. You just repeat, and repeat, and repeat. In reality, I quite appreciate reasonable discussion. But, I doubt I will get much from you. > But for what reason. Do you want an actual discussion, or do you hate > the reiserfs developers so much that you want to antagonize any and all > other Linux file systems developers? Why do you think I hate reiserfs developers? That is an insane claim. Why would I hate reiser3 developers? Why would I hate reiser4 developers? Why would I even dislike them? I think Hans Reiser is a genius. Is that what you mean by hate? Answer this question. Why do YOU think I am antagonizing reiserfs developers? You must have a reason for stating what you have. > > By the way, I have pulled the plug on my REISER4 system, a number of > > times now, and it recovers without problem. > > Very nice for you. I guess you have also not been hit by out of memory > conditions or failing partial writes. So I guess we can ignore the patch > that was just sent a day or two ago to the mailing list because you > succesfully pulled the plug, a number of times at that. Why are you attacking me with sarcasm, when I have just stated a simple fact? > > > > I have set up a Reiser4 partition with gzip compression, here is the > > > > difference in disk usage of a typical Debian installation on two 10GB > > > > partitions, one with Reiser3 and the other with Reiser4. > > > > > > > > debian:/# df > > > > Filesystem 1K-blocks Used Available Use% Mounted on > > > > /dev/sda3 10490104 6379164 4110940 61% /3 > > > > /dev/sda7 9967960 2632488 7335472 27% /7 > > > ... > > > > Partition 3 is Reiser3 -- uses 6.4 GB. > > > > Partition 7 is Reiser4 -- uses 2.6 GB. > > > > > > > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3 > > > > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same > > > > info). > > > > > > Wow, consider me totally and completely, unimpressed. > > > > > Here is the part of my email that you seemed to totally ignore, Yes, I ignored it,.... is that a crime? > You've just saved yourself $3.80, now go get yourself a latte. > (see. http://tomayko.com/weblog/2006/09/11/that-dilbert-cartoon) > > Seriously, disk storage is getting less expensive all the time, you can > already buy a 250GB SATA drive for $70. Also, compression doesn't help > once you store already compressed data such as jpeg images, mp3 files, > or mpeg2/4/divx video files. Not only are the savings non-existant, but > we still end up with the disadvantage that corruption propagates to > multiple files because of the compression in the file system. I believe that compression in REISER4 is just a wrapper for gzip or lzo. So each file is compressed and your "corruption propagation" is just in your imagination. > And if it doesn't propagate across multiple files, the compression can't > be all that good It can be as good as gzipping every file, ie, pretty damn good. > since it can't benefit as much from similarity between > files. So if that is the case and you really want to save diskspace you > almost have to look at read-only compressed filesystems such as cramfs, > squashfs, zisofs, cloop and various other variants in combination with > a unionfs overlay to get read/write functionality. > > But in the end everything is a tradeoff. You can save diskspace, but > increase the cost of corruption. You deliberately ignored the fact that bad blocks are NOT dealt with by the filesystem,... but by the operating system. Like I said: If your filesystem is writing to bad blocks, then throw away your operating system. > Use a better compression algorithm, but > that uses more CPU or which is visible in performance of the > application. Not necessarily, just look at the bonnie++ tests. The tests show that using compression (with todays hardware) leads to read speeds that are some FOUR times the physical disk read rate,... but you ignore this as you have some strange anti-REISER religion. Think about it,... read speeds that are some FOUR times the physical disk read rate,... impossible without the use of compression (or something similar). > This can be offset by caching more, and being lazier with > writebacks, but that hurts on-disk consistency, creates more memory > pressure (more swapout/paging) and generally slows down other > applications that aren't actually accessing the disk. Having a fast > multi-core cpu and lots of memory helps a lot, but at some point what > tradeoff did we just make, we saved a couple of dollars not having to > buy a larger disk, but we spend considerably more on the more > expensive cpu and memory. > Wow, consider me STILL totally impressed by your AMAZING BIAS. Would you like to tell me why you are SO BIASED against REISER4? John. -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - The professional email service ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel 2007-04-08 21:50 ` johnrobertbanks (?) @ 2007-04-08 22:58 ` Richard Knutsson 2007-04-09 5:14 ` johnrobertbanks -1 siblings, 1 reply; 64+ messages in thread From: Richard Knutsson @ 2007-04-08 22:58 UTC (permalink / raw) To: johnrobertbanks Cc: Christer Weinigel, linux-kernel, linux-fsdevel, reiserfs-list Wow, I'm impressed. Think you got the record on how many mails you referenced to in a reply... But dude, please calm down, the caps-lock is not the answer. You have got some rude answers and you have called them back on it + you have repeated the same statement several times, that is not the best way of convincing people. I believe you picked up the "anti-Reiser religion"-phrase from previous rant-wars (otherwise, why does that "religion"-phrase always come up, and (almost) only when dealing with Reiser-fs), and yes, there has been some clashes caused by both sides, so please be careful when dealing with this matter. Would you be willing to benchmark Reiser4 with some compressed binary-blob and show the time as well as the CPU-usage? And document how it is set up so it can be reproduced. After all, Windows is suppose to be more stable, maintained and cost-efficient then Linux, but they don't tell us how ;) >> since it can't benefit as much from similarity between >> files. So if that is the case and you really want to save diskspace you >> almost have to look at read-only compressed filesystems such as cramfs, >> squashfs, zisofs, cloop and various other variants in combination with >> a unionfs overlay to get read/write functionality. >> >> But in the end everything is a tradeoff. You can save diskspace, but >> increase the cost of corruption. >> > > You deliberately ignored the fact that bad blocks are NOT dealt with by > the filesystem,... but by the operating system. Like I said: If your > filesystem is writing to bad blocks, then throw away your operating > system. > I may have missed something, but if my room-mate took my harddrive, screwed it open, wrote a love-letter on the disk with a pencil and then returned it (ok, there may be some more plausible reasons for corruption), is the OS really suppose to handle it? Yes, it should not assign any new data to those blocks but should it not also fall into the file-systems domain to be able to restore some/all data? Just my 2c to the pond Richard Knutsson ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel 2007-04-08 22:58 ` Richard Knutsson @ 2007-04-09 5:14 ` johnrobertbanks 2007-04-09 7:07 ` Willy Tarreau 2007-04-09 16:10 ` Richard Knutsson 0 siblings, 2 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-09 5:14 UTC (permalink / raw) To: Richard Knutsson Cc: Christer Weinigel, linux-kernel, linux-fsdevel, reiserfs-list On Mon, 09 Apr 2007 00:58:53 +0200, "Richard Knutsson" <ricknu-0@student.ltu.se> said: > Wow, I'm impressed. Think you got the record on how many mails you > referenced to in a reply... TWO actually. I guess you are easily impressed. A simple cut and paste error. > You have got some rude answers and you have called them back on it Yeah, I (fairly closely) mimicked their behavior to make a point. > + you have repeated the same statement several times, that is > not the best way of convincing people. I know you DON'T believe that, as you are about the tenth person to repeat that "repeating stuff has no effect." > I believe you picked up the "anti-Reiser religion"-phrase from previous > rant-wars (otherwise, why does that "religion"-phrase always come up, > and (almost) only when dealing with Reiser-fs), and yes, there has been > some clashes caused by both sides, so please be careful when dealing > with this matter. NO. You people simply come across as zealots who work together, against Reiser4. Hence the term "anti-Reiser religion." > Would you be willing to benchmark Reiser4 with some compressed > binary-blob and show the time as well as the CPU-usage? I might be. I don't really know how to set it all up. Perhaps if you guided me through it. > > > > You deliberately ignored the fact that bad blocks are NOT dealt with by > > the filesystem,... but by the operating system. Like I said: If your > > filesystem is writing to bad blocks, then throw away your operating > > system. > > > I may have missed something, but if my room-mate took my harddrive, > screwed it open, wrote a love-letter on the disk with a pencil and then > returned it (ok, there may be some more plausible reasons for > corruption), is the OS really suppose to handle it? Yeah, I can't see how the OS could read the love-letter either. But one thing is for sure. The FS ain't responsible for reading it. > Yes, it should not > assign any new data to those blocks but should it not also fall into the > file-systems domain to be able to restore some/all data? It's a tough ask of any FS. Microsoft's filesystem checker totally roasted all my data on an XP-box last night. I had used ntfsresize to reduce the partition size and had a power outage. Later, Windows booted, ran the filesystem checker, seemed OK. Next time I boot, all I get is Input/Output error. > > Just my 2c to the pond > Richard Knutsson > Addin my 2c John. -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - A no graphics, no pop-ups email service ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel 2007-04-09 5:14 ` johnrobertbanks @ 2007-04-09 7:07 ` Willy Tarreau 2007-04-09 16:10 ` Richard Knutsson 1 sibling, 0 replies; 64+ messages in thread From: Willy Tarreau @ 2007-04-09 7:07 UTC (permalink / raw) To: johnrobertbanks Cc: Richard Knutsson, Christer Weinigel, linux-kernel, linux-fsdevel, reiserfs-list On Sun, Apr 08, 2007 at 10:14:18PM -0700, johnrobertbanks@fastmail.fm wrote: > NO. You people simply come across as zealots who work together, against > Reiser4. Poor guy ! People are not against Reiser4, they are against the stupid and irritating person who pollutes the lists always sending the same results without any comment because he doesn't even understand the results. Just like the kid on the beach "Look Ma, I found a soft shell!". "Leave it overthere, it's a jellyfish !". "I don't know what a jellyfish is, I will take all those soft shells with me". You keep saying people do not want to read you, but there is nothing to read. In fact, you hope that people will comment on your results so that you will finally understand them, but people keep saying there is nothing to read there. When will you stop annoying people with your noisy toys ? Oh, and please send your conspiracy claims somewhere else. Willy ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel 2007-04-09 5:14 ` johnrobertbanks 2007-04-09 7:07 ` Willy Tarreau @ 2007-04-09 16:10 ` Richard Knutsson 1 sibling, 0 replies; 64+ messages in thread From: Richard Knutsson @ 2007-04-09 16:10 UTC (permalink / raw) To: johnrobertbanks Cc: Christer Weinigel, linux-kernel, linux-fsdevel, reiserfs-list johnrobertbanks@fastmail.fm wrote: > On Mon, 09 Apr 2007 00:58:53 +0200, "Richard Knutsson" > <ricknu-0@student.ltu.se> said: > >> Wow, I'm impressed. Think you got the record on how many mails you >> referenced to in a reply... >> > > TWO actually. I guess you are easily impressed. > Oh, took it to be from 5-6 sources... >> + you have repeated the same statement several times, that is >> not the best way of convincing people. >> > > I know you DON'T believe that, as you are about the tenth person to > repeat that "repeating stuff has no effect." > Why should we change our response to the same error? The only solution to this loop is when people stops answering you and you "lose". >> I believe you picked up the "anti-Reiser religion"-phrase from previous >> rant-wars (otherwise, why does that "religion"-phrase always come up, >> and (almost) only when dealing with Reiser-fs), and yes, there has been >> some clashes caused by both sides, so please be careful when dealing >> with this matter. >> > > NO. You people simply come across as zealots who work together, against > Reiser4. > > Hence the term "anti-Reiser religion." > Please, don't address someone you meet for the first time as "you people"! Yes, we do _work_ together, it is a community and as a community you have to follow the social rules agreed upon. Without all those pro-Reiser peoples who knew how to work with the rest, there would not be a ResierFS/Reiser3 in the kernel. Unfortunately, Hans is in this case his own worst enemy and has ruffed quite a few feathers over the time. I don't think you would like someone who tells you "if you do it my way, then you are doing it wrong"... But personally, even if I find Hans a bit too strong-headed, he got some interesting design-ideas and the Reiser-filesystem is something I think many find interesting as a concept but not yet trust-worthy for their own machines. >> Would you be willing to benchmark Reiser4 with some compressed >> binary-blob and show the time as well as the CPU-usage? >> > > I might be. I don't really know how to set it all up. > > Perhaps if you guided me through it. > Am not sure how much help I would be but from the responses to your benchmark-list, there seems to be many who could help you. But first I think you should set up a system to test on, and then after some tests and made the result public, there will (most likely) be people who ask you to test it in some specific way. >> I may have missed something, but if my room-mate took my harddrive, >> screwed it open, wrote a love-letter on the disk with a pencil and then >> returned it (ok, there may be some more plausible reasons for >> corruption), is the OS really suppose to handle it? >> > > Yeah, I can't see how the OS could read the love-letter either. > > But one thing is for sure. The FS ain't responsible for reading it. > And no-one has asked the file-system to _read_ the disk, but to be designed to help restore the file-structure. This I have found to be the main-point people complains about. It is like arguing against air-bags in a car. Of course the car should not be responsible for preventing accidents, but they are designed so _if_ it happens, you should not be totally screwed. >> Yes, it should not >> assign any new data to those blocks but should it not also fall into the >> file-systems domain to be able to restore some/all data? >> > > It's a tough ask of any FS. > > Microsoft's filesystem checker totally roasted all my data on an XP-box > last night. > Sorry to hear that, but two wrongs does not make it right. Richard Knutsson ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel 2007-04-08 21:50 ` johnrobertbanks (?) (?) @ 2007-04-09 1:24 ` Christer Weinigel 2007-04-09 3:38 ` David H. Lynch Jr -1 siblings, 1 reply; 64+ messages in thread From: Christer Weinigel @ 2007-04-09 1:24 UTC (permalink / raw) To: johnrobertbanks; +Cc: Christer Weinigel, linux-fsdevel, reiserfs-list [linux-kernel trimmed from the Cc list] johnrobertbanks@fastmail.fm writes: > [lots of drivel with lots of capital letters elided] > [a totally confused mess of responses to at least three different mails] Wow. You are absolutely amazing. But, oh, well. > By the way: Did I thank you "delightful" people for the "pleasant" > welcome to the linux-kernel mailing list? You are so welcome. *grin* > > So the two bonnie benchmarks with lzo and gzip are > > totally meaningless for any real life usages. > > YOU (yes, the one with no experience and next to NO knowledge on the > subject) claim that because bonnie++ writes files that are mostly zeros, > the results are meaningless. It should be mentioned that bonnie++ writes > files that are mostly zero for all the filesystems compared. So the > results are meaningful, contrary to would you claim. Ok, lets take this really slowly so that you may understand. Compression in the file system can be useful, there is no doubt about that, but you have to be aware of the tradeoffs. First of all, very few people have any use for storing files consisting of just zeroes. Trying to make any decision based on a file systems ability to compress zeroes is just plain dumb. Bonnie++ assumes that the data it writes will end up being written to disk and not be compressed. Right now it allocates a buffer which is filled with zeroes and half a dozen bytes that the beginning of the buffer are filled in with some random data. So to make the bonnie runs have any meaning on any compressed file systems you really want to be able to choose what data it writes to disk. If you modified bonnie to do multiple test runs, one with zeroes, another one with some easily compressed data such as a syslog, some not so easily compressed data such as the contents of /bin/bash and some uncompressible data such as from /dev/urandom that would be a lot better benchmark. Then you have to be aware of the cost of that compression. First of all, it is going to use some CPU, so measuring the CPU load during the benchmark is a good start. Another thing with compression is that it requires you to keep both some compressed and the uncompressed data in RAM at the same time, so the memory pressure will increase. This is harder to measure and quantify. Finally, since the CPU has to get involved at compressing and decompressing the data doing this will pull both the uncompressed and compressed data into the CPU cache and may evict data from the cache that other processes would have found useful. This cache pollution is even harder to measure. And none of these costs make any difference for benchmarks run on a lightly loaded system, but may make a difference in real life on any system that tries to do something useful at the same time. Then you have to consider your use cases. As I said in my previous mail, for my only space constrained disk, I store a lot of large flac encoded CD images. That data is basically uncompressible, so compression buys me nothing, it just costs me a lot of extra CPU to try to compress uncompressible data. In addition, each CD image is a quite large too, about 300MByte for a full size album, so whatever savings I can get though the tail merging that Reiser4 (and Reiser3) does is marginal for my use case. Other use cases might have a lot to gain from compression. > ALSO YOU IGNORE examples offered by others, on lkml, which contradict > your assertion: FOR EXAMPLE: > > > I see the same thing with my nightly scripts that do syslog > > analysis, last year I trimmed 2 hours from the nightly run by > > processing compressed files instead of uncompressed ones (after I > > did this I configured it to compress the files as they are rolled, > > but rolling every 5 min the compression takes <20 seconds, so the > > compression is < 30 min) > > David has said that compressing the logs takes > > 24 x 12 x 20 secs = 5,760 secs = 1.6 hours of CPU time (over the day) > > but he saves 2 hours of CPU time on the daily syslog analysis. > > For a total (minimum) saving of 24 minutes. So lets look at the syslog case then. First of all, lets compress my syslog with gzip: gzip -c /var/log/messages >whole.gz du -h /var/log/messages whole.gz 532K messages 64K whole.gz Unfortunately, this compressed format isn't very efficient for some use cases, lets say that I want to read the last 10 lines of the syslog. On a normal uncompressed file system I can just seek to the end of the file, read the last block and get those 10 lines (or if the last block didn't have 10 lines, I can try the block before that). But with a compressed file, I have to uncompress the whole file and throw away 531 kBytes at the begining of the file to do that. So a file system that wants to give the user efficient random access to a files can't compress the whole file as done above. It has to make some tradeoffs to make random access practically usable. Most compressing file systems do that by splitting the file into fixed chunks which are compressed independently of each other. So lets simulate that by splitting the file into 4k chunks and compressing those separately and then combining them together: split -b 4096 /var/log/messages chunks gzip chunks* cat chunks*.gz >combined.gz du -h combined.gz 120K combined.gz The reason for the loss of compression is that the chunk based compression can't reuse knowledge across chunks. It's possible to mitigate this by increasing the chunks size: 84K combined-16368-chunk-size.gz 72K combined-65536-chunk-size.gz once again that has a downside, the bigger the chunks, the more data will be uncompressed unneccesarily when doing random accesses in the file. So the syslog example you are quoting above does not tell you how well reiser4 will do on that specific use case. A lot of the benefit in David's example comes from knowing that he wants to process the file as a whole and doesn't need random access. So having application specific knowledge and doing the compression outside of the file system is what gives him that gain. Of course, it may also be that the convenience of having transparent compression in the file system is more worth more than the 50% benefit in size from having to compress the syslog manually. That depends. > but he saves 2 hours of CPU time on the daily syslog analysis. And no, he spends 1.6 hours of CPU time (maybe, some IO wait is probaby included in that number) to save 2 hours of runtime (mostly IO wait I assume). So it seems that the disk is the bottle neck in his case. On a slightly different system CPU might be the bottle neck because the same machine has to do a lot of processing at the same time, so that it's better to skip the compression. Once again, it depends. What I'm trying to get at here is that yes, compression can be useful, but it is usese dependent, and it's impossible to catch all the nuances of all use cases in one single number, especially an extremely artificial number such as a bonnie++ run with files mostly consisting of zeroes. You can foam at the mouth and post the same meaningless, benchmark figures over and over again and yell even louder, but that still don't make them relevant. It's not reiser4 that is the problem, but the way you try to present reiser4 as the best thing since sliced bread. To misquote Scott Adams: I'm not anti-Reiser4, I'm anti-idiot. /Christer -- "Just how much can I get away with and still go to heaven?" Christer Weinigel <christer@weinigel.se> http://www.weinigel.se ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel 2007-04-09 1:24 ` Christer Weinigel @ 2007-04-09 3:38 ` David H. Lynch Jr 0 siblings, 0 replies; 64+ messages in thread From: David H. Lynch Jr @ 2007-04-09 3:38 UTC (permalink / raw) To: Christer Weinigel; +Cc: linux-fsdevel Christer Weinigel wrote: > > To misquote Scott Adams: I'm not anti-Reiser4, I'm anti-idiot. > > /Christer > So can we ignore the benchmarks issue entirely and deal with the issue of what might it actually take now that Hans is not pissing all over everyone to get Reiser4 submitted with some hope of being accepted ? Your analysis was excellent. It make the case that just is there will be a few cases where Reiser4 is absolutely compelling and a few cases where it makes no sense at all that there are alot of cases in the middle. I can keep coming up with examples where the attributes unique to Reiser4 are an excellent idea, and you can just as easily come up with cases where it is a bad idea. If the standard is that it must be all things to all people, then we need to quit letting any filesystems in. Let's say John takes your advice and tweaks bonnie or builds whatever benchmark test bed you want, and proves Reiser4 outperforms everything in all cases (highly unlikely) does that mean it should get it ? What about if it is 10% slower in most normal cases ? Is that a good enough reason for excluding it ? Is anyone ready to claim that there are no cases (beyond the extremely rare case of compressing zero's) where Reiser4 may prove to be the best choice ? Compression itself is almost a religious war subject. for some people it is ALWAYS a bad idea, to others it is ALWAYS a good idea. In the real world it varies. Application level compression is generally superior to filesystem compression - there is more knowledge about the data leading to better choice of compression algorithms. and fortunately Linux provides one of the best environments for incorporating application level compression. But generally is not the same as ALWAYS. While I raised a specific case I was aware of where my understanding was compressed data could result in a net overall gain in performance - network servers where compression decompression were handled at the client, several other instances have been raised. Depending on the speed of the CPU, size of memory, size and speed of cache, speed of disk, type of data, .... sometimes compressed data will not only save space but improve performance - and sometimes it will be costly to performance. That does not make it a bad idea. We have several scheduling algorithms, because one size does not fit all. Besides I doubt given the complexity of the issue that John or anyone else can put together a benchmark that I can't poke holes in. There are so many cases that there is no such thing as the general case. Even performance compressing zeros starts to almost look like a rational measure when you start to calculate all the permutations in the data compression matrix. -- Dave Lynch DLA Systems Software Development: Embedded Linux 717.627.3770 dhlii@dlasys.net http://www.dlasys.net fax: 1.253.369.9244 Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel 2007-04-08 21:50 ` johnrobertbanks ` (2 preceding siblings ...) (?) @ 2007-04-09 3:16 ` David H. Lynch Jr 2007-04-09 4:25 ` johnrobertbanks -1 siblings, 1 reply; 64+ messages in thread From: David H. Lynch Jr @ 2007-04-09 3:16 UTC (permalink / raw) To: johnrobertbanks, linux-fsdevel johnrobertbanks@fastmail.fm wrote: > Christer Weinigel: Until YOU, have actually used the REISER4 filesystem > yourself, I think YOU OWE IT to the people on the linux-kernel mailing > list, to, AS YOU SAY, shut the fuck up. > > Even reading up on the REISER4 filesystem would help. > > Applying a little intelligence would undoubtedly help too. > John; I would like to see Reiser4 get into the Kernel too. But angry rhetoric ... is not going to do it. If it was enough Hans did a much better job of insulting the naysayers without resulting to vulgarity, And it got him pretty much nowhere. It is my personal oppinion that the reason Reiser4 did not reach the kernel long ago was because Hans pissed all over way to many people. Making this about people will serve no one. We can all call each other assholes, go home feeling vindicated, but Reiser4 will be no closer to the kernel, and any potential that anyone might have benefited from that will just be pissed away. After that the statistics you are reporting are not TOTALLY meaningless. But they are also not compelling. It is not wise to be arguing benchmarks anyway. I hope that few on this list are not willing to conceed that Reiser4 is fast. Whether it is 10% slower than the best or 3 times as fast is probably not relevant to whether it gets included. All the worst case scenarious of Reiser4 performance and behavior should all still be sufficient to justify letting it in as Experimental. You can argue your benchmarks forever. Your benchmark has narrow applicability - pretty much every benchmark will. And all you do is provoke a firestorm of debate over whether compressing zeros means anything, whether compression is even a good idea, whether, ...... The answer to ALL of these questions will be "it depends" I have dealt with huge data sets that were mostly zero's, a file system that had a factor of 3 space savings and or performance benefit might have mattered alot there - but they are not the norm. I have dealt with instances where compression increased performance rather than decreased it. Every "feature" of Reiser4 is an asset in some environments and a liability in others. Whether it is suitable for most environments is nto going to be established by a benchmark. -- Dave Lynch DLA Systems Software Development: Embedded Linux 717.627.3770 dhlii@dlasys.net http://www.dlasys.net fax: 1.253.369.9244 Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel 2007-04-09 3:16 ` David H. Lynch Jr @ 2007-04-09 4:25 ` johnrobertbanks 0 siblings, 0 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-09 4:25 UTC (permalink / raw) To: David H. Lynch Jr, linux-fsdevel On Sun, 08 Apr 2007 23:16:18 -0400, "David H. Lynch Jr" <dhlii@comcast.net> said: > johnrobertbanks@fastmail.fm wrote: > > Christer Weinigel: Until YOU, have actually used the REISER4 filesystem > > yourself, I think YOU OWE IT to the people on the linux-kernel mailing > > list, to, AS YOU SAY, shut the fuck up. > > > > Even reading up on the REISER4 filesystem would help. > > > > Applying a little intelligence would undoubtedly help too. > > > John; > I would like to see Reiser4 get into the Kernel too. But angry > rhetoric ... is not going to do it. > I WAS JUST POINTING OUT THE VERY ANGRY RHETORIC COMING FROM CERTAIN FOLK ON THE MAILING LIST. I think it really needs to be pointed out as some of those here are extremely rude, bordering on unpleasant. I only use angry rhetoric to make a point. HAS THAT POINT BEEN MADE? -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - Same, same, but different - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-08 4:32 ` Christer Weinigel 2007-04-08 21:50 ` johnrobertbanks @ 2007-04-09 18:35 ` Nate Diller 1 sibling, 0 replies; 64+ messages in thread From: Nate Diller @ 2007-04-09 18:35 UTC (permalink / raw) To: Christer Weinigel Cc: johnrobertbanks, Lennart Sorensen, H. Peter Anvin, Ignatich, reiserfs-list, linux-kernel, linux-fsdevel On 08 Apr 2007 06:32:26 +0200, Christer Weinigel <christer@weinigel.se> wrote: > johnrobertbanks@fastmail.fm writes: > > > Lennart. Tell me again that these results from > > > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > > > are not of interest to you. I still don't understand why you have your > > head in the sand. > > Oh, for fucks sake, stop sounding like a broken record. You have > repeated the same totally meaningless statistics more times than I > care to count. Please shut the fuck up. wow, it's really amazing how reiser4 can still inspire flamewars so easily when Hans isn't even around to antagonize people and escalate things > As you discovered yourself (even though you seem to fail to understand > the significance of your discovery), bonnie writes files that consist > of mostly zeroes. If your normal use cases consist of creating a > bunch of files containing zeroes, reiser4 with compression will do > great. Just lovely. Except that nobody sane would store a lot of > files containing zeroes, except an an excercize in mental > masturbation. So the two bonnie benchmarks with lzo and gzip are > totally meaningless for any real life usages. yeah, i sure wish Grev was still around running the benchmarks and regression testing, cause I thought she came up with a good, QA oriented mix of real benchmarks. aside from a number of streaming video benchmarks i did, those were the only results i actually trusted to compare reiser4 with other systems. I know Ted doesn't like the Mongo suite, cause it focuses on small files and shows the common weakness of block-aligned storage ... personally i thought it was great for its primary purpose, making sure reiser4 was optimized for its target workload. i also recall that the distribution of small files to large ones in mongo was pulled from some paper out of CMU, but i can't find the reference to that study right now. > As for the amount of disk needed to store three kernel trees, the > figures you quote show that Reiser4 does tail combining where the tail > of multiple files are stored in one disk block. A nice trick that > seems save you about 15% disk space compared to ext3. Now you have to > realise what that means, it means that if the disk block containing > those tails (or any metadata pointing at that block) gets corrupted, > instead of just losing one disk block for one file, you will have lost > the tail for all the files sharing that disk block. Depending on your > personal prioritites, saving 15% of the space may be worth the risk to > you, or maybe not. Personally, for the only disk I'm short on space > on, I mostly store flac encoded images of my CD collection, and saving > 2kByte out of every 300MByte disk simply doesn't make any difference, > and I much prefer a stable file system that I can trust not to lose my > data. You might make different choices. well, it turns out that reiser4 does things a little differently, since tail packing has bad performance effects (i always turn it off on my reiserfs partitions). Reiser4 guarantees a file will be stored contiguously if it is below a certain size (20K?), and instead stores the whole file unaligned, so that many files can be packed together without slack space. this gives the best of both worlds performance-wise, at the expense of some complicated flush code to pack everything together in the tree before it gets written. that combined with the fine-grained locking scheme (per-node -- reiserfs just has a global lock) is the primary reason the code is so convoluted ... not poor coding. > The same goes for just about every feature that you tout, it has its > advantages, and it has its disadvantages. Doing compression on data > is great if the data you store is compressible, and sucks if it isn't. > Doing compression on each disk block and then packing multiple > compressed blocks into each physical disk block will probably save > some space if the data is compressible, but at the same time it means > that you will spend a lot of CPU (and cache footprint) compressing and > uncompressing that data. On a single user system where the CPU is > mostly idle it might not make much of a difference, on a heavily > loaded multiuser system it might do. my understanding of the code is that it uses a heuristic to decide if a file is already compressed, so that the system doesn't waste time on them and simply writes them out directly. there may also be a way to turn it off for certain classes of files, this would be most useful for executables and the like that are frequently mmap()ed and we care more about page-alignment than read bandwidth or data density. edward? > Logs can be compressed quite well using a block based compression > scheme, but the logs can be compressed even better by doing > compression on the whole file with gzip. So what's the best choice, > to do transparent compression on the fly giving ok compression or > teaching the userspace tools to do compression of old logs and get > really good compression? Or maybe disk space really isn't that > important anyway and the best thing is to just leave the logs > uncompressed. i guess the idea with reiser4's compression (encryption and compression, actually) is that you can get the feature for files you care about without having to use *extra* CPU time, it only does the work at flush time so that you can take advantage of cache effects for files that see lots of modifications. ATM i doubt this works well though, cause you'd have to manually increase dirty_background_ratio to keep things from continually flushing and using background CPU NATE ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-07 19:17 ` Lennart Sorensen 2007-04-08 0:44 ` johnrobertbanks @ 2007-04-08 4:06 ` David H. Lynch Jr 2007-04-08 9:58 ` Jeff Garzik 1 sibling, 1 reply; 64+ messages in thread From: David H. Lynch Jr @ 2007-04-08 4:06 UTC (permalink / raw) To: reiserfs-list, linux-fsdevel Lennart Sorensen wrote: > Nope, if they are accurate and they have something to do with your > particular usage and applications, then they are relevant. But it > requires both to make them relevant. Although it may be possible for a > benchmark to be relevant even if not particularly accurate. I do not care about the benchmarks or the compression. They are pointless anyway, Reiser4 is not and can not be all things to all people. I do not care how many angels can dance on the head of a pin. Whether it is most suitable for broad use or only narrow cases remains to seen. Hans Reiser is now in jail awaiting trial for the murder of his wife - I doubt he is going to be posting and stirring things up anytime soon Those who need to assurance of their moral superiority can take that as proof if they need. I do care about getting Reiser4 into the kernel so that it can actually get a real test, and frankly do not see any compelling reason that should not happen. Their may be better filesystems currently in the kernel - there certainly are crappier ones. I have used ReiserFs extensively, and on the whole been extremely happy with it. I have been slightly singed - a long time ago, but it is certainly not the only file system that has caused me grief. I can not think of an existing Linux (or other) filesystem that is the be all or end all. I have no time for either the zealots that think Reiser4 is the next best thing to sliced bread nor those who are going to find some case to demonstrate it is total crap - no matter what. Most people are wise not to use Reiser4 - or any other new filesystem until it has been rigorously tested. I have no bone to pick with that. I will try it once it is available in standard kernel sources, but I am juggling enough patches as it is, and I am not the patch wizard that some of the rest of you are. If I get burned - so be it. It won't be the first time. This game has gone on sufficiently long that Reiser4 has zero hope of becoming more than a footnote used primarily by oddballs and misfits. But there is still some hope that a slightly wider audience might result in numerous positive benefits to other filesystems. I have read many of the critiques - but even if they are valid, I thought Linux was a place where competing ideas were tested in use not rhetoric. -- Dave Lynch DLA Systems Software Development: Embedded Linux 717.627.3770 dhlii@dlasys.net http://www.dlasys.net fax: 1.253.369.9244 Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-08 4:06 ` David H. Lynch Jr @ 2007-04-08 9:58 ` Jeff Garzik 2007-04-09 2:52 ` David H. Lynch Jr 0 siblings, 1 reply; 64+ messages in thread From: Jeff Garzik @ 2007-04-08 9:58 UTC (permalink / raw) To: David H. Lynch Jr; +Cc: reiserfs-list, linux-fsdevel David H. Lynch Jr wrote: > I do care about getting Reiser4 into the kernel so that it can > actually get a real test, > and frankly do not see any compelling reason that should not happen. If the compelling reason is that it needs a test, I'd say its not ready. Jeff ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-08 9:58 ` Jeff Garzik @ 2007-04-09 2:52 ` David H. Lynch Jr 2007-04-09 3:14 ` Jeff Garzik 0 siblings, 1 reply; 64+ messages in thread From: David H. Lynch Jr @ 2007-04-09 2:52 UTC (permalink / raw) To: linux-fsdevel Jeff Garzik wrote: > > If the compelling reason is that it needs a test, I'd say its not ready. Can you please elaborate ? I am not sure I understand what you are arguing ? Despite his substantially less than polite rhetoric, I have read Hans's post from months if not years ago. Aside from the pissing contests - which where not entirely one sided, I actually beleive that Hans made a reasonable case that Reiser4 had gone about as far as it could reasonably go with regard to testing, robustness, ... without the broader base of use that even an experimental filesystem in distribution tree would get. I for one would atleast play with it if it were in the distribution tree. As far as I could tell pretty much everything else that was demanded Hans eventually caved and provided - albeit with much pissing and moaning, and holy than thou rhetoric. The argument that anything that needs testing can't get into the distribution tree's is specious. There is alot of poorly tested crap in the distribution trees. But separately, there is the issue of scale. Namesys claims that they have no currently know bugs, faults ... - with their base of internal and external users. I would fully expect new failures to crop up with any filesystem, driver, ... moving up an order of magnitude in users. Are you going to subject all filesystems and drivers to the same high standards you are placing on Reiser4 ? If so then we need to strip the distribution tree now. I am not looking to defend Hans - he is likely to be in jail and no longer a factor for a long time. Nor am I looking to make or support claims for Reiser4. But I am asking - why we can not get past the bad blood, rhetoric, and zealotry -which to my eyes has not been all one sided. I am NOT looking for a technical explanation of all the relative merits and demerits of Reiser4. I do not care for arguments about whether it compresses 0's well, or that tail combining is a bad thing. They may have merit, but there is not a filesystem that is going to be all things to all people. Whether Reiser4 is a small niche filesystem or a significant general use one, is a decision that should be reached by its performance in practice, not it rhetoric. Regardless, even as a niche filesystem, I beleive at this point it merits inclusion. -- Dave Lynch DLA Systems Software Development: Embedded Linux 717.627.3770 dhlii@dlasys.net http://www.dlasys.net fax: 1.253.369.9244 Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-09 2:52 ` David H. Lynch Jr @ 2007-04-09 3:14 ` Jeff Garzik 2007-04-09 4:40 ` David H. Lynch Jr 0 siblings, 1 reply; 64+ messages in thread From: Jeff Garzik @ 2007-04-09 3:14 UTC (permalink / raw) To: David H. Lynch Jr; +Cc: linux-fsdevel David H. Lynch Jr wrote: > Jeff Garzik wrote: >> If the compelling reason is that it needs a test, I'd say its not ready. > > Can you please elaborate ? I am not sure I understand what you are > arguing ? > > Despite his substantially less than polite rhetoric, I have read > Hans's post from months if not years ago. > Aside from the pissing contests - which where not entirely one > sided, I actually beleive that Hans made a reasonable case > that Reiser4 had gone about as far as it could reasonably go with > regard to testing, robustness, ... without the broader base of > use that even an experimental filesystem in distribution tree would get. > > I for one would atleast play with it if it were in the distribution > tree. > As far as I could tell pretty much everything else that was demanded > Hans eventually caved and provided - albeit with much pissing and moaning, > and holy than thou rhetoric. > > The argument that anything that needs testing can't get into the > distribution tree's is specious. There is alot of poorly tested crap in > the distribution trees. I'm arguing against circular logic: the claim that one cannot determine reiser4's true usefulness unless its in the tree. The better method is to get a distro to add reiser4, _then_ if it proves worthy add it to the kernel tree. Not the other way around. Jeff ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-09 3:14 ` Jeff Garzik @ 2007-04-09 4:40 ` David H. Lynch Jr 2007-04-09 4:58 ` Jeff Garzik 0 siblings, 1 reply; 64+ messages in thread From: David H. Lynch Jr @ 2007-04-09 4:40 UTC (permalink / raw) To: Jeff Garzik; +Cc: linux-fsdevel Jeff Garzik wrote: > David H. Lynch Jr wrote: > > I'm arguing against circular logic: the claim that one cannot > determine reiser4's true usefulness unless its in the tree. > > The better method is to get a distro to add reiser4, _then_ if it > proves worthy add it to the kernel tree. > > Not the other way around. And is that how other filesystems made it into the tree ? I am actually trying to be rational about this. But if my logic is circular then the above is even more so. I am aware of distributions that would incorporate Reiser4 if it was in the tree, but not really aware of much that works the other way around. Besides after eliminating say the top 4 distributions, how is it that getting included as a burried alternate in an obscure distribution is going to change the total number of existing Reiser4 users to a meaningful extent. It would be my guess that Reiser4 already has more users than any filesystem has ever had prior to inclusion in the kernel. Aside from a few instances where distributors are also contributors, and a few instances where specific drivers such as gaming video drivers are in extremely high demand, how frequently do drivers go into distributions before they go into the tree ? Is that supposed to be the standard for all drivers ? I regularly see drivers with very little in the way of testing go straight nearly straight into the tree - without even getting tagged as experimental. And what is CONFIG_EXERIMENTAL supposed to be for ? All I am asking is that now that Hans is not pissing all over people that the Reiser4 Filesystem get exactly the same treatment that pretty much every other filesystem already in or slated for inclusion in the kernel has received. I am sure that their are others who may have a more accurate perception of history. But my recollection is that the only filesystem driver that went through a distribution prior to getting into the Kernel was ReiserFS. I am not even asking that it get accepted exactly asis - though it is my perception that despite Hans's pissing and moaning in the end he made most every change that was asked. Regardless, now that you don't have Hans to deal with anymore. Give Namesys an honest and reasonable set of requirements to get a decent review - the same shot any other filesystem would get. -- Dave Lynch DLA Systems Software Development: Embedded Linux 717.627.3770 dhlii@dlasys.net http://www.dlasys.net fax: 1.253.369.9244 Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: Reiser4. BEST FILESYSTEM EVER. 2007-04-09 4:40 ` David H. Lynch Jr @ 2007-04-09 4:58 ` Jeff Garzik 0 siblings, 0 replies; 64+ messages in thread From: Jeff Garzik @ 2007-04-09 4:58 UTC (permalink / raw) To: David H. Lynch Jr; +Cc: linux-fsdevel David H. Lynch Jr wrote: > Jeff Garzik wrote: >> David H. Lynch Jr wrote: >> >> I'm arguing against circular logic: the claim that one cannot >> determine reiser4's true usefulness unless its in the tree. >> >> The better method is to get a distro to add reiser4, _then_ if it >> proves worthy add it to the kernel tree. >> >> Not the other way around. > And is that how other filesystems made it into the tree ? In the case of most major filesystems, yes. Distros are a proving ground for new stuff, not the upstream kernel. > I regularly see drivers with very little in the way of testing go > straight nearly straight into the tree - without even getting tagged as > experimental. Hardware drivers are vastly different from filesystem drivers. Jeff ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: COMPILING AND CONFIGURING A NEW KERNEL. 2007-04-06 0:05 ` johnrobertbanks @ 2007-04-07 1:26 ` johnrobertbanks -1 siblings, 0 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-07 1:26 UTC (permalink / raw) To: johnrobertbanks, Ignatich, reiserfs-list, linux-kernel This is a reply to an email that I accidentally deleted. COMPILING AND CONFIGURING A NEW KERNEL. Download a recent kernel from http://www.kernel.org/ I will use the kernel linux-2.6.20.tar.bz2 You will have to change details of the following to suit your purposes. Save it in /usr/src/ # mv linux-2.6.20.tar.bz2 /usr/src/ Unzip the kernel package # tar -jxf linux-2.6.20.tar.bz2 Copy the original kernel configuration file (that came with your distro) to .config # cp /boot/config-2.6.20 /usr/src/linux-2.6.20/.config Look at the available kernel building options # make help Run oldconfig to update the original kernel config to a current config # make oldconfig Use menuconfig (or xconfig or gconfig) to make any further changes # make menuconfig YOU SHOULD compile all the drivers necessary to boot your system, into the kernel (ie, such drivers should not be built as modules). This way you will NOT need an initrd file. Use rpm-pkg to create a Red Hat RPM kernel package. # make rpm-pkg Went built, the RPM package is put in /usr/src/packages/RPMS/*your*architecture* # cd /usr/src/packages/RPMS/x86_64 Install the package (you may have to un-install previous installs) # rpm -i kernel-2.6.20-1.x86_64.rpm Use deb-pkg to create a Debian .deb kernel package. # make deb-pkg Went built, the .deb package is put in /usr/src/ # cd /usr/src/ Install the package (you may have to un-install previous installs) # dpkg --install linux-2.6.20_2.6.20_amd64.deb If you were unable to determine which drivers you need (to boot), then you will need an initrd file. To build it use the command # mkinitrd -o /boot/initrd-2.6.20 IF YOU ARE CUSTOMIZING YOUR KERNEL, YOU SHOULD PUT IN THE EFFORT TO BUILD A KERNEL THAT DOES NOT NEED AN INITRD FILE. ------------------------------------------------------ Now you need to configure your kernel. Using GRUB you need to change the menu.lst file. # emacs /boot/grub/menu.lst & The grub entry that you boot with will look something like: ###Don't change this comment - YaST2 identifier: Original name: linux### title SUSE LINUX 10.0 root (hd0,2) kernel /boot/vmlinuz root=/dev/hda3 resume=/dev/hda5 vga=0x317 video=vesafb:nomtrr splash=silent initrd /boot/initrd Leave the old boot entry so you can boot it if things go wrong with the new kernel. Cut and paste a copy of it above the old one. Then adjust the copy for the new kernel. ###Don't change this comment - YaST2 identifier: Original name: linux### title MY NEW KERNEL root (hd0,2) kernel /boot/linux-2.6.20 root=/dev/hda3 resume=/dev/hda5 vga=0x317 video=vesafb:nomtrr splash=silent Of course, you don't need a initrd entry as you have compiled in all the vital drivers,... right? If you could not determine the vital drivers and needed to build an initrd file, then you need an entry, like initrd /boot/initrd-2.6.20 ------------------------------------------------------ If your new kernel is destined to have the same name as the old one, you need to do something about it (unless you do not mind the old one being overwritten). Use your favorite text editor to change the top level Makefile # emacs /usr/src/linux-2.6.20/Makefile & change the line EXTRAVERSION to EXTRAVERSION = something This will change the name of the new kernel to linux-2.6.20-something Your /boot/grub/menu.lst entry will now look something like: ###Don't change this comment - YaST2 identifier: Original name: linux### title MY NEW KERNEL root (hd0,2) kernel /boot/linux-2.6.20-something root=/dev/hda3 resume=/dev/hda5 vga=0x317 video=vesafb:nomtrr splash=silent and perhaps an entry initrd /boot/initrd-2.6.20-something ------------------------------------------------------ Now reboot and choose the "MY NEW KERNEL" entry from the GRUB boot menu, and see how you went. ------------------------------------------------------ On Thu, 05 Apr 2007 17:05:21 -0700, johnrobertbanks@fastmail.fm said: > Hi Ignatich, > > After seeing the following benchmarks at > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > The Reiser4 benchmarks are so good, I have decided to try the Reiser4 > filesystem. > > .-------------------------. > | FILESYSTEM | TIME |DISK | > | TYPE |(secs)|USAGE| > .-------------------------. > |REISER4 lzo | 1938 | 278 | > |REISER4 gzip| 2295 | 213 | > |REISER4 | 3462 | 692 | > |EXT2 | 4092 | 816 | > |JFS | 4225 | 806 | > |EXT4 | 4408 | 816 | > |EXT3 | 4421 | 816 | > |XFS | 4625 | 779 | > |REISER3 | 6178 | 793 | > |FAT32 |12342 | 988 | > |NTFS-3g |10414 | 772 | > .-------------------------. > > Column one measures the time taken to complete the bonnie++ benchmarking > test (run with the parameters bonnie++ -n128:128k:0) > > Column two, Disk Usage: measures the amount of disk used to store 655MB > of raw data (which was 3 different copies of the Linux kernel sources). > > Anyway, I have patched the 2.6.20 kernel and have a partition formatted > with Reiser4. > > However, I am having trouble getting LILO or GRUB working (with > Reiser4). > > Could you guys who know all about this, help me, or point me to some > help. > > Thanks a lot, John. > > > On Fri, 06 Apr 2007 02:42:35 +0400, "Ignatich" <ignatich@gmail.com> > said: > > While trying to find the cause of problems with reiser4 in recent > > kernels I came across this. > > > > Incomplete write handling seem to be missing from reiser4_write_extent() > > thanks to reiser4-temp-fix.patch. Strangely, there is a patch by Edward > > Shishkin that should address that issue, but it is missing from -mm > > tree. Please check. > > > > Max > > > -- > > johnrobertbanks@fastmail.fm > > -- > http://www.fastmail.fm - And now for something completely different > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" > in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - mmm... Fastmail... ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: COMPILING AND CONFIGURING A NEW KERNEL. @ 2007-04-07 1:26 ` johnrobertbanks 0 siblings, 0 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-07 1:26 UTC (permalink / raw) To: johnrobertbanks, Ignatich, reiserfs-list, linux-kernel This is a reply to an email that I accidentally deleted. COMPILING AND CONFIGURING A NEW KERNEL. Download a recent kernel from http://www.kernel.org/ I will use the kernel linux-2.6.20.tar.bz2 You will have to change details of the following to suit your purposes. Save it in /usr/src/ # mv linux-2.6.20.tar.bz2 /usr/src/ Unzip the kernel package # tar -jxf linux-2.6.20.tar.bz2 Copy the original kernel configuration file (that came with your distro) to .config # cp /boot/config-2.6.20 /usr/src/linux-2.6.20/.config Look at the available kernel building options # make help Run oldconfig to update the original kernel config to a current config # make oldconfig Use menuconfig (or xconfig or gconfig) to make any further changes # make menuconfig YOU SHOULD compile all the drivers necessary to boot your system, into the kernel (ie, such drivers should not be built as modules). This way you will NOT need an initrd file. Use rpm-pkg to create a Red Hat RPM kernel package. # make rpm-pkg Went built, the RPM package is put in /usr/src/packages/RPMS/*your*architecture* # cd /usr/src/packages/RPMS/x86_64 Install the package (you may have to un-install previous installs) # rpm -i kernel-2.6.20-1.x86_64.rpm Use deb-pkg to create a Debian .deb kernel package. # make deb-pkg Went built, the .deb package is put in /usr/src/ # cd /usr/src/ Install the package (you may have to un-install previous installs) # dpkg --install linux-2.6.20_2.6.20_amd64.deb If you were unable to determine which drivers you need (to boot), then you will need an initrd file. To build it use the command # mkinitrd -o /boot/initrd-2.6.20 IF YOU ARE CUSTOMIZING YOUR KERNEL, YOU SHOULD PUT IN THE EFFORT TO BUILD A KERNEL THAT DOES NOT NEED AN INITRD FILE. ------------------------------------------------------ Now you need to configure your kernel. Using GRUB you need to change the menu.lst file. # emacs /boot/grub/menu.lst & The grub entry that you boot with will look something like: ###Don't change this comment - YaST2 identifier: Original name: linux### title SUSE LINUX 10.0 root (hd0,2) kernel /boot/vmlinuz root=/dev/hda3 resume=/dev/hda5 vga=0x317 video=vesafb:nomtrr splash=silent initrd /boot/initrd Leave the old boot entry so you can boot it if things go wrong with the new kernel. Cut and paste a copy of it above the old one. Then adjust the copy for the new kernel. ###Don't change this comment - YaST2 identifier: Original name: linux### title MY NEW KERNEL root (hd0,2) kernel /boot/linux-2.6.20 root=/dev/hda3 resume=/dev/hda5 vga=0x317 video=vesafb:nomtrr splash=silent Of course, you don't need a initrd entry as you have compiled in all the vital drivers,... right? If you could not determine the vital drivers and needed to build an initrd file, then you need an entry, like initrd /boot/initrd-2.6.20 ------------------------------------------------------ If your new kernel is destined to have the same name as the old one, you need to do something about it (unless you do not mind the old one being overwritten). Use your favorite text editor to change the top level Makefile # emacs /usr/src/linux-2.6.20/Makefile & change the line EXTRAVERSION to EXTRAVERSION = something This will change the name of the new kernel to linux-2.6.20-something Your /boot/grub/menu.lst entry will now look something like: ###Don't change this comment - YaST2 identifier: Original name: linux### title MY NEW KERNEL root (hd0,2) kernel /boot/linux-2.6.20-something root=/dev/hda3 resume=/dev/hda5 vga=0x317 video=vesafb:nomtrr splash=silent and perhaps an entry initrd /boot/initrd-2.6.20-something ------------------------------------------------------ Now reboot and choose the "MY NEW KERNEL" entry from the GRUB boot menu, and see how you went. ------------------------------------------------------ On Thu, 05 Apr 2007 17:05:21 -0700, johnrobertbanks@fastmail.fm said: > Hi Ignatich, > > After seeing the following benchmarks at > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm > > The Reiser4 benchmarks are so good, I have decided to try the Reiser4 > filesystem. > > .-------------------------. > | FILESYSTEM | TIME |DISK | > | TYPE |(secs)|USAGE| > .-------------------------. > |REISER4 lzo | 1938 | 278 | > |REISER4 gzip| 2295 | 213 | > |REISER4 | 3462 | 692 | > |EXT2 | 4092 | 816 | > |JFS | 4225 | 806 | > |EXT4 | 4408 | 816 | > |EXT3 | 4421 | 816 | > |XFS | 4625 | 779 | > |REISER3 | 6178 | 793 | > |FAT32 |12342 | 988 | > |NTFS-3g |10414 | 772 | > .-------------------------. > > Column one measures the time taken to complete the bonnie++ benchmarking > test (run with the parameters bonnie++ -n128:128k:0) > > Column two, Disk Usage: measures the amount of disk used to store 655MB > of raw data (which was 3 different copies of the Linux kernel sources). > > Anyway, I have patched the 2.6.20 kernel and have a partition formatted > with Reiser4. > > However, I am having trouble getting LILO or GRUB working (with > Reiser4). > > Could you guys who know all about this, help me, or point me to some > help. > > Thanks a lot, John. > > > On Fri, 06 Apr 2007 02:42:35 +0400, "Ignatich" <ignatich@gmail.com> > said: > > While trying to find the cause of problems with reiser4 in recent > > kernels I came across this. > > > > Incomplete write handling seem to be missing from reiser4_write_extent() > > thanks to reiser4-temp-fix.patch. Strangely, there is a patch by Edward > > Shishkin that should address that issue, but it is missing from -mm > > tree. Please check. > > > > Max > > > -- > > johnrobertbanks@fastmail.fm > > -- > http://www.fastmail.fm - And now for something completely different… > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" > in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - mmm... Fastmail... ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: COMPILING AND CONFIGURING A NEW KERNEL. 2007-04-07 1:26 ` johnrobertbanks (?) @ 2007-04-07 7:45 ` johnrobertbanks 2007-04-07 16:57 ` Valdis.Kletnieks -1 siblings, 1 reply; 64+ messages in thread From: johnrobertbanks @ 2007-04-07 7:45 UTC (permalink / raw) To: johnrobertbanks, Ignatich, reiserfs-list, linux-kernel, linux-fsdevel Just correcting some errors and typos. Wouldn't want you to say that the linux kernel mailing list gave you incorrect info. COMPILING AND CONFIGURING A NEW KERNEL. Download a recent kernel from http://www.kernel.org/ I will use the kernel linux-2.6.20.tar.bz2 You will have to change details of the following to suit your purposes. Save it in /usr/src/ # mv linux-2.6.20.tar.bz2 /usr/src/ Unzip the kernel package # tar -jxf linux-2.6.20.tar.bz2 Copy the original kernel configuration file (that came with your distro) to .config # cp /boot/config-2.6.13-15-default /usr/src/linux-2.6.20/.config Change to the new kernel source directory # cd /usr/src/linux-2.6.20/ Look at the available kernel building options # make help Run oldconfig to update the original kernel configuration to a current configuration # make oldconfig Use menuconfig (or xconfig or gconfig) to make any further changes # make menuconfig YOU SHOULD compile all the drivers necessary to boot your system, into the kernel (ie, such drivers should not be built as modules). This way you will NOT need an initrd file. Use rpm-pkg to create a Red Hat RPM kernel package. # make rpm-pkg When built, the RPM package is put in /usr/src/packages/RPMS/*your*architecture* # cd /usr/src/packages/RPMS/x86_64 Install the package (you may have to un-install previous installs) # rpm -i kernel-2.6.20-1.x86_64.rpm Use deb-pkg to create a Debian .deb kernel package. # make deb-pkg When built, the .deb package is put in /usr/src/ # cd /usr/src/ Install the package (you may have to un-install previous installs) # dpkg --install linux-2.6.20_2.6.20_amd64.deb If you were unable to determine which drivers you need (to boot), then you will need an initrd file. To build it use the command # mkinitrd -o /boot/initrd-2.6.20 IF YOU ARE CUSTOMIZING YOUR KERNEL, YOU SHOULD PUT IN THE EFFORT TO BUILD A KERNEL THAT DOES NOT NEED AN INITRD FILE. It is possible that deb-pkg and rpm-pkg take care of creating the initrd automatically. I have always compiled in the important drivers, so I do not know. Does any caring person here know the answer to this question? ------------------------------------------------------ Now you need to configure your kernel. Using GRUB you need to change the menu.lst file. # emacs /boot/grub/menu.lst & The grub entry that you presently boot with, will look something like: ###Don't change this comment - YaST2 identifier: Original name: linux### title SUSE LINUX 10.0 root (hd0,2) kernel /boot/vmlinuz-2.6.13-15-default root=/dev/hda3 resume=/dev/hda5 vga=0x317 video=vesafb:nomtrr splash=silent initrd /boot/initrd Do NOT delete the old boot entry, so you can boot it, if things go wrong with the new kernel. Cut a copy of it and paste it above the original. Then adjust the copy for the new kernel. ###Don't change this comment - YaST2 identifier: Original name: linux### title MY NEW KERNEL root (hd0,2) kernel /boot/linux-2.6.20 root=/dev/hda3 resume=/dev/hda5 vga=0x317 video=vesafb:nomtrr splash=silent Of course, you don't need a initrd entry as you have compiled in all the vital drivers,... right? If you could not determine the vital drivers and needed to build an initrd file, then you need an entry, like initrd /boot/initrd-2.6.20 ------------------------------------------------------ If your new kernel is destined to have the same name as the old one, you need to do something about it (unless you do not mind the old one being overwritten). Use your favorite text editor to change the top level Makefile # emacs /usr/src/linux-2.6.20/Makefile & change the line EXTRAVERSION to EXTRAVERSION = something This will change the name of the new kernel to linux-2.6.20-something Your /boot/grub/menu.lst entry will now look something like: ###Don't change this comment - YaST2 identifier: Original name: linux### title MY NEW KERNEL root (hd0,2) kernel /boot/linux-2.6.20-something root=/dev/hda3 resume=/dev/hda5 vga=0x317 video=vesafb:nomtrr splash=silent and perhaps an entry initrd /boot/initrd-2.6.20-something ------------------------------------------------------ Now reboot and choose the "MY NEW KERNEL" entry from the GRUB boot menu, and see how you went. -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - Send your email first class ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: COMPILING AND CONFIGURING A NEW KERNEL. 2007-04-07 7:45 ` johnrobertbanks @ 2007-04-07 16:57 ` Valdis.Kletnieks 2007-04-08 1:11 ` johnrobertbanks 0 siblings, 1 reply; 64+ messages in thread From: Valdis.Kletnieks @ 2007-04-07 16:57 UTC (permalink / raw) To: johnrobertbanks; +Cc: Ignatich, reiserfs-list, linux-kernel, linux-fsdevel [-- Attachment #1: Type: text/plain, Size: 778 bytes --] On Sat, 07 Apr 2007 00:45:32 PDT, johnrobertbanks@fastmail.fm said: > Use rpm-pkg to create a Red Hat RPM kernel package. > # make rpm-pkg > > When built, the RPM package is put in > /usr/src/packages/RPMS/*your*architecture* > > # cd /usr/src/packages/RPMS/x86_64 > > Install the package (you may have to un-install previous installs) > # rpm -i kernel-2.6.20-1.x86_64.rpm It is *highly* recommended that you change the kernel identifier at least slightly, so that you can install '2.6.20-1.local' without overlaying the vendor-supplied 2.6.20-1 kernel. Among other things, this lets you boot back to the equivalent code level in the vendor kernel, so you can figure out if it's your .config file that's broken, or if you hit a bug upggrading from 2.6.19-10 to 2.6.20-1. [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: COMPILING AND CONFIGURING A NEW KERNEL. 2007-04-07 16:57 ` Valdis.Kletnieks @ 2007-04-08 1:11 ` johnrobertbanks 0 siblings, 0 replies; 64+ messages in thread From: johnrobertbanks @ 2007-04-08 1:11 UTC (permalink / raw) To: Valdis.Kletnieks; +Cc: Ignatich, reiserfs-list, linux-kernel, linux-fsdevel > It is *highly* recommended that you change the kernel identifier at > least slightly, so that you can install '2.6.20-1.local' without > overlaying > the vendor-supplied 2.6.20-1 kernel. Among other things, this lets you > boot back to the equivalent code level in the vendor kernel, > so you can figure > out if it's your .config file that's broken, or if you hit a bug > upggrading from 2.6.19-10 to 2.6.20-1. I agree. I think your advice is *highly* recommended. I had this problem once after forcing an upgrade, which removed the working kernel. I just booted the kernel and stuff, from another partition. John. -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - The way an email service should be ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: COMPILING AND CONFIGURING A NEW KERNEL. 2007-04-07 1:26 ` johnrobertbanks (?) (?) @ 2007-04-07 16:42 ` Valdis.Kletnieks 2007-04-08 1:02 ` johnrobertbanks -1 siblings, 1 reply; 64+ messages in thread From: Valdis.Kletnieks @ 2007-04-07 16:42 UTC (permalink / raw) To: johnrobertbanks; +Cc: Ignatich, reiserfs-list, linux-kernel [-- Attachment #1: Type: text/plain, Size: 589 bytes --] On Fri, 06 Apr 2007 18:26:45 PDT, johnrobertbanks@fastmail.fm said: > > YOU SHOULD compile all the drivers necessary to boot your system, into > the kernel (ie, such drivers should not be built as modules). > > This way you will NOT need an initrd file. It is quite possible to build a kernel that has all the drivers built-in, but still require an initrd file. For instance, if you have a recent RedHat or Fedora system, '/' may very well be on an LVM partition, which means you need an initrd to do a 'lvm varyonvg' before mounting your real root filesystem will work.... [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: COMPILING AND CONFIGURING A NEW KERNEL. 2007-04-07 16:42 ` Valdis.Kletnieks @ 2007-04-08 1:02 ` johnrobertbanks 2007-04-08 1:42 ` Lennart Sorensen 0 siblings, 1 reply; 64+ messages in thread From: johnrobertbanks @ 2007-04-08 1:02 UTC (permalink / raw) To: Valdis.Kletnieks; +Cc: Ignatich, reiserfs-list, linux-kernel > It is quite possible to build a kernel that has all the drivers built-in, > but still require an initrd file. For instance, if you have a recent > RedHat or Fedora system, '/' may very well be on an LVM partition, which > means you need an initrd to do a 'lvm varyonvg' before mounting your real > root filesystem will work.... Thanks Valdis, Thats interesting, I didn't know that. Do you know if deb-pkg and rpm-pkg take care of creating the initrd automatically. I seriously doubt they do. Actually, I guess I only need to compile another kernel to find out. John. -- johnrobertbanks@fastmail.fm -- http://www.fastmail.fm - Choose from over 50 domains or use your own ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: COMPILING AND CONFIGURING A NEW KERNEL. 2007-04-08 1:02 ` johnrobertbanks @ 2007-04-08 1:42 ` Lennart Sorensen 0 siblings, 0 replies; 64+ messages in thread From: Lennart Sorensen @ 2007-04-08 1:42 UTC (permalink / raw) To: johnrobertbanks; +Cc: Valdis.Kletnieks, Ignatich, reiserfs-list, linux-kernel On Sat, Apr 07, 2007 at 06:02:37PM -0700, johnrobertbanks@fastmail.fm wrote: > Thats interesting, I didn't know that. > > Do you know if deb-pkg and rpm-pkg take care of creating the initrd > automatically. > > I seriously doubt they do. > > Actually, I guess I only need to compile another kernel to find out. Certainly on debian make-kpkg actually does have options for initrd (--initrd) and will call mkinitrd/update-initramfs/yaird at package install time to generate an initrd for the kernel to deal with raid, lvm, etc. It works quite well in general. I would be surprised if redhat didn't do something similar. -- Len Sorensen ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: REISER4: fix for reiser4_write_extent 2007-04-05 22:42 REISER4: fix for reiser4_write_extent Ignatich @ 2007-04-07 12:51 ` Laurent Riffard 2007-04-07 12:51 ` Laurent Riffard 1 sibling, 0 replies; 64+ messages in thread From: Laurent Riffard @ 2007-04-07 12:51 UTC (permalink / raw) To: Ignatich; +Cc: reiserfs-list, linux-kernel, Edward Shishkin Le 06.04.2007 00:42, Ignatich a écrit : > While trying to find the cause of problems with reiser4 in recent > kernels I came across this. > > Incomplete write handling seem to be missing from reiser4_write_extent() > thanks to reiser4-temp-fix.patch. Strangely, there is a patch by Edward > Shishkin that should address that issue, but it is missing from -mm > tree. Please check. > > Max > This patch was added to -mm tree the 14 Dec 2006 (see http://www.mail-archive.com/mm-commits@vger.kernel.org/msg05338.html). It was then dropped from -mm tree the 05 Mar 2007 (see http://www.mail-archive.com/mm-commits@vger.kernel.org/msg10818.html), with this comment: "This patch was dropped because it is obsolete" No idea why it was obsolete. Does somebody know ? ~~ laurent ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: REISER4: fix for reiser4_write_extent @ 2007-04-07 12:51 ` Laurent Riffard 0 siblings, 0 replies; 64+ messages in thread From: Laurent Riffard @ 2007-04-07 12:51 UTC (permalink / raw) To: Ignatich; +Cc: reiserfs-list, linux-kernel, Edward Shishkin Le 06.04.2007 00:42, Ignatich a écrit : > While trying to find the cause of problems with reiser4 in recent > kernels I came across this. > > Incomplete write handling seem to be missing from reiser4_write_extent() > thanks to reiser4-temp-fix.patch. Strangely, there is a patch by Edward > Shishkin that should address that issue, but it is missing from -mm > tree. Please check. > > Max > This patch was added to -mm tree the 14 Dec 2006 (see http://www.mail-archive.com/mm-commits@vger.kernel.org/msg05338.html). It was then dropped from -mm tree the 05 Mar 2007 (see http://www.mail-archive.com/mm-commits@vger.kernel.org/msg10818.html), with this comment: "This patch was dropped because it is obsolete" No idea why it was obsolete. Does somebody know ? ~~ laurent ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: REISER4: fix for reiser4_write_extent 2007-04-07 12:51 ` Laurent Riffard @ 2007-04-07 19:29 ` Edward Shishkin -1 siblings, 0 replies; 64+ messages in thread From: Edward Shishkin @ 2007-04-07 19:29 UTC (permalink / raw) To: Laurent Riffard; +Cc: Ignatich, reiserfs-list, linux-kernel Laurent Riffard wrote: > Le 06.04.2007 00:42, Ignatich a écrit : > >> While trying to find the cause of problems with reiser4 in recent >> kernels I came across this. >> >> Incomplete write handling seem to be missing from >> reiser4_write_extent() thanks to reiser4-temp-fix.patch. Strangely, >> there is a patch by Edward Shishkin that should address that issue, >> but it is missing from -mm tree. Please check. >> >> Max >> > > This patch was added to -mm tree the 14 Dec 2006 (see > http://www.mail-archive.com/mm-commits@vger.kernel.org/msg05338.html). > > It was then dropped from -mm tree the 05 Mar 2007 (see > http://www.mail-archive.com/mm-commits@vger.kernel.org/msg10818.html), > with this comment: > "This patch was dropped because it is obsolete" > > No idea why it was obsolete. Does somebody know ? This uses not settled interface filemap_copy_from_user_atomic/nonatomic However, those things should be fixed. I'll prepare the patch a bit later.. Thanks, Edward. ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: REISER4: fix for reiser4_write_extent @ 2007-04-07 19:29 ` Edward Shishkin 0 siblings, 0 replies; 64+ messages in thread From: Edward Shishkin @ 2007-04-07 19:29 UTC (permalink / raw) To: Laurent Riffard; +Cc: Ignatich, reiserfs-list, linux-kernel Laurent Riffard wrote: > Le 06.04.2007 00:42, Ignatich a écrit : > >> While trying to find the cause of problems with reiser4 in recent >> kernels I came across this. >> >> Incomplete write handling seem to be missing from >> reiser4_write_extent() thanks to reiser4-temp-fix.patch. Strangely, >> there is a patch by Edward Shishkin that should address that issue, >> but it is missing from -mm tree. Please check. >> >> Max >> > > This patch was added to -mm tree the 14 Dec 2006 (see > http://www.mail-archive.com/mm-commits@vger.kernel.org/msg05338.html). > > It was then dropped from -mm tree the 05 Mar 2007 (see > http://www.mail-archive.com/mm-commits@vger.kernel.org/msg10818.html), > with this comment: > "This patch was dropped because it is obsolete" > > No idea why it was obsolete. Does somebody know ? This uses not settled interface filemap_copy_from_user_atomic/nonatomic However, those things should be fixed. I'll prepare the patch a bit later.. Thanks, Edward. ^ permalink raw reply [flat|nested] 64+ messages in thread
end of thread, other threads:[~2007-04-09 18:36 UTC | newest] Thread overview: 64+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2007-04-05 22:42 REISER4: fix for reiser4_write_extent Ignatich 2007-04-06 0:05 ` Reiser4. BEST FILESYSTEM EVER? I need help johnrobertbanks 2007-04-06 0:05 ` johnrobertbanks 2007-04-06 0:23 ` H. Peter Anvin 2007-04-06 0:34 ` johnrobertbanks 2007-04-06 0:39 ` H. Peter Anvin 2007-04-06 1:34 ` Reiser4. BEST FILESYSTEM EVER johnrobertbanks 2007-04-06 3:12 ` Valdis.Kletnieks 2007-04-06 4:07 ` H. Peter Anvin 2007-04-06 4:32 ` johnrobertbanks [not found] ` <20070406152119.GC4228@delft.aura.cs.cmu.edu> 2007-04-07 2:47 ` johnrobertbanks 2007-04-07 3:30 ` Jan Harkes 2007-04-07 5:58 ` johnrobertbanks 2007-04-07 7:15 ` Willy Tarreau 2007-04-07 13:47 ` johnrobertbanks 2007-04-07 14:11 ` Krzysztof Halasa 2007-04-07 15:07 ` johnrobertbanks 2007-04-07 16:05 ` Pekka Enberg 2007-04-07 17:10 ` Valdis.Kletnieks 2007-04-08 16:31 ` Adrian Bunk 2007-04-07 17:21 ` Valdis.Kletnieks 2007-04-08 0:41 ` Krzysztof Halasa 2007-04-07 17:39 ` Valdis.Kletnieks 2007-04-08 4:32 ` David H. Lynch Jr 2007-04-07 19:17 ` Lennart Sorensen 2007-04-08 0:44 ` johnrobertbanks 2007-04-08 1:27 ` Lennart Sorensen 2007-04-08 2:56 ` Theodore Tso 2007-04-08 4:13 ` johnrobertbanks 2007-04-08 12:48 ` Jose Celestino 2007-04-08 13:21 ` johnrobertbanks 2007-04-08 14:14 ` Willy Tarreau 2007-04-08 17:03 ` Theodore Tso 2007-04-08 18:18 ` Jeff Mahoney 2007-04-08 4:32 ` Christer Weinigel 2007-04-08 21:50 ` Reiser4. BEST FILESYSTEM EVER - Christer Weinigel johnrobertbanks 2007-04-08 21:50 ` johnrobertbanks 2007-04-08 22:58 ` Richard Knutsson 2007-04-09 5:14 ` johnrobertbanks 2007-04-09 7:07 ` Willy Tarreau 2007-04-09 16:10 ` Richard Knutsson 2007-04-09 1:24 ` Christer Weinigel 2007-04-09 3:38 ` David H. Lynch Jr 2007-04-09 3:16 ` David H. Lynch Jr 2007-04-09 4:25 ` johnrobertbanks 2007-04-09 18:35 ` Reiser4. BEST FILESYSTEM EVER Nate Diller 2007-04-08 4:06 ` David H. Lynch Jr 2007-04-08 9:58 ` Jeff Garzik 2007-04-09 2:52 ` David H. Lynch Jr 2007-04-09 3:14 ` Jeff Garzik 2007-04-09 4:40 ` David H. Lynch Jr 2007-04-09 4:58 ` Jeff Garzik 2007-04-07 1:26 ` COMPILING AND CONFIGURING A NEW KERNEL johnrobertbanks 2007-04-07 1:26 ` johnrobertbanks 2007-04-07 7:45 ` johnrobertbanks 2007-04-07 16:57 ` Valdis.Kletnieks 2007-04-08 1:11 ` johnrobertbanks 2007-04-07 16:42 ` Valdis.Kletnieks 2007-04-08 1:02 ` johnrobertbanks 2007-04-08 1:42 ` Lennart Sorensen 2007-04-07 12:51 ` REISER4: fix for reiser4_write_extent Laurent Riffard 2007-04-07 12:51 ` Laurent Riffard 2007-04-07 19:29 ` Edward Shishkin 2007-04-07 19:29 ` Edward Shishkin
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.