* corrupt leaf; unaligned key offset for csum item @ 2018-06-09 12:22 Simon Kaiser 2018-06-09 22:53 ` Chris Murphy 2018-06-10 1:56 ` Qu Wenruo 0 siblings, 2 replies; 11+ messages in thread From: Simon Kaiser @ 2018-06-09 12:22 UTC (permalink / raw) To: linux-btrfs Hi all, I'm having trouble with my btrfs volume on a Samsung SSD 840 pro. On startup I immediately get the error BTRFS critical (device sda5): corrupt leaf: root=1 block=73263579136 slot=105, unaligned key offset for csum item, have 1271496708 should be aligned to 4096 I used to make regular backups, but slacked off recently. Fixing the filesystem isn't essential in any way, but would save me quite some time on config tinkering and weeding out vacation photos. The second of which I don't enjoy at all... On similar issues faulty RAM seemed to be the most prominent cause. I added a third memory module quite recently which could have caused this, but I couldn't run memtest long enough yet to verify. Here's the trimmed dmesg output (full output at https://pastebin.com/7XF9i09K) [ 4.062748] BTRFS critical (device sda5): corrupt leaf: root=1 block=73263579136 slot=105, unaligned key offset for csum item, have 1271496708 should be aligned to 4096 [ 4.063806] BTRFS info (device sda5): no csum found for inode 7810710 start 0 [ 4.064260] BTRFS warning (device sda5): csum failed root 257 ino 7810710 off 109201939968 csum 0x5307fa6f expected csum 0x00000000 mirror 1 [ 4.075834] BTRFS info (device sdb3): use lzo compression, level 0 [ 4.075836] BTRFS info (device sdb3): enabling auto defrag [ 4.075837] BTRFS info (device sdb3): disk space caching is enabled `brfs inspect-internal dump-tree -b 73263579136 /dev/sda5` (trimmed, full output at https://pastebin.com/dqBJ3b6D) btrfs-progs v4.16.1 leaf 73263579136 items 194 free space 4089 generation 530006 owner CSUM_TREE leaf 73263579136 flags 0x1(WRITTEN) backref revision 1 fs uuid 95b4974b-a798-44b3-99aa-a4eef990aeeb chunk uuid 9c8f46c3-ba51-4175-857c-8041543fa813 item 0 key (EXTENT_CSUM EXTENT_CSUM 1268850688) itemoff 16263 itemsize 20 range start 1268850688 end 1268871168 length 20480 item 1 key (EXTENT_CSUM EXTENT_CSUM 1268871168) itemoff 16259 itemsize 4 range start 1268871168 end 1268875264 length 4096 item 2 key (EXTENT_CSUM EXTENT_CSUM 1268875264) itemoff 16255 itemsize 4 range start 1268875264 end 1268879360 length 4096 item 3 key (EXTENT_CSUM EXTENT_CSUM 1268879360) itemoff 16239 itemsize 16 range start 1268879360 end 1268895744 length 16384 item 4 key (EXTENT_CSUM EXTENT_CSUM 1268895744) itemoff 16235 itemsize 4 range start 1268895744 end 1268899840 length 4096 item 5 key (EXTENT_CSUM EXTENT_CSUM 1268899840) itemoff 16223 itemsize 12 range start 1268899840 end 1268912128 length 12288 [...] item 100 key (EXTENT_CSUM EXTENT_CSUM 1271410688) itemoff 13827 itemsize 8 range start 1271410688 end 1271418880 length 8192 item 101 key (EXTENT_CSUM EXTENT_CSUM 1271418880) itemoff 13811 itemsize 16 range start 1271418880 end 1271435264 length 16384 item 102 key (EXTENT_CSUM EXTENT_CSUM 1271435264) itemoff 13807 itemsize 4 range start 1271435264 end 1271439360 length 4096 item 103 key (EXTENT_CSUM EXTENT_CSUM 1271439360) itemoff 13771 itemsize 36 range start 1271439360 end 1271476224 length 36864 item 104 key (EXTENT_CSUM EXTENT_CSUM 1271488512) itemoff 13763 itemsize 8 range start 1271488512 end 1271496704 length 8192 item 105 key (EXTENT_CSUM EXTENT_CSUM 1271496708) itemoff 13403 itemsize 360 range start 1271496708 end 1271865348 length 368640 item 106 key (EXTENT_CSUM EXTENT_CSUM 1271873536) itemoff 13399 itemsize 4 range start 1271873536 end 1271877632 length 4096 item 107 key (EXTENT_CSUM EXTENT_CSUM 1271877632) itemoff 13379 itemsize 20 range start 1271877632 end 1271898112 length 20480 item 108 key (EXTENT_CSUM EXTENT_CSUM 1271898112) itemoff 13375 itemsize 4 range start 1271898112 end 1271902208 length 4096 item 109 key (EXTENT_CSUM EXTENT_CSUM 1271918592) itemoff 13363 itemsize 12 range start 1271918592 end 1271930880 length 12288 item 110 key (EXTENT_CSUM EXTENT_CSUM 1271930880) itemoff 13351 itemsize 12 range start 1271930880 end 1271943168 length 12288 [...] `uname -a` Linux hostname 4.16.12-1-ARCH #1 SMP PREEMPT Fri May 25 23:30:31 UTC 2018 x86_64 GNU/Linux `btrfs fi show` Label: 'arch' uuid: 95b4974b-a798-44b3-99aa-a4eef990aeeb Total devices 1 FS bytes used 78.10GiB devid 1 size 100.00GiB used 94.05GiB path /dev/sda5 `btrfs fi df` Data, single: total=90.01GiB, used=76.32GiB System, single: total=32.00MiB, used=16.00KiB Metadata, single: total=4.01GiB, used=1.78GiB GlobalReserve, single: total=512.00MiB, used=0.00B Is there any chance this can be fixed? Your help would be greatly apperciated! Yours, Simon ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: corrupt leaf; unaligned key offset for csum item 2018-06-09 12:22 corrupt leaf; unaligned key offset for csum item Simon Kaiser @ 2018-06-09 22:53 ` Chris Murphy 2018-06-10 1:56 ` Qu Wenruo 1 sibling, 0 replies; 11+ messages in thread From: Chris Murphy @ 2018-06-09 22:53 UTC (permalink / raw) To: Simon Kaiser; +Cc: Btrfs BTRFS On Sat, Jun 9, 2018 at 6:22 AM, Simon Kaiser <simon.kaiser@kit.edu> wrote: > Hi all, > > I'm having trouble with my btrfs volume on a Samsung SSD 840 pro. On > startup I immediately get the error > > BTRFS critical (device sda5): corrupt leaf: root=1 block=73263579136 > slot=105, unaligned key offset for csum item, have 1271496708 should be > aligned to 4096 > > I used to make regular backups, but slacked off recently. Fixing the > filesystem isn't essential in any way, but would save me quite some time > on config tinkering and weeding out vacation photos. The second of > which I don't enjoy at all... > > On similar issues faulty RAM seemed to be the most prominent cause. I > added a third memory module quite recently which could have caused this, > but I couldn't run memtest long enough yet to verify. > > Here's the trimmed dmesg output (full output at > https://pastebin.com/7XF9i09K) > > [ 4.062748] BTRFS critical (device sda5): corrupt leaf: root=1 block=73263579136 slot=105, unaligned key offset for csum item, have 1271496708 should be aligned to 4096 > [ 4.063806] BTRFS info (device sda5): no csum found for inode 7810710 start 0 > [ 4.064260] BTRFS warning (device sda5): csum failed root 257 ino 7810710 off 109201939968 csum 0x5307fa6f expected csum 0x00000000 mirror 1 > [ 4.075834] BTRFS info (device sdb3): use lzo compression, level 0 > [ 4.075836] BTRFS info (device sdb3): enabling auto defrag > [ 4.075837] BTRFS info (device sdb3): disk space caching is enabled What are your mount options? I suggest removing the suspected bad ram first, and then boot off alternate media: btrfs insp dump-s -f <dev> Try to mount with -o ro, and if that doesn't work try -o ro,norecovery. Read only mount is more tolerant of problems than rw mount. If either work, then update your backups while you have a chance. It's easier to backup from ro mount than it is to use 'btrfs restore' If neither -o ro nor -o ro,norecovery work try: -o usebackuproot There might be some recent changes data loss using that option. But if you're using the discard mount option, it probably will fail. If -o ro does not work, but -o ro,norecovery did work, it's a candidate for 'btrfs rescue zero-log' again there might be some recent change data loss if you zero the log. btrfs check --repair is a last resort so I don't recommend it until you've got a backup some other way. -- Chris Murphy ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: corrupt leaf; unaligned key offset for csum item 2018-06-09 12:22 corrupt leaf; unaligned key offset for csum item Simon Kaiser 2018-06-09 22:53 ` Chris Murphy @ 2018-06-10 1:56 ` Qu Wenruo 2018-06-10 3:53 ` Chris Murphy 2018-06-11 8:10 ` Qu Wenruo 1 sibling, 2 replies; 11+ messages in thread From: Qu Wenruo @ 2018-06-10 1:56 UTC (permalink / raw) To: Simon Kaiser, linux-btrfs [-- Attachment #1.1: Type: text/plain, Size: 5583 bytes --] On 2018年06月09日 20:22, Simon Kaiser wrote: > Hi all, > > I'm having trouble with my btrfs volume on a Samsung SSD 840 pro. On > startup I immediately get the error > > BTRFS critical (device sda5): corrupt leaf: root=1 block=73263579136 > slot=105, unaligned key offset for csum item, have 1271496708 should be > aligned to 4096 Tree-checker detects something. And indeed the bytenr is not even aligned to 512. So definitely a corruption. > > I used to make regular backups, but slacked off recently. Fixing the > filesystem isn't essential in any way, but would save me quite some time > on config tinkering and weeding out vacation photos. The second of > which I don't enjoy at all... > > On similar issues faulty RAM seemed to be the most prominent cause. I > added a third memory module quite recently which could have caused this, > but I couldn't run memtest long enough yet to verify. Indeed, the corrupted bytenr is 0x4bc98004. Looks pretty like a bit flip in the 3rd lowest bit. It can be fixed by manually patching the corrupted leaf to get rid of the bitflip. I could provide a special branch of btrfs-progs to fix it easily. But before that, it's better to do a scrub to see if there is other similar problems, so I could fix them all. Thanks, Qu > > Here's the trimmed dmesg output (full output at > https://pastebin.com/7XF9i09K) > > [ 4.062748] BTRFS critical (device sda5): corrupt leaf: root=1 block=73263579136 slot=105, unaligned key offset for csum item, have 1271496708 should be aligned to 4096 > [ 4.063806] BTRFS info (device sda5): no csum found for inode 7810710 start 0 > [ 4.064260] BTRFS warning (device sda5): csum failed root 257 ino 7810710 off 109201939968 csum 0x5307fa6f expected csum 0x00000000 mirror 1 > [ 4.075834] BTRFS info (device sdb3): use lzo compression, level 0 > [ 4.075836] BTRFS info (device sdb3): enabling auto defrag > [ 4.075837] BTRFS info (device sdb3): disk space caching is enabled > > `brfs inspect-internal dump-tree -b 73263579136 /dev/sda5` (trimmed, > full output at https://pastebin.com/dqBJ3b6D) > > btrfs-progs v4.16.1 > leaf 73263579136 items 194 free space 4089 generation 530006 owner CSUM_TREE > leaf 73263579136 flags 0x1(WRITTEN) backref revision 1 > fs uuid 95b4974b-a798-44b3-99aa-a4eef990aeeb > chunk uuid 9c8f46c3-ba51-4175-857c-8041543fa813 > item 0 key (EXTENT_CSUM EXTENT_CSUM 1268850688) itemoff 16263 itemsize 20 > range start 1268850688 end 1268871168 length 20480 > item 1 key (EXTENT_CSUM EXTENT_CSUM 1268871168) itemoff 16259 itemsize 4 > range start 1268871168 end 1268875264 length 4096 > item 2 key (EXTENT_CSUM EXTENT_CSUM 1268875264) itemoff 16255 itemsize 4 > range start 1268875264 end 1268879360 length 4096 > item 3 key (EXTENT_CSUM EXTENT_CSUM 1268879360) itemoff 16239 itemsize 16 > range start 1268879360 end 1268895744 length 16384 > item 4 key (EXTENT_CSUM EXTENT_CSUM 1268895744) itemoff 16235 itemsize 4 > range start 1268895744 end 1268899840 length 4096 > item 5 key (EXTENT_CSUM EXTENT_CSUM 1268899840) itemoff 16223 itemsize 12 > range start 1268899840 end 1268912128 length 12288 > [...] > item 100 key (EXTENT_CSUM EXTENT_CSUM 1271410688) itemoff 13827 itemsize 8 > range start 1271410688 end 1271418880 length 8192 > item 101 key (EXTENT_CSUM EXTENT_CSUM 1271418880) itemoff 13811 itemsize 16 > range start 1271418880 end 1271435264 length 16384 > item 102 key (EXTENT_CSUM EXTENT_CSUM 1271435264) itemoff 13807 itemsize 4 > range start 1271435264 end 1271439360 length 4096 > item 103 key (EXTENT_CSUM EXTENT_CSUM 1271439360) itemoff 13771 itemsize 36 > range start 1271439360 end 1271476224 length 36864 > item 104 key (EXTENT_CSUM EXTENT_CSUM 1271488512) itemoff 13763 itemsize 8 > range start 1271488512 end 1271496704 length 8192 > item 105 key (EXTENT_CSUM EXTENT_CSUM 1271496708) itemoff 13403 itemsize 360 > range start 1271496708 end 1271865348 length 368640 > item 106 key (EXTENT_CSUM EXTENT_CSUM 1271873536) itemoff 13399 itemsize 4 > range start 1271873536 end 1271877632 length 4096 > item 107 key (EXTENT_CSUM EXTENT_CSUM 1271877632) itemoff 13379 itemsize 20 > range start 1271877632 end 1271898112 length 20480 > item 108 key (EXTENT_CSUM EXTENT_CSUM 1271898112) itemoff 13375 itemsize 4 > range start 1271898112 end 1271902208 length 4096 > item 109 key (EXTENT_CSUM EXTENT_CSUM 1271918592) itemoff 13363 itemsize 12 > range start 1271918592 end 1271930880 length 12288 > item 110 key (EXTENT_CSUM EXTENT_CSUM 1271930880) itemoff 13351 itemsize 12 > range start 1271930880 end 1271943168 length 12288 > [...] > > `uname -a` > > Linux hostname 4.16.12-1-ARCH #1 SMP PREEMPT Fri May 25 23:30:31 UTC 2018 x86_64 GNU/Linux > > `btrfs fi show` > > Label: 'arch' uuid: 95b4974b-a798-44b3-99aa-a4eef990aeeb > Total devices 1 FS bytes used 78.10GiB > devid 1 size 100.00GiB used 94.05GiB path /dev/sda5 > > `btrfs fi df` > > Data, single: total=90.01GiB, used=76.32GiB > System, single: total=32.00MiB, used=16.00KiB > Metadata, single: total=4.01GiB, used=1.78GiB > GlobalReserve, single: total=512.00MiB, used=0.00B > > Is there any chance this can be fixed? Your help would be greatly > apperciated! > > Yours, > Simon > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: corrupt leaf; unaligned key offset for csum item 2018-06-10 1:56 ` Qu Wenruo @ 2018-06-10 3:53 ` Chris Murphy 2018-06-10 5:12 ` Qu Wenruo 2018-06-11 8:10 ` Qu Wenruo 1 sibling, 1 reply; 11+ messages in thread From: Chris Murphy @ 2018-06-10 3:53 UTC (permalink / raw) To: Qu Wenruo; +Cc: Simon Kaiser, Btrfs BTRFS On Sat, Jun 9, 2018 at 7:56 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > Indeed, the corrupted bytenr is 0x4bc98004. > Looks pretty like a bit flip in the 3rd lowest bit. > > It can be fixed by manually patching the corrupted leaf to get rid of > the bitflip. > I could provide a special branch of btrfs-progs to fix it easily. > > But before that, it's better to do a scrub to see if there is other > similar problems, so I could fix them all. > Do you think Simon should try -o ro and -o ro,norecovery and see if he can update backups the easy way first? And then use the offline scrub to check for additional problems? Simon, the offline scrub is done unmounted with 'btrfs check --check-data-csum <dev>' and it is a read-only check. -- Chris Murphy ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: corrupt leaf; unaligned key offset for csum item 2018-06-10 3:53 ` Chris Murphy @ 2018-06-10 5:12 ` Qu Wenruo 2018-06-10 13:38 ` Simon Kaiser 0 siblings, 1 reply; 11+ messages in thread From: Qu Wenruo @ 2018-06-10 5:12 UTC (permalink / raw) To: Chris Murphy; +Cc: Simon Kaiser, Btrfs BTRFS [-- Attachment #1.1: Type: text/plain, Size: 1149 bytes --] On 2018年06月10日 11:53, Chris Murphy wrote: > On Sat, Jun 9, 2018 at 7:56 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >> >> Indeed, the corrupted bytenr is 0x4bc98004. >> Looks pretty like a bit flip in the 3rd lowest bit. >> >> It can be fixed by manually patching the corrupted leaf to get rid of >> the bitflip. >> I could provide a special branch of btrfs-progs to fix it easily. >> >> But before that, it's better to do a scrub to see if there is other >> similar problems, so I could fix them all. >> > > Do you think Simon should try -o ro and -o ro,norecovery and see if he > can update backups the easy way first? If the bit flip is the only problem, I could fix it manually and nothing is lost, the fs can be used as usual. > And then use the offline scrub > to check for additional problems? I mean online scrub. I didn't notice extra error, and I don't believe even for a faulty memory, bit flip is that easy to happen, so on-line scrub should do the work. > > Simon, the offline scrub is done unmounted with 'btrfs check > --check-data-csum <dev>' and it is a read-only check. > > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: corrupt leaf; unaligned key offset for csum item 2018-06-10 5:12 ` Qu Wenruo @ 2018-06-10 13:38 ` Simon Kaiser 2018-06-10 21:30 ` Chris Murphy 0 siblings, 1 reply; 11+ messages in thread From: Simon Kaiser @ 2018-06-10 13:38 UTC (permalink / raw) To: Qu Wenruo, Chris Murphy; +Cc: Btrfs BTRFS Qu Wenruo <quwenruo.btrfs@gmx.com> writes: > On 2018年06月10日 11:53, Chris Murphy wrote: >> On Sat, Jun 9, 2018 at 7:56 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>> >>> Indeed, the corrupted bytenr is 0x4bc98004. >>> Looks pretty like a bit flip in the 3rd lowest bit. >>> >>> It can be fixed by manually patching the corrupted leaf to get rid of >>> the bitflip. >>> I could provide a special branch of btrfs-progs to fix it easily. >>> >>> But before that, it's better to do a scrub to see if there is other >>> similar problems, so I could fix them all. >>> Online scrub `btrfs scrub start -B -r /mnt/` (`mount -o ro,norecovery`) ERROR: scrubbing /mnt/ failed for device id 1: ret=-1, errno=5 (Input/output error) scrub canceled for 95b4974b-a798-44b3-99aa-a4eef990aeeb scrub started at Sun Jun 10 13:25:37 2018 and was aborted after 00:00:03 total bytes scrubbed: 1.09GiB with 0 errors Offline scrub `btrfs check --check-data-csum /dev/sda5` Checking filesystem on /dev/sda5 UUID: 95b4974b-a798-44b3-99aa-a4eef990aeeb checking extents checking free space cache checking fs roots root 257 inode 7984709 errors 1000, some csum missing root 257 inode 8016535 errors 800, odd csum item root 407 inode 7984709 errors 1000, some csum missing ERROR: errors found in fs roots found 83859496960 bytes used, error(s) found total csum bytes: 79588008 total tree bytes: 1907539968 total fs tree bytes: 1707540480 total extent tree bytes: 93782016 btree space waste bytes: 338263641 file data blocks allocated: 97127669760 referenced 108545990656 btrfs check --check-data-csum /dev/sda5 10.70s user 1.51s system 82% cpu 14.736 total >> >> Do you think Simon should try -o ro and -o ro,norecovery and see if he >> can update backups the easy way first? On copying from this drive, btrfs throws the error I posted initally on lots of files, but the rest look ok. I also dd'ed the partition on a backup drive, so I should be good to go on. > > If the bit flip is the only problem, I could fix it manually and nothing > is lost, the fs can be used as usual. > >> And then use the offline scrub >> to check for additional problems? > > I mean online scrub. > > I didn't notice extra error, and I don't believe even for a faulty > memory, bit flip is that easy to happen, so on-line scrub should do the > work. > >> >> Simon, the offline scrub is done unmounted with 'btrfs check >> --check-data-csum <dev>' and it is a read-only check. >> >> Thank you both for looking into this! Yours, Simon ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: corrupt leaf; unaligned key offset for csum item 2018-06-10 13:38 ` Simon Kaiser @ 2018-06-10 21:30 ` Chris Murphy 2018-06-10 22:32 ` Simon Kaiser 0 siblings, 1 reply; 11+ messages in thread From: Chris Murphy @ 2018-06-10 21:30 UTC (permalink / raw) To: Simon Kaiser; +Cc: Qu Wenruo, Chris Murphy, Btrfs BTRFS On Sun, Jun 10, 2018 at 7:38 AM, Simon Kaiser <simon.kaiser@kit.edu> wrote: > Qu Wenruo <quwenruo.btrfs@gmx.com> writes: > >> On 2018年06月10日 11:53, Chris Murphy wrote: >>> On Sat, Jun 9, 2018 at 7:56 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>>> >>>> Indeed, the corrupted bytenr is 0x4bc98004. >>>> Looks pretty like a bit flip in the 3rd lowest bit. >>>> >>>> It can be fixed by manually patching the corrupted leaf to get rid of >>>> the bitflip. >>>> I could provide a special branch of btrfs-progs to fix it easily. >>>> >>>> But before that, it's better to do a scrub to see if there is other >>>> similar problems, so I could fix them all. >>>> > > Online scrub `btrfs scrub start -B -r /mnt/` (`mount -o ro,norecovery`) > > ERROR: scrubbing /mnt/ failed for device id 1: ret=-1, errno=5 (Input/output error) > scrub canceled for 95b4974b-a798-44b3-99aa-a4eef990aeeb > scrub started at Sun Jun 10 13:25:37 2018 and was aborted after 00:00:03 > total bytes scrubbed: 1.09GiB with 0 errors Is there any kernel message at the time of this i/o error? Either Btrfs or device error? -- Chris Murphy ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: corrupt leaf; unaligned key offset for csum item 2018-06-10 21:30 ` Chris Murphy @ 2018-06-10 22:32 ` Simon Kaiser 0 siblings, 0 replies; 11+ messages in thread From: Simon Kaiser @ 2018-06-10 22:32 UTC (permalink / raw) To: Chris Murphy; +Cc: Qu Wenruo, Btrfs BTRFS Chris Murphy <lists@colorremedies.com> writes: >> Online scrub `btrfs scrub start -B -r /mnt/` (`mount -o ro,norecovery`) >> >> ERROR: scrubbing /mnt/ failed for device id 1: ret=-1, errno=5 (Input/output error) >> scrub canceled for 95b4974b-a798-44b3-99aa-a4eef990aeeb >> scrub started at Sun Jun 10 13:25:37 2018 and was aborted after 00:00:03 >> total bytes scrubbed: 1.09GiB with 0 errors > > Is there any kernel message at the time of this i/o error? Either > Btrfs or device error? Around the time of the error, dmesg shows only the btrfs error I initially posted: [ 73.453640] BTRFS critical (device sda5): corrupt leaf: root=1 block=73263579136 slot=105, unaligned key offset for csum item, have 1271496708 should be aligned to 4096 [ 73.791479] BTRFS critical (device sda5): corrupt leaf: root=1 block=73263579136 slot=105, unaligned key offset for csum item, have 1271496708 should be aligned to 4096 Yours, Simon ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: corrupt leaf; unaligned key offset for csum item 2018-06-10 1:56 ` Qu Wenruo 2018-06-10 3:53 ` Chris Murphy @ 2018-06-11 8:10 ` Qu Wenruo 2018-06-11 16:42 ` Simon Kaiser 1 sibling, 1 reply; 11+ messages in thread From: Qu Wenruo @ 2018-06-11 8:10 UTC (permalink / raw) To: Simon Kaiser, linux-btrfs [-- Attachment #1.1: Type: text/plain, Size: 6146 bytes --] On 2018年06月10日 09:56, Qu Wenruo wrote: > > > On 2018年06月09日 20:22, Simon Kaiser wrote: >> Hi all, >> >> I'm having trouble with my btrfs volume on a Samsung SSD 840 pro. On >> startup I immediately get the error >> >> BTRFS critical (device sda5): corrupt leaf: root=1 block=73263579136 >> slot=105, unaligned key offset for csum item, have 1271496708 should be >> aligned to 4096 > > Tree-checker detects something. > > And indeed the bytenr is not even aligned to 512. > So definitely a corruption. > >> >> I used to make regular backups, but slacked off recently. Fixing the >> filesystem isn't essential in any way, but would save me quite some time >> on config tinkering and weeding out vacation photos. The second of >> which I don't enjoy at all... >> >> On similar issues faulty RAM seemed to be the most prominent cause. I >> added a third memory module quite recently which could have caused this, >> but I couldn't run memtest long enough yet to verify. > > Indeed, the corrupted bytenr is 0x4bc98004. > Looks pretty like a bit flip in the 3rd lowest bit. > > It can be fixed by manually patching the corrupted leaf to get rid of > the bitflip. > I could provide a special branch of btrfs-progs to fix it easily. > > But before that, it's better to do a scrub to see if there is other > similar problems, so I could fix them all. Since there is no other csum error exposed by btrfs check/scrub, the fix for this particular csum item key bit flip can be fetched here: https://github.com/adam900710/btrfs-progs/tree/dirty_fix The usage will be (after the normal compile procedure) $ ./btrfs-corrupt-block -X <device> It needs to be executed offline (the <device> must be unmounted). Thanks, Qu > > Thanks, > Qu > >> >> Here's the trimmed dmesg output (full output at >> https://pastebin.com/7XF9i09K) >> >> [ 4.062748] BTRFS critical (device sda5): corrupt leaf: root=1 block=73263579136 slot=105, unaligned key offset for csum item, have 1271496708 should be aligned to 4096 >> [ 4.063806] BTRFS info (device sda5): no csum found for inode 7810710 start 0 >> [ 4.064260] BTRFS warning (device sda5): csum failed root 257 ino 7810710 off 109201939968 csum 0x5307fa6f expected csum 0x00000000 mirror 1 >> [ 4.075834] BTRFS info (device sdb3): use lzo compression, level 0 >> [ 4.075836] BTRFS info (device sdb3): enabling auto defrag >> [ 4.075837] BTRFS info (device sdb3): disk space caching is enabled >> >> `brfs inspect-internal dump-tree -b 73263579136 /dev/sda5` (trimmed, >> full output at https://pastebin.com/dqBJ3b6D) >> >> btrfs-progs v4.16.1 >> leaf 73263579136 items 194 free space 4089 generation 530006 owner CSUM_TREE >> leaf 73263579136 flags 0x1(WRITTEN) backref revision 1 >> fs uuid 95b4974b-a798-44b3-99aa-a4eef990aeeb >> chunk uuid 9c8f46c3-ba51-4175-857c-8041543fa813 >> item 0 key (EXTENT_CSUM EXTENT_CSUM 1268850688) itemoff 16263 itemsize 20 >> range start 1268850688 end 1268871168 length 20480 >> item 1 key (EXTENT_CSUM EXTENT_CSUM 1268871168) itemoff 16259 itemsize 4 >> range start 1268871168 end 1268875264 length 4096 >> item 2 key (EXTENT_CSUM EXTENT_CSUM 1268875264) itemoff 16255 itemsize 4 >> range start 1268875264 end 1268879360 length 4096 >> item 3 key (EXTENT_CSUM EXTENT_CSUM 1268879360) itemoff 16239 itemsize 16 >> range start 1268879360 end 1268895744 length 16384 >> item 4 key (EXTENT_CSUM EXTENT_CSUM 1268895744) itemoff 16235 itemsize 4 >> range start 1268895744 end 1268899840 length 4096 >> item 5 key (EXTENT_CSUM EXTENT_CSUM 1268899840) itemoff 16223 itemsize 12 >> range start 1268899840 end 1268912128 length 12288 >> [...] >> item 100 key (EXTENT_CSUM EXTENT_CSUM 1271410688) itemoff 13827 itemsize 8 >> range start 1271410688 end 1271418880 length 8192 >> item 101 key (EXTENT_CSUM EXTENT_CSUM 1271418880) itemoff 13811 itemsize 16 >> range start 1271418880 end 1271435264 length 16384 >> item 102 key (EXTENT_CSUM EXTENT_CSUM 1271435264) itemoff 13807 itemsize 4 >> range start 1271435264 end 1271439360 length 4096 >> item 103 key (EXTENT_CSUM EXTENT_CSUM 1271439360) itemoff 13771 itemsize 36 >> range start 1271439360 end 1271476224 length 36864 >> item 104 key (EXTENT_CSUM EXTENT_CSUM 1271488512) itemoff 13763 itemsize 8 >> range start 1271488512 end 1271496704 length 8192 >> item 105 key (EXTENT_CSUM EXTENT_CSUM 1271496708) itemoff 13403 itemsize 360 >> range start 1271496708 end 1271865348 length 368640 >> item 106 key (EXTENT_CSUM EXTENT_CSUM 1271873536) itemoff 13399 itemsize 4 >> range start 1271873536 end 1271877632 length 4096 >> item 107 key (EXTENT_CSUM EXTENT_CSUM 1271877632) itemoff 13379 itemsize 20 >> range start 1271877632 end 1271898112 length 20480 >> item 108 key (EXTENT_CSUM EXTENT_CSUM 1271898112) itemoff 13375 itemsize 4 >> range start 1271898112 end 1271902208 length 4096 >> item 109 key (EXTENT_CSUM EXTENT_CSUM 1271918592) itemoff 13363 itemsize 12 >> range start 1271918592 end 1271930880 length 12288 >> item 110 key (EXTENT_CSUM EXTENT_CSUM 1271930880) itemoff 13351 itemsize 12 >> range start 1271930880 end 1271943168 length 12288 >> [...] >> >> `uname -a` >> >> Linux hostname 4.16.12-1-ARCH #1 SMP PREEMPT Fri May 25 23:30:31 UTC 2018 x86_64 GNU/Linux >> >> `btrfs fi show` >> >> Label: 'arch' uuid: 95b4974b-a798-44b3-99aa-a4eef990aeeb >> Total devices 1 FS bytes used 78.10GiB >> devid 1 size 100.00GiB used 94.05GiB path /dev/sda5 >> >> `btrfs fi df` >> >> Data, single: total=90.01GiB, used=76.32GiB >> System, single: total=32.00MiB, used=16.00KiB >> Metadata, single: total=4.01GiB, used=1.78GiB >> GlobalReserve, single: total=512.00MiB, used=0.00B >> >> Is there any chance this can be fixed? Your help would be greatly >> apperciated! >> >> Yours, >> Simon >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: corrupt leaf; unaligned key offset for csum item 2018-06-11 8:10 ` Qu Wenruo @ 2018-06-11 16:42 ` Simon Kaiser 2018-06-12 1:18 ` Qu Wenruo 0 siblings, 1 reply; 11+ messages in thread From: Simon Kaiser @ 2018-06-11 16:42 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs > On 2018年06月10日 09:56, Qu Wenruo wrote: > > Since there is no other csum error exposed by btrfs check/scrub, the fix > for this particular csum item key bit flip can be fetched here: > https://github.com/adam900710/btrfs-progs/tree/dirty_fix > > The usage will be (after the normal compile procedure) > $ ./btrfs-corrupt-block -X <device> > > It needs to be executed offline (the <device> must be unmounted). > > Thanks, > Qu > Thank you very much, that fixed the issue :) btrfs scrub now shows 5 checksum errors on a single file [ 1869.456426] BTRFS warning (device sda5): checksum error at logical 1271386112 on dev /dev/sda5, physical 1271386112, root 257, inode 8016535, offset 655360, length 4096, links 1 (path: var/log/journal/31d9cc011 2694cdab3887013fb7bbcf9/system@d8ae7586248c492d8d51c44efd7316da-0000000000000001-00056e3429e252bb.journal) [ 1869.456429] BTRFS error (device sda5): bdev /dev/sda5 errs: wr 1, rd 370, flush 0, corrupt 1, gen 0 [ 1869.457975] BTRFS warning (device sda5): checksum error at logical 1271390208 on dev /dev/sda5, physical 1271390208, root 257, inode 8016535, offset 655360, length 4096, links 1 (path: var/log/journal/31d9cc011 2694cdab3887013fb7bbcf9/system@d8ae7586248c492d8d51c44efd7316da-0000000000000001-00056e3429e252bb.journal) [ 1869.457978] BTRFS error (device sda5): bdev /dev/sda5 errs: wr 1, rd 370, flush 0, corrupt 2, gen 0 [ 1869.459381] BTRFS warning (device sda5): checksum error at logical 1271394304 on dev /dev/sda5, physical 1271394304, root 257, inode 8016535, offset 655360, length 4096, links 1 (path: var/log/journal/31d9cc011 2694cdab3887013fb7bbcf9/system@d8ae7586248c492d8d51c44efd7316da-0000000000000001-00056e3429e252bb.journal) [ 1869.459383] BTRFS error (device sda5): bdev /dev/sda5 errs: wr 1, rd 370, flush 0, corrupt 3, gen 0 [ 1869.460827] BTRFS warning (device sda5): checksum error at logical 1271398400 on dev /dev/sda5, physical 1271398400, root 257, inode 8016535, offset 655360, length 4096, links 1 (path: var/log/journal/31d9cc011 2694cdab3887013fb7bbcf9/system@d8ae7586248c492d8d51c44efd7316da-0000000000000001-00056e3429e252bb.journal) [ 1869.460829] BTRFS error (device sda5): bdev /dev/sda5 errs: wr 1, rd 370, flush 0, corrupt 4, gen 0 [ 1869.462118] BTRFS warning (device sda5): checksum error at logical 1271402496 on dev /dev/sda5, physical 1271402496, root 257, inode 8016535, offset 655360, length 4096, links 1 (path: var/log/journal/31d9cc011 2694cdab3887013fb7bbcf9/system@d8ae7586248c492d8d51c44efd7316da-0000000000000001-00056e3429e252bb.journal) [ 1869.462120] BTRFS error (device sda5): bdev /dev/sda5 errs: wr 1, rd 370, flush 0, corrupt 5, gen 0 I don't really need that file, can I simply delete it and forget about it or is there something else I should do? Yours, Simon ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: corrupt leaf; unaligned key offset for csum item 2018-06-11 16:42 ` Simon Kaiser @ 2018-06-12 1:18 ` Qu Wenruo 0 siblings, 0 replies; 11+ messages in thread From: Qu Wenruo @ 2018-06-12 1:18 UTC (permalink / raw) To: Simon Kaiser, linux-btrfs [-- Attachment #1.1: Type: text/plain, Size: 3854 bytes --] On 2018年06月12日 00:42, Simon Kaiser wrote: >> On 2018年06月10日 09:56, Qu Wenruo wrote: >> >> Since there is no other csum error exposed by btrfs check/scrub, the fix >> for this particular csum item key bit flip can be fetched here: >> https://github.com/adam900710/btrfs-progs/tree/dirty_fix >> >> The usage will be (after the normal compile procedure) >> $ ./btrfs-corrupt-block -X <device> >> >> It needs to be executed offline (the <device> must be unmounted). >> >> Thanks, >> Qu >> > > Thank you very much, that fixed the issue :) > > btrfs scrub now shows 5 checksum errors on a single file > > [ 1869.456426] BTRFS warning (device sda5): checksum error at logical 1271386112 on dev /dev/sda5, physical 1271386112, root 257, inode 8016535, offset 655360, length 4096, links 1 (path: var/log/journal/31d9cc011 > 2694cdab3887013fb7bbcf9/system@d8ae7586248c492d8d51c44efd7316da-0000000000000001-00056e3429e252bb.journal) > [ 1869.456429] BTRFS error (device sda5): bdev /dev/sda5 errs: wr 1, rd 370, flush 0, corrupt 1, gen 0 > [ 1869.457975] BTRFS warning (device sda5): checksum error at logical 1271390208 on dev /dev/sda5, physical 1271390208, root 257, inode 8016535, offset 655360, length 4096, links 1 (path: var/log/journal/31d9cc011 > 2694cdab3887013fb7bbcf9/system@d8ae7586248c492d8d51c44efd7316da-0000000000000001-00056e3429e252bb.journal) > [ 1869.457978] BTRFS error (device sda5): bdev /dev/sda5 errs: wr 1, rd 370, flush 0, corrupt 2, gen 0 > [ 1869.459381] BTRFS warning (device sda5): checksum error at logical 1271394304 on dev /dev/sda5, physical 1271394304, root 257, inode 8016535, offset 655360, length 4096, links 1 (path: var/log/journal/31d9cc011 > 2694cdab3887013fb7bbcf9/system@d8ae7586248c492d8d51c44efd7316da-0000000000000001-00056e3429e252bb.journal) > [ 1869.459383] BTRFS error (device sda5): bdev /dev/sda5 errs: wr 1, rd 370, flush 0, corrupt 3, gen 0 > [ 1869.460827] BTRFS warning (device sda5): checksum error at logical 1271398400 on dev /dev/sda5, physical 1271398400, root 257, inode 8016535, offset 655360, length 4096, links 1 (path: var/log/journal/31d9cc011 > 2694cdab3887013fb7bbcf9/system@d8ae7586248c492d8d51c44efd7316da-0000000000000001-00056e3429e252bb.journal) > [ 1869.460829] BTRFS error (device sda5): bdev /dev/sda5 errs: wr 1, rd 370, flush 0, corrupt 4, gen 0 > [ 1869.462118] BTRFS warning (device sda5): checksum error at logical 1271402496 on dev /dev/sda5, physical 1271402496, root 257, inode 8016535, offset 655360, length 4096, links 1 (path: var/log/journal/31d9cc011 > 2694cdab3887013fb7bbcf9/system@d8ae7586248c492d8d51c44efd7316da-0000000000000001-00056e3429e252bb.journal) > [ 1869.462120] BTRFS error (device sda5): bdev /dev/sda5 errs: wr 1, rd 370, flush 0, corrupt 5, gen 0 > > I don't really need that file, can I simply delete it and forget about > it or is there something else I should do? Of course you can delete it, and if nothing else went wrong, it should fix the problem. And of course, remove that offending memory stick. Thanks, Qu > > Yours, > Simon > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2018-06-12 1:18 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-06-09 12:22 corrupt leaf; unaligned key offset for csum item Simon Kaiser 2018-06-09 22:53 ` Chris Murphy 2018-06-10 1:56 ` Qu Wenruo 2018-06-10 3:53 ` Chris Murphy 2018-06-10 5:12 ` Qu Wenruo 2018-06-10 13:38 ` Simon Kaiser 2018-06-10 21:30 ` Chris Murphy 2018-06-10 22:32 ` Simon Kaiser 2018-06-11 8:10 ` Qu Wenruo 2018-06-11 16:42 ` Simon Kaiser 2018-06-12 1:18 ` Qu Wenruo
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).