* read time tree block corruption detected @ 2020-01-16 13:40 Peter Luladjiev 2020-01-16 16:12 ` Nikolay Borisov 0 siblings, 1 reply; 75+ messages in thread From: Peter Luladjiev @ 2020-01-16 13:40 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1175 bytes --] Hello, I'm having a "read time tree block corruption detected" error after upgrading to kernel 5.4, reverting to kernel 5.3 fixes the problem. I'm using openSuse Tumbleweed, btrfs-progs v5.4 Here is the dmesg output: [ 57.817776] BTRFS critical (device dm-1): corrupt leaf: block=52197507072 slot=74 extent bytenr=1497022464 len=4096 invalid extent refs, have 1 expect >= inline 3087007745 [ 57.817778] BTRFS error (device dm-1): block=52197507072 read time tree block corruption detected [ 57.817783] BTRFS: error (device dm-1) in btrfs_run_delayed_refs:2188: errno=-5 IO failure [ 57.817785] BTRFS info (device dm-1): forced readonly btrfs fi show Label: none uuid: 9639a3e6-cd08-4270-b4d1-d2946d2b8d2e Total devices 1 FS bytes used 49.85GiB devid 1 size 70.00GiB used 54.04GiB path /dev/mapper/system-root btrfs fi df / Data, single: total=52.01GiB, used=48.40GiB System, single: total=32.00MiB, used=16.00KiB Metadata, single: total=2.00GiB, used=1.44GiB GlobalReserve, single: total=123.92MiB, used=0.00B I'm attaching the output of btrfs ins dump-tree -b 52197507072 /dev/mapper/system-root What are my options? Thanks in advance! [-- Attachment #2: dump-tree.txt --] [-- Type: text/plain, Size: 27666 bytes --] btrfs-progs v5.4 leaf 52197507072 items 139 free space 4636 generation 538071 owner EXTENT_TREE leaf 52197507072 flags 0x1(WRITTEN) backref revision 1 fs uuid 9639a3e6-cd08-4270-b4d1-d2946d2b8d2e chunk uuid 004aeeba-bb0d-4915-b7c1-c15fb4745c4f item 0 key (1495171072 EXTENT_ITEM 8192) itemoff 16181 itemsize 102 refs 6 gen 500598 flags DATA shared data backref parent 51538460672 count 1 shared data backref parent 988725248 count 1 shared data backref parent 897351680 count 1 shared data backref parent 821608448 count 1 shared data backref parent 587972608 count 1 shared data backref parent 508870656 count 1 item 1 key (1495179264 EXTENT_ITEM 4096) itemoff 16092 itemsize 89 refs 5 gen 403758 flags DATA shared data backref parent 52216168448 count 1 shared data backref parent 52140294144 count 1 shared data backref parent 51774226432 count 1 shared data backref parent 980254720 count 1 shared data backref parent 218169344 count 1 item 2 key (1495183360 EXTENT_ITEM 8192) itemoff 16039 itemsize 53 refs 1 gen 292437 flags DATA extent data backref root 260 objectid 698805 offset 0 count 1 item 3 key (1495191552 EXTENT_ITEM 8192) itemoff 15921 itemsize 118 refs 6 gen 113455 flags DATA extent data backref root 1338 objectid 2699311 offset 0 count 1 shared data backref parent 52132085760 count 1 shared data backref parent 51829489664 count 1 shared data backref parent 51288211456 count 1 shared data backref parent 505987072 count 1 shared data backref parent 180977664 count 1 item 4 key (1495199744 EXTENT_ITEM 16384) itemoff 15816 itemsize 105 refs 5 gen 501446 flags DATA extent data backref root 1338 objectid 4646245 offset 0 count 1 shared data backref parent 51833847808 count 1 shared data backref parent 51359678464 count 1 shared data backref parent 936361984 count 1 shared data backref parent 415236096 count 1 item 5 key (1495216128 EXTENT_ITEM 12288) itemoff 15711 itemsize 105 refs 5 gen 501446 flags DATA extent data backref root 1338 objectid 4646255 offset 0 count 1 shared data backref parent 51833798656 count 1 shared data backref parent 51359760384 count 1 shared data backref parent 936394752 count 1 shared data backref parent 415219712 count 1 item 6 key (1495228416 EXTENT_ITEM 8192) itemoff 15658 itemsize 53 refs 1 gen 292032 flags DATA extent data backref root 258 objectid 1754582 offset 0 count 1 item 7 key (1495236608 EXTENT_ITEM 4096) itemoff 15621 itemsize 37 refs 1 gen 501446 flags DATA shared data backref parent 51376160768 count 1 item 8 key (1495240704 EXTENT_ITEM 12288) itemoff 15503 itemsize 118 refs 6 gen 113455 flags DATA extent data backref root 1338 objectid 2699310 offset 0 count 1 shared data backref parent 52132085760 count 1 shared data backref parent 51829489664 count 1 shared data backref parent 51288211456 count 1 shared data backref parent 505987072 count 1 shared data backref parent 180977664 count 1 item 9 key (1495252992 EXTENT_ITEM 20480) itemoff 15385 itemsize 118 refs 6 gen 113455 flags DATA extent data backref root 1338 objectid 2699312 offset 0 count 1 shared data backref parent 52132085760 count 1 shared data backref parent 51829489664 count 1 shared data backref parent 51288211456 count 1 shared data backref parent 505987072 count 1 shared data backref parent 180977664 count 1 item 10 key (1495273472 EXTENT_ITEM 8192) itemoff 15267 itemsize 118 refs 6 gen 113455 flags DATA extent data backref root 1338 objectid 2699313 offset 0 count 1 shared data backref parent 52132085760 count 1 shared data backref parent 51829489664 count 1 shared data backref parent 51288211456 count 1 shared data backref parent 505987072 count 1 shared data backref parent 180977664 count 1 item 11 key (1495281664 EXTENT_ITEM 8192) itemoff 15214 itemsize 53 refs 1 gen 521727 flags DATA extent data backref root 258 objectid 2177295 offset 0 count 1 item 12 key (1495289856 EXTENT_ITEM 77824) itemoff 15177 itemsize 37 refs 1 gen 501446 flags DATA shared data backref parent 51300941824 count 1 item 13 key (1495367680 EXTENT_ITEM 4096) itemoff 15124 itemsize 53 refs 1 gen 288275 flags DATA extent data backref root 260 objectid 680046 offset 0 count 1 item 14 key (1495371776 EXTENT_ITEM 4096) itemoff 15071 itemsize 53 refs 1 gen 288214 flags DATA extent data backref root 260 objectid 679425 offset 8192 count 1 item 15 key (1495375872 EXTENT_ITEM 24576) itemoff 15018 itemsize 53 refs 1 gen 288035 flags DATA extent data backref root 260 objectid 678605 offset 0 count 1 item 16 key (1495400448 EXTENT_ITEM 20480) itemoff 14965 itemsize 53 refs 1 gen 288035 flags DATA extent data backref root 260 objectid 678606 offset 0 count 1 item 17 key (1495420928 EXTENT_ITEM 24576) itemoff 14912 itemsize 53 refs 1 gen 288035 flags DATA extent data backref root 260 objectid 678607 offset 0 count 1 item 18 key (1495445504 EXTENT_ITEM 118784) itemoff 14859 itemsize 53 refs 5 gen 288212 flags DATA extent data backref root 260 objectid 679426 offset 0 count 5 item 19 key (1495564288 EXTENT_ITEM 65536) itemoff 14806 itemsize 53 refs 1 gen 288212 flags DATA extent data backref root 260 objectid 679425 offset 0 count 1 item 20 key (1495629824 EXTENT_ITEM 20480) itemoff 14753 itemsize 53 refs 1 gen 288213 flags DATA extent data backref root 260 objectid 679460 offset 0 count 1 item 21 key (1495650304 EXTENT_ITEM 4096) itemoff 14700 itemsize 53 refs 1 gen 288275 flags DATA extent data backref root 260 objectid 680098 offset 0 count 1 item 22 key (1495654400 EXTENT_ITEM 36864) itemoff 14663 itemsize 37 refs 1 gen 400951 flags DATA shared data backref parent 51916881920 count 1 item 23 key (1495691264 EXTENT_ITEM 12288) itemoff 14558 itemsize 105 refs 5 gen 501446 flags DATA extent data backref root 1338 objectid 4646266 offset 0 count 1 shared data backref parent 51833798656 count 1 shared data backref parent 51359760384 count 1 shared data backref parent 936394752 count 1 shared data backref parent 415219712 count 1 item 24 key (1495703552 EXTENT_ITEM 12288) itemoff 14521 itemsize 37 refs 1 gen 360665 flags DATA shared data backref parent 51275005952 count 1 item 25 key (1495715840 EXTENT_ITEM 8192) itemoff 14468 itemsize 53 refs 1 gen 422702 flags DATA extent data backref root 260 objectid 919536 offset 12288 count 1 item 26 key (1495724032 EXTENT_ITEM 4096) itemoff 14431 itemsize 37 refs 1 gen 501446 flags DATA shared data backref parent 51376160768 count 1 item 27 key (1495728128 EXTENT_ITEM 225280) itemoff 14378 itemsize 53 refs 1 gen 108502 flags DATA extent data backref root 260 objectid 530211 offset 0 count 1 item 28 key (1495953408 EXTENT_ITEM 36864) itemoff 14328 itemsize 50 refs 2 gen 270767 flags DATA shared data backref parent 51854098432 count 1 shared data backref parent 218349568 count 1 item 29 key (1495990272 EXTENT_ITEM 8192) itemoff 14275 itemsize 53 refs 1 gen 422705 flags DATA extent data backref root 260 objectid 919420 offset 0 count 1 item 30 key (1495998464 EXTENT_ITEM 12288) itemoff 14222 itemsize 53 refs 1 gen 472554 flags DATA extent data backref root 260 objectid 974375 offset 0 count 1 item 31 key (1496010752 EXTENT_ITEM 4096) itemoff 14169 itemsize 53 refs 1 gen 472554 flags DATA extent data backref root 260 objectid 974414 offset 0 count 1 item 32 key (1496014848 EXTENT_ITEM 8192) itemoff 14116 itemsize 53 refs 1 gen 292437 flags DATA extent data backref root 260 objectid 698839 offset 0 count 1 item 33 key (1496023040 EXTENT_ITEM 24576) itemoff 14079 itemsize 37 refs 1 gen 450713 flags DATA shared data backref parent 51439534080 count 1 item 34 key (1496047616 EXTENT_ITEM 8192) itemoff 14042 itemsize 37 refs 1 gen 450713 flags DATA shared data backref parent 51920568320 count 1 item 35 key (1496055808 EXTENT_ITEM 16384) itemoff 13937 itemsize 105 refs 5 gen 501446 flags DATA extent data backref root 1338 objectid 4646272 offset 0 count 1 shared data backref parent 51833798656 count 1 shared data backref parent 51359760384 count 1 shared data backref parent 936394752 count 1 shared data backref parent 415219712 count 1 item 36 key (1496072192 EXTENT_ITEM 12288) itemoff 13835 itemsize 102 refs 6 gen 500598 flags DATA shared data backref parent 51538460672 count 1 shared data backref parent 988725248 count 1 shared data backref parent 897351680 count 1 shared data backref parent 821608448 count 1 shared data backref parent 587972608 count 1 shared data backref parent 508870656 count 1 item 37 key (1496084480 EXTENT_ITEM 24576) itemoff 13782 itemsize 53 refs 1 gen 131540 flags DATA extent data backref root 260 objectid 567818 offset 0 count 1 item 38 key (1496109056 EXTENT_ITEM 94208) itemoff 13745 itemsize 37 refs 1 gen 501446 flags DATA shared data backref parent 51300515840 count 1 item 39 key (1496203264 EXTENT_ITEM 8192) itemoff 13708 itemsize 37 refs 1 gen 501446 flags DATA shared data backref parent 222543872 count 1 item 40 key (1496211456 EXTENT_ITEM 8192) itemoff 13655 itemsize 53 refs 1 gen 292437 flags DATA extent data backref root 260 objectid 698847 offset 0 count 1 item 41 key (1496219648 EXTENT_ITEM 20480) itemoff 13550 itemsize 105 refs 5 gen 501446 flags DATA extent data backref root 1338 objectid 4646246 offset 0 count 1 shared data backref parent 51833847808 count 1 shared data backref parent 51359678464 count 1 shared data backref parent 936361984 count 1 shared data backref parent 415236096 count 1 item 42 key (1496240128 EXTENT_ITEM 4096) itemoff 13513 itemsize 37 refs 1 gen 501446 flags DATA shared data backref parent 51376160768 count 1 item 43 key (1496244224 EXTENT_ITEM 8192) itemoff 13476 itemsize 37 refs 1 gen 392397 flags DATA shared data backref parent 1076494336 count 1 item 44 key (1496252416 EXTENT_ITEM 208896) itemoff 13423 itemsize 53 refs 1 gen 292028 flags DATA extent data backref root 258 objectid 1752577 offset 0 count 1 item 45 key (1496461312 EXTENT_ITEM 4096) itemoff 13370 itemsize 53 refs 1 gen 473090 flags DATA extent data backref root 260 objectid 984975 offset 0 count 1 item 46 key (1496465408 EXTENT_ITEM 8192) itemoff 13317 itemsize 53 refs 1 gen 473090 flags DATA extent data backref root 260 objectid 984976 offset 0 count 1 item 47 key (1496473600 EXTENT_ITEM 4096) itemoff 13280 itemsize 37 refs 1 gen 392397 flags DATA shared data backref parent 1077231616 count 1 item 48 key (1496477696 EXTENT_ITEM 28672) itemoff 13175 itemsize 105 refs 5 gen 501446 flags DATA extent data backref root 1338 objectid 4646256 offset 0 count 1 shared data backref parent 51833798656 count 1 shared data backref parent 51359760384 count 1 shared data backref parent 936394752 count 1 shared data backref parent 415219712 count 1 item 49 key (1496506368 EXTENT_ITEM 8192) itemoff 13122 itemsize 53 refs 1 gen 292437 flags DATA extent data backref root 260 objectid 698849 offset 0 count 1 item 50 key (1496514560 EXTENT_ITEM 8192) itemoff 13085 itemsize 37 refs 1 gen 435902 flags DATA shared data backref parent 52115750912 count 1 item 51 key (1496522752 EXTENT_ITEM 8192) itemoff 13048 itemsize 37 refs 1 gen 435902 flags DATA shared data backref parent 52115750912 count 1 item 52 key (1496530944 EXTENT_ITEM 86016) itemoff 13011 itemsize 37 refs 1 gen 501446 flags DATA shared data backref parent 51300515840 count 1 item 53 key (1496616960 EXTENT_ITEM 8192) itemoff 12974 itemsize 37 refs 1 gen 501446 flags DATA shared data backref parent 222543872 count 1 item 54 key (1496625152 EXTENT_ITEM 12288) itemoff 12739 itemsize 235 refs 15 gen 185300 flags DATA extent data backref root 1338 objectid 3452741 offset 0 count 1 shared data backref parent 52073168896 count 1 shared data backref parent 52042973184 count 1 shared data backref parent 51914752000 count 1 shared data backref parent 51798016000 count 1 shared data backref parent 51570212864 count 1 shared data backref parent 51487080448 count 1 shared data backref parent 51324616704 count 1 shared data backref parent 51299778560 count 1 shared data backref parent 997015552 count 1 shared data backref parent 904347648 count 1 shared data backref parent 550174720 count 1 shared data backref parent 388661248 count 1 shared data backref parent 223444992 count 1 shared data backref parent 103546880 count 1 item 55 key (1496637440 EXTENT_ITEM 143360) itemoff 12686 itemsize 53 refs 1 gen 527818 flags DATA extent data backref root 258 objectid 2242165 offset 0 count 1 item 56 key (1496780800 EXTENT_ITEM 12288) itemoff 12633 itemsize 53 refs 1 gen 537609 flags DATA extent data backref root 1338 objectid 4788704 offset 0 count 1 item 57 key (1496793088 EXTENT_ITEM 4096) itemoff 12583 itemsize 50 refs 2 gen 435905 flags DATA shared data backref parent 52144455680 count 1 shared data backref parent 51798720512 count 1 item 58 key (1496797184 EXTENT_ITEM 53248) itemoff 12530 itemsize 53 refs 1 gen 292030 flags DATA extent data backref root 258 objectid 1753023 offset 0 count 1 item 59 key (1496850432 EXTENT_ITEM 8192) itemoff 12477 itemsize 53 refs 1 gen 473090 flags DATA extent data backref root 260 objectid 984977 offset 0 count 1 item 60 key (1496858624 EXTENT_ITEM 53248) itemoff 12440 itemsize 37 refs 1 gen 400951 flags DATA shared data backref parent 70762496 count 1 item 61 key (1496911872 EXTENT_ITEM 8192) itemoff 12387 itemsize 53 refs 1 gen 537609 flags DATA extent data backref root 1338 objectid 4788723 offset 0 count 1 item 62 key (1496920064 EXTENT_ITEM 4096) itemoff 12350 itemsize 37 refs 1 gen 400938 flags DATA shared data backref parent 51236257792 count 1 item 63 key (1496924160 EXTENT_ITEM 8192) itemoff 12300 itemsize 50 refs 2 gen 138497 flags DATA shared data backref parent 51306774528 count 1 shared data backref parent 1033256960 count 1 item 64 key (1496932352 EXTENT_ITEM 8192) itemoff 12247 itemsize 53 refs 1 gen 392402 flags DATA extent data backref root 258 objectid 4159 offset 1048576 count 1 item 65 key (1496940544 EXTENT_ITEM 4096) itemoff 12194 itemsize 53 refs 1 gen 394519 flags DATA extent data backref root 258 objectid 46599 offset 742244352 count 1 item 66 key (1496944640 EXTENT_ITEM 8192) itemoff 12141 itemsize 53 refs 1 gen 423224 flags DATA extent data backref root 260 objectid 920886 offset 0 count 1 item 67 key (1496952832 EXTENT_ITEM 36864) itemoff 11900 itemsize 241 refs 13 gen 400951 flags DATA extent data backref root 1338 objectid 4284886 offset 0 count 1 extent data backref root 1310 objectid 4284886 offset 0 count 1 extent data backref root 1312 objectid 4284886 offset 0 count 1 shared data backref parent 52220051456 count 1 shared data backref parent 51942096896 count 1 shared data backref parent 51859947520 count 1 shared data backref parent 51690700800 count 1 shared data backref parent 51331710976 count 1 shared data backref parent 386252800 count 1 shared data backref parent 304005120 count 1 shared data backref parent 247447552 count 1 shared data backref parent 71696384 count 1 shared data backref parent 70762496 count 1 item 68 key (1496989696 EXTENT_ITEM 12288) itemoff 11847 itemsize 53 refs 1 gen 292549 flags DATA extent data backref root 260 objectid 701302 offset 0 count 1 item 69 key (1497001984 EXTENT_ITEM 4096) itemoff 11794 itemsize 53 refs 1 gen 292555 flags DATA extent data backref root 260 objectid 701342 offset 0 count 1 item 70 key (1497006080 EXTENT_ITEM 4096) itemoff 11757 itemsize 37 refs 1 gen 17889 flags DATA shared data backref parent 51611369472 count 1 item 71 key (1497010176 EXTENT_ITEM 4096) itemoff 11720 itemsize 37 refs 1 gen 17890 flags DATA shared data backref parent 51611369472 count 1 item 72 key (1497014272 EXTENT_ITEM 4096) itemoff 11683 itemsize 37 refs 1 gen 17892 flags DATA shared data backref parent 51611369472 count 1 item 73 key (1497018368 EXTENT_ITEM 4096) itemoff 11646 itemsize 37 refs 72057183177116417 gen 17894 flags DATA shared data backref parent 51611369472 count 1 item 74 key (1497022464 EXTENT_ITEM 4096) itemoff 11609 itemsize 37 refs 1 gen 17896 flags DATA shared data backref parent 51611369472 count 3087007745 item 75 key (1497026560 EXTENT_ITEM 32768) itemoff 11572 itemsize 37 refs 1 gen 450713 flags DATA shared data backref parent 51439534080 count 1 item 76 key (1497059328 EXTENT_ITEM 4096) itemoff 11535 itemsize 37 refs 1 gen 450713 flags DATA shared data backref parent 51455557632 count 1 item 77 key (1497063424 EXTENT_ITEM 229376) itemoff 11482 itemsize 53 refs 1 gen 58759 flags DATA extent data backref root 260 objectid 278424 offset 0 count 1 item 78 key (1497292800 EXTENT_ITEM 8192) itemoff 11380 itemsize 102 refs 6 gen 500598 flags DATA shared data backref parent 51538460672 count 1 shared data backref parent 988725248 count 1 shared data backref parent 897351680 count 1 shared data backref parent 821608448 count 1 shared data backref parent 587972608 count 1 shared data backref parent 508870656 count 1 item 79 key (1497300992 EXTENT_ITEM 4096) itemoff 11291 itemsize 89 refs 5 gen 403758 flags DATA shared data backref parent 52216168448 count 1 shared data backref parent 52140294144 count 1 shared data backref parent 51774226432 count 1 shared data backref parent 980254720 count 1 shared data backref parent 218169344 count 1 item 80 key (1497305088 EXTENT_ITEM 147456) itemoff 11238 itemsize 53 refs 1 gen 111262 flags DATA extent data backref root 258 objectid 994234 offset 0 count 1 item 81 key (1497452544 EXTENT_ITEM 61440) itemoff 11120 itemsize 118 refs 6 gen 400951 flags DATA extent data backref root 1338 objectid 4284778 offset 0 count 1 shared data backref parent 52106543104 count 1 shared data backref parent 277561344 count 1 shared data backref parent 252542976 count 1 shared data backref parent 169377792 count 1 shared data backref parent 28966912 count 1 item 82 key (1497513984 EXTENT_ITEM 4096) itemoff 11083 itemsize 37 refs 1 gen 501446 flags DATA shared data backref parent 51376160768 count 1 item 83 key (1497518080 EXTENT_ITEM 12288) itemoff 11046 itemsize 37 refs 1 gen 400938 flags DATA shared data backref parent 51233325056 count 1 item 84 key (1497530368 EXTENT_ITEM 8192) itemoff 10993 itemsize 53 refs 1 gen 292437 flags DATA extent data backref root 260 objectid 698850 offset 0 count 1 item 85 key (1497538560 EXTENT_ITEM 24576) itemoff 10888 itemsize 105 refs 5 gen 501446 flags DATA extent data backref root 1338 objectid 4646264 offset 0 count 1 shared data backref parent 51833798656 count 1 shared data backref parent 51359760384 count 1 shared data backref parent 936394752 count 1 shared data backref parent 415219712 count 1 item 86 key (1497563136 EXTENT_ITEM 4096) itemoff 10851 itemsize 37 refs 1 gen 501446 flags DATA shared data backref parent 51376160768 count 1 item 87 key (1497567232 EXTENT_ITEM 4096) itemoff 10798 itemsize 53 refs 1 gen 292555 flags DATA extent data backref root 260 objectid 701344 offset 0 count 1 item 88 key (1497571328 EXTENT_ITEM 4096) itemoff 10761 itemsize 37 refs 1 gen 501446 flags DATA shared data backref parent 51376209920 count 1 item 89 key (1497575424 EXTENT_ITEM 16384) itemoff 10708 itemsize 53 refs 1 gen 116513 flags DATA extent data backref root 258 objectid 932671 offset 4128768 count 1 item 90 key (1497591808 EXTENT_ITEM 4096) itemoff 10655 itemsize 53 refs 1 gen 292555 flags DATA extent data backref root 260 objectid 701345 offset 0 count 1 item 91 key (1497595904 EXTENT_ITEM 4096) itemoff 10602 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70213 offset 0 count 1 item 92 key (1497600000 EXTENT_ITEM 8192) itemoff 10549 itemsize 53 refs 1 gen 15911 flags DATA extent data backref root 260 objectid 73333 offset 12288 count 1 item 93 key (1497608192 EXTENT_ITEM 4096) itemoff 10496 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70218 offset 0 count 1 item 94 key (1497612288 EXTENT_ITEM 12288) itemoff 10443 itemsize 53 refs 1 gen 15911 flags DATA extent data backref root 260 objectid 73263 offset 139264 count 1 item 95 key (1497624576 EXTENT_ITEM 16384) itemoff 10390 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70208 offset 0 count 1 item 96 key (1497640960 EXTENT_ITEM 8192) itemoff 10337 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70214 offset 0 count 1 item 97 key (1497649152 EXTENT_ITEM 8192) itemoff 10284 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70215 offset 0 count 1 item 98 key (1497657344 EXTENT_ITEM 8192) itemoff 10231 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70216 offset 0 count 1 item 99 key (1497665536 EXTENT_ITEM 8192) itemoff 10178 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70217 offset 0 count 1 item 100 key (1497673728 EXTENT_ITEM 57344) itemoff 10125 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70220 offset 0 count 1 item 101 key (1497731072 EXTENT_ITEM 8192) itemoff 10072 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70223 offset 0 count 1 item 102 key (1497739264 EXTENT_ITEM 8192) itemoff 10019 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70224 offset 0 count 1 item 103 key (1497747456 EXTENT_ITEM 8192) itemoff 9966 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70227 offset 0 count 1 item 104 key (1497755648 EXTENT_ITEM 24576) itemoff 9913 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70228 offset 0 count 1 item 105 key (1497780224 EXTENT_ITEM 24576) itemoff 9860 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 70229 offset 0 count 1 item 106 key (1497804800 EXTENT_ITEM 12288) itemoff 9807 itemsize 53 refs 1 gen 15914 flags DATA extent data backref root 260 objectid 73718 offset 0 count 1 item 107 key (1497817088 EXTENT_ITEM 4096) itemoff 9754 itemsize 53 refs 1 gen 15918 flags DATA extent data backref root 260 objectid 73703 offset 45056 count 1 item 108 key (1497821184 EXTENT_ITEM 8192) itemoff 9701 itemsize 53 refs 1 gen 15796 flags DATA extent data backref root 260 objectid 69911 offset 12288 count 1 item 109 key (1497829376 EXTENT_ITEM 12288) itemoff 9648 itemsize 53 refs 1 gen 15910 flags DATA extent data backref root 260 objectid 73263 offset 57344 count 1 item 110 key (1497841664 EXTENT_ITEM 12288) itemoff 9595 itemsize 53 refs 1 gen 15911 flags DATA extent data backref root 260 objectid 73259 offset 0 count 1 item 111 key (1497853952 EXTENT_ITEM 4096) itemoff 9542 itemsize 53 refs 1 gen 15911 flags DATA extent data backref root 260 objectid 73259 offset 122880 count 1 item 112 key (1497858048 EXTENT_ITEM 8192) itemoff 9489 itemsize 53 refs 1 gen 15910 flags DATA extent data backref root 260 objectid 73332 offset 0 count 1 item 113 key (1497866240 EXTENT_ITEM 8192) itemoff 9436 itemsize 53 refs 1 gen 15910 flags DATA extent data backref root 260 objectid 73349 offset 0 count 1 item 114 key (1497874432 EXTENT_ITEM 12288) itemoff 9383 itemsize 53 refs 1 gen 15910 flags DATA extent data backref root 260 objectid 73350 offset 0 count 1 item 115 key (1497886720 EXTENT_ITEM 8192) itemoff 9330 itemsize 53 refs 1 gen 15910 flags DATA extent data backref root 260 objectid 73351 offset 0 count 1 item 116 key (1497894912 EXTENT_ITEM 126976) itemoff 9277 itemsize 53 refs 1 gen 15854 flags DATA extent data backref root 260 objectid 70597 offset 0 count 1 item 117 key (1498021888 EXTENT_ITEM 24576) itemoff 9224 itemsize 53 refs 1 gen 15911 flags DATA extent data backref root 260 objectid 73263 offset 106496 count 1 item 118 key (1498046464 EXTENT_ITEM 24576) itemoff 9171 itemsize 53 refs 1 gen 15911 flags DATA extent data backref root 260 objectid 73672 offset 0 count 1 item 119 key (1498071040 EXTENT_ITEM 8192) itemoff 9118 itemsize 53 refs 1 gen 422730 flags DATA extent data backref root 260 objectid 920176 offset 0 count 1 item 120 key (1498079232 EXTENT_ITEM 131072) itemoff 9065 itemsize 53 refs 1 gen 15910 flags DATA extent data backref root 260 objectid 73342 offset 0 count 1 item 121 key (1498210304 EXTENT_ITEM 8192) itemoff 9012 itemsize 53 refs 1 gen 15910 flags DATA extent data backref root 260 objectid 73352 offset 0 count 1 item 122 key (1498218496 EXTENT_ITEM 4096) itemoff 8959 itemsize 53 refs 1 gen 15911 flags DATA extent data backref root 260 objectid 73263 offset 0 count 1 item 123 key (1498222592 EXTENT_ITEM 4096) itemoff 8906 itemsize 53 refs 1 gen 15918 flags DATA extent data backref root 260 objectid 73703 offset 57344 count 1 item 124 key (1498226688 EXTENT_ITEM 8192) itemoff 8853 itemsize 53 refs 1 gen 15910 flags DATA extent data backref root 260 objectid 73273 offset 0 count 1 item 125 key (1498234880 EXTENT_ITEM 8192) itemoff 8800 itemsize 53 refs 1 gen 15911 flags DATA extent data backref root 260 objectid 73263 offset 32768 count 1 item 126 key (1498243072 EXTENT_ITEM 8192) itemoff 8747 itemsize 53 refs 1 gen 15916 flags DATA extent data backref root 260 objectid 73777 offset 12288 count 1 item 127 key (1498251264 EXTENT_ITEM 12288) itemoff 8694 itemsize 53 refs 1 gen 15910 flags DATA extent data backref root 260 objectid 73272 offset 12288 count 1 item 128 key (1498263552 EXTENT_ITEM 12288) itemoff 8641 itemsize 53 refs 1 gen 15914 flags DATA extent data backref root 260 objectid 73726 offset 0 count 1 item 129 key (1498275840 EXTENT_ITEM 24576) itemoff 8588 itemsize 53 refs 1 gen 15910 flags DATA extent data backref root 260 objectid 73272 offset 65536 count 1 item 130 key (1498300416 EXTENT_ITEM 16384) itemoff 8535 itemsize 53 refs 1 gen 15910 flags DATA extent data backref root 260 objectid 73278 offset 4096 count 1 item 131 key (1498316800 EXTENT_ITEM 20480) itemoff 8482 itemsize 53 refs 1 gen 15914 flags DATA extent data backref root 260 objectid 73697 offset 0 count 1 item 132 key (1498337280 EXTENT_ITEM 20480) itemoff 8429 itemsize 53 refs 1 gen 15914 flags DATA extent data backref root 260 objectid 73733 offset 0 count 1 item 133 key (1498357760 EXTENT_ITEM 20480) itemoff 8376 itemsize 53 refs 1 gen 15914 flags DATA extent data backref root 260 objectid 73735 offset 0 count 1 item 134 key (1498378240 EXTENT_ITEM 12288) itemoff 8323 itemsize 53 refs 1 gen 15918 flags DATA extent data backref root 260 objectid 73703 offset 0 count 1 item 135 key (1498390528 EXTENT_ITEM 4096) itemoff 8270 itemsize 53 refs 1 gen 15918 flags DATA extent data backref root 260 objectid 73703 offset 69632 count 1 item 136 key (1498394624 EXTENT_ITEM 24576) itemoff 8217 itemsize 53 refs 1 gen 15912 flags DATA extent data backref root 260 objectid 73694 offset 0 count 1 item 137 key (1498419200 EXTENT_ITEM 28672) itemoff 8164 itemsize 53 refs 1 gen 15912 flags DATA extent data backref root 260 objectid 73695 offset 0 count 1 item 138 key (1498447872 EXTENT_ITEM 20480) itemoff 8111 itemsize 53 refs 1 gen 15916 flags DATA extent data backref root 260 objectid 73764 offset 0 count 1 ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2020-01-16 13:40 read time tree block corruption detected Peter Luladjiev @ 2020-01-16 16:12 ` Nikolay Borisov [not found] ` <CA+ZCqs5=N5Hdf3NxZAmPCnA8wbcJPrcH8zM-fRbt-w8tL+TjUQ@mail.gmail.com> 0 siblings, 1 reply; 75+ messages in thread From: Nikolay Borisov @ 2020-01-16 16:12 UTC (permalink / raw) To: Peter Luladjiev, linux-btrfs On 16.01.20 г. 15:40 ч., Peter Luladjiev wrote: > Hello, > > I'm having a "read time tree block corruption detected" error after > upgrading to kernel 5.4, reverting to kernel 5.3 fixes the problem. > I'm using openSuse Tumbleweed, btrfs-progs v5.4 The reason is that the read-time validator got added in v5.4. Looking at the provided dump: item 70 key (1497006080 EXTENT_ITEM 4096) itemoff 11757 itemsize 37 refs 1 gen 17889 flags DATA shared data backref parent 51611369472 count 1 item 71 key (1497010176 EXTENT_ITEM 4096) itemoff 11720 itemsize 37 refs 1 gen 17890 flags DATA shared data backref parent 51611369472 count 1 item 72 key (1497014272 EXTENT_ITEM 4096) itemoff 11683 itemsize 37 refs 1 gen 17892 flags DATA shared data backref parent 51611369472 count 1 item 73 key (1497018368 EXTENT_ITEM 4096) itemoff 11646 itemsize 37 refs 72057183177116417 gen 17894 flags DATA shared data backref parent 51611369472 count 1 item 74 key (1497022464 EXTENT_ITEM 4096) itemoff 11609 itemsize 37 refs 1 gen 17896 flags DATA shared data backref parent 51611369472 count 3087007745 Clearly the problematic extent has an abnormal count: 3087007745, in binary this makes: 10111000000000000000000000000001 so the topmost 5 bits (apart from 1) has been set to 1 whereas I'd assume only the LSB (1) should be set, judging based on the the rest of the extents that have 51611369472 as their parent. Can you run btrfs check on /dev/mapper/system-root and post the log here. Note this will be a read-only operation to you don't risk breaking anything. You might also have to set the --force option since this looks like to be your root fs which is always mounted (barring working from a rescue disk). ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <CA+ZCqs5=N5Hdf3NxZAmPCnA8wbcJPrcH8zM-fRbt-w8tL+TjUQ@mail.gmail.com>]
* Re: read time tree block corruption detected [not found] ` <CA+ZCqs5=N5Hdf3NxZAmPCnA8wbcJPrcH8zM-fRbt-w8tL+TjUQ@mail.gmail.com> @ 2020-01-17 7:34 ` Nikolay Borisov 2020-01-17 7:51 ` Peter Luladjiev 0 siblings, 1 reply; 75+ messages in thread From: Nikolay Borisov @ 2020-01-17 7:34 UTC (permalink / raw) To: Peter Luladjiev; +Cc: linux-btrfs On 16.01.20 г. 18:53 ч., Peter Luladjiev wrote: > Hello, thanks for helping, here is the output: > > btrfs check --force /dev/mapper/system-root > > Opening filesystem to check... > WARNING: filesystem mounted, continuing because of --force > Checking filesystem on /dev/mapper/system-root > UUID: 9639a3e6-cd08-4270-b4d1-d2946d2b8d2e > [1/7] checking root items > [2/7] checking extents > ref mismatch on [1497018368 4096] extent item 72057183177116417, found 1 > incorrect local backref count on 1497022464 parent 51611369472 owner 0 > offset 0 found 1 wanted 3087007745 back 0x564582174c70 > backpointer mismatch on [1497022464 4096] > ERROR: errors found in extent allocation tree or chunk allocation > [3/7] checking free space cache > [4/7] checking fs roots > [5/7] checking only csums items (without verifying data) > [6/7] checking root refs > [7/7] checking quota groups skipped (not enabled on this FS) > found 53532647424 bytes used, error(s) found > total csum bytes: 43476204 > total tree bytes: 1551368192 > total fs tree bytes: 1421295616 > total extent tree bytes: 70238208 > btree space waste bytes: 276638054 > file data blocks allocated: 331679563776 > referenced 105423634432 > Right so it seems this the only error. Just to be sure run btrfs check --mode lowmem since it provides more readable output. If this is the only error in the filesystem then btrfs check --repair --mode lowmem should be able to fix it. ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2020-01-17 7:34 ` Nikolay Borisov @ 2020-01-17 7:51 ` Peter Luladjiev 2020-01-17 7:54 ` Peter Luladjiev 0 siblings, 1 reply; 75+ messages in thread From: Peter Luladjiev @ 2020-01-17 7:51 UTC (permalink / raw) To: Nikolay Borisov; +Cc: linux-btrfs Here is the output: btrfs check --force --mode lowmem /dev/mapper/system-root Opening filesystem to check... WARNING: filesystem mounted, continuing because of --force Checking filesystem on /dev/mapper/system-root UUID: 9639a3e6-cd08-4270-b4d1-d2946d2b8d2e [1/7] checking root items [2/7] checking extents [3/7] checking free space cache btrfs: space cache generation (539645) does not match inode (539641) failed to load free space cache for block group 22020096 btrfs: space cache generation (539645) does not match inode (539641) failed to load free space cache for block group 1095761920 btrfs: space cache generation (539643) does not match inode (539640) failed to load free space cache for block group 102161711104 [4/7] checking fs roots [5/7] checking only csums items (without verifying data) [6/7] checking root refs done with fs roots in lowmem mode, skipping [7/7] checking quota groups skipped (not enabled on this FS) found 53501751296 bytes used, no error found total csum bytes: 43476196 total tree bytes: 1552203776 total fs tree bytes: 1422196736 total extent tree bytes: 70172672 btree space waste bytes: 276902557 file data blocks allocated: 331882188800 referenced 105424904192 On Fri, 17 Jan 2020 at 09:34, Nikolay Borisov <nborisov@suse.com> wrote: > > > > On 16.01.20 г. 18:53 ч., Peter Luladjiev wrote: > > Hello, thanks for helping, here is the output: > > > > btrfs check --force /dev/mapper/system-root > > > > Opening filesystem to check... > > WARNING: filesystem mounted, continuing because of --force > > Checking filesystem on /dev/mapper/system-root > > UUID: 9639a3e6-cd08-4270-b4d1-d2946d2b8d2e > > [1/7] checking root items > > [2/7] checking extents > > ref mismatch on [1497018368 4096] extent item 72057183177116417, found 1 > > incorrect local backref count on 1497022464 parent 51611369472 owner 0 > > offset 0 found 1 wanted 3087007745 back 0x564582174c70 > > backpointer mismatch on [1497022464 4096] > > ERROR: errors found in extent allocation tree or chunk allocation > > [3/7] checking free space cache > > [4/7] checking fs roots > > [5/7] checking only csums items (without verifying data) > > [6/7] checking root refs > > [7/7] checking quota groups skipped (not enabled on this FS) > > found 53532647424 bytes used, error(s) found > > total csum bytes: 43476204 > > total tree bytes: 1551368192 > > total fs tree bytes: 1421295616 > > total extent tree bytes: 70238208 > > btree space waste bytes: 276638054 > > file data blocks allocated: 331679563776 > > referenced 105423634432 > > > > > Right so it seems this the only error. Just to be sure run btrfs check > --mode lowmem since it provides more readable output. If this is the > only error in the filesystem then btrfs check --repair --mode lowmem > should be able to fix it. ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2020-01-17 7:51 ` Peter Luladjiev @ 2020-01-17 7:54 ` Peter Luladjiev 2020-01-17 7:59 ` Qu Wenruo 2020-01-17 8:14 ` Nikolay Borisov 0 siblings, 2 replies; 75+ messages in thread From: Peter Luladjiev @ 2020-01-17 7:54 UTC (permalink / raw) To: Nikolay Borisov; +Cc: linux-btrfs Should I run with repair flag? On Fri, 17 Jan 2020 at 09:51, Peter Luladjiev <luladjiev@gmail.com> wrote: > > Here is the output: > > btrfs check --force --mode lowmem /dev/mapper/system-root > > Opening filesystem to check... > WARNING: filesystem mounted, continuing because of --force > Checking filesystem on /dev/mapper/system-root > UUID: 9639a3e6-cd08-4270-b4d1-d2946d2b8d2e > [1/7] checking root items > [2/7] checking extents > [3/7] checking free space cache > btrfs: space cache generation (539645) does not match inode (539641) > failed to load free space cache for block group 22020096 > btrfs: space cache generation (539645) does not match inode (539641) > failed to load free space cache for block group 1095761920 > btrfs: space cache generation (539643) does not match inode (539640) > failed to load free space cache for block group 102161711104 > [4/7] checking fs roots > [5/7] checking only csums items (without verifying data) > [6/7] checking root refs done with fs roots in lowmem mode, skipping > [7/7] checking quota groups skipped (not enabled on this FS) > found 53501751296 bytes used, no error found > total csum bytes: 43476196 > total tree bytes: 1552203776 > total fs tree bytes: 1422196736 > total extent tree bytes: 70172672 > btree space waste bytes: 276902557 > file data blocks allocated: 331882188800 > referenced 105424904192 > > On Fri, 17 Jan 2020 at 09:34, Nikolay Borisov <nborisov@suse.com> wrote: > > > > > > > > On 16.01.20 г. 18:53 ч., Peter Luladjiev wrote: > > > Hello, thanks for helping, here is the output: > > > > > > btrfs check --force /dev/mapper/system-root > > > > > > Opening filesystem to check... > > > WARNING: filesystem mounted, continuing because of --force > > > Checking filesystem on /dev/mapper/system-root > > > UUID: 9639a3e6-cd08-4270-b4d1-d2946d2b8d2e > > > [1/7] checking root items > > > [2/7] checking extents > > > ref mismatch on [1497018368 4096] extent item 72057183177116417, found 1 > > > incorrect local backref count on 1497022464 parent 51611369472 owner 0 > > > offset 0 found 1 wanted 3087007745 back 0x564582174c70 > > > backpointer mismatch on [1497022464 4096] > > > ERROR: errors found in extent allocation tree or chunk allocation > > > [3/7] checking free space cache > > > [4/7] checking fs roots > > > [5/7] checking only csums items (without verifying data) > > > [6/7] checking root refs > > > [7/7] checking quota groups skipped (not enabled on this FS) > > > found 53532647424 bytes used, error(s) found > > > total csum bytes: 43476204 > > > total tree bytes: 1551368192 > > > total fs tree bytes: 1421295616 > > > total extent tree bytes: 70238208 > > > btree space waste bytes: 276638054 > > > file data blocks allocated: 331679563776 > > > referenced 105423634432 > > > > > > > > > Right so it seems this the only error. Just to be sure run btrfs check > > --mode lowmem since it provides more readable output. If this is the > > only error in the filesystem then btrfs check --repair --mode lowmem > > should be able to fix it. ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2020-01-17 7:54 ` Peter Luladjiev @ 2020-01-17 7:59 ` Qu Wenruo 2020-01-17 8:14 ` Nikolay Borisov 1 sibling, 0 replies; 75+ messages in thread From: Qu Wenruo @ 2020-01-17 7:59 UTC (permalink / raw) To: Peter Luladjiev, Nikolay Borisov; +Cc: linux-btrfs On 2020/1/17 下午3:54, Peter Luladjiev wrote: > Should I run with repair flag? Please only run --repair with the fs unmounted. Thanks, Qu > > On Fri, 17 Jan 2020 at 09:51, Peter Luladjiev <luladjiev@gmail.com> wrote: >> >> Here is the output: >> >> btrfs check --force --mode lowmem /dev/mapper/system-root >> >> Opening filesystem to check... >> WARNING: filesystem mounted, continuing because of --force >> Checking filesystem on /dev/mapper/system-root >> UUID: 9639a3e6-cd08-4270-b4d1-d2946d2b8d2e >> [1/7] checking root items >> [2/7] checking extents >> [3/7] checking free space cache >> btrfs: space cache generation (539645) does not match inode (539641) >> failed to load free space cache for block group 22020096 >> btrfs: space cache generation (539645) does not match inode (539641) >> failed to load free space cache for block group 1095761920 >> btrfs: space cache generation (539643) does not match inode (539640) >> failed to load free space cache for block group 102161711104 >> [4/7] checking fs roots >> [5/7] checking only csums items (without verifying data) >> [6/7] checking root refs done with fs roots in lowmem mode, skipping >> [7/7] checking quota groups skipped (not enabled on this FS) >> found 53501751296 bytes used, no error found >> total csum bytes: 43476196 >> total tree bytes: 1552203776 >> total fs tree bytes: 1422196736 >> total extent tree bytes: 70172672 >> btree space waste bytes: 276902557 >> file data blocks allocated: 331882188800 >> referenced 105424904192 >> >> On Fri, 17 Jan 2020 at 09:34, Nikolay Borisov <nborisov@suse.com> wrote: >>> >>> >>> >>> On 16.01.20 г. 18:53 ч., Peter Luladjiev wrote: >>>> Hello, thanks for helping, here is the output: >>>> >>>> btrfs check --force /dev/mapper/system-root >>>> >>>> Opening filesystem to check... >>>> WARNING: filesystem mounted, continuing because of --force >>>> Checking filesystem on /dev/mapper/system-root >>>> UUID: 9639a3e6-cd08-4270-b4d1-d2946d2b8d2e >>>> [1/7] checking root items >>>> [2/7] checking extents >>>> ref mismatch on [1497018368 4096] extent item 72057183177116417, found 1 >>>> incorrect local backref count on 1497022464 parent 51611369472 owner 0 >>>> offset 0 found 1 wanted 3087007745 back 0x564582174c70 >>>> backpointer mismatch on [1497022464 4096] >>>> ERROR: errors found in extent allocation tree or chunk allocation >>>> [3/7] checking free space cache >>>> [4/7] checking fs roots >>>> [5/7] checking only csums items (without verifying data) >>>> [6/7] checking root refs >>>> [7/7] checking quota groups skipped (not enabled on this FS) >>>> found 53532647424 bytes used, error(s) found >>>> total csum bytes: 43476204 >>>> total tree bytes: 1551368192 >>>> total fs tree bytes: 1421295616 >>>> total extent tree bytes: 70238208 >>>> btree space waste bytes: 276638054 >>>> file data blocks allocated: 331679563776 >>>> referenced 105423634432 >>>> >>> >>> >>> Right so it seems this the only error. Just to be sure run btrfs check >>> --mode lowmem since it provides more readable output. If this is the >>> only error in the filesystem then btrfs check --repair --mode lowmem >>> should be able to fix it. ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2020-01-17 7:54 ` Peter Luladjiev 2020-01-17 7:59 ` Qu Wenruo @ 2020-01-17 8:14 ` Nikolay Borisov 2020-01-17 8:22 ` Peter Luladjiev 1 sibling, 1 reply; 75+ messages in thread From: Nikolay Borisov @ 2020-01-17 8:14 UTC (permalink / raw) To: Peter Luladjiev; +Cc: linux-btrfs On 17.01.20 г. 9:54 ч., Peter Luladjiev wrote: > Should I run with repair flag? > Which version of progs do you have ? Because it seems original mode (Default) finds the corrupted backref but lowmem doesn't ? ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2020-01-17 8:14 ` Nikolay Borisov @ 2020-01-17 8:22 ` Peter Luladjiev 2020-01-17 9:10 ` Nikolay Borisov 0 siblings, 1 reply; 75+ messages in thread From: Peter Luladjiev @ 2020-01-17 8:22 UTC (permalink / raw) To: Nikolay Borisov; +Cc: linux-btrfs btrfs-progs v5.4 On Fri, 17 Jan 2020 at 10:14, Nikolay Borisov <nborisov@suse.com> wrote: > > > > On 17.01.20 г. 9:54 ч., Peter Luladjiev wrote: > > Should I run with repair flag? > > > > > Which version of progs do you have ? Because it seems original mode > (Default) finds the corrupted backref but lowmem doesn't ? > > > ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2020-01-17 8:22 ` Peter Luladjiev @ 2020-01-17 9:10 ` Nikolay Borisov 2020-01-17 12:04 ` Peter Luladjiev 0 siblings, 1 reply; 75+ messages in thread From: Nikolay Borisov @ 2020-01-17 9:10 UTC (permalink / raw) To: Peter Luladjiev; +Cc: linux-btrfs On 17.01.20 г. 10:22 ч., Peter Luladjiev wrote: > btrfs-progs v5.4 As Qu mentioned - with the rootfs unmounted you can run btrfs check --repair and it should repair that one broken backref. > > On Fri, 17 Jan 2020 at 10:14, Nikolay Borisov <nborisov@suse.com> wrote: >> >> >> >> On 17.01.20 г. 9:54 ч., Peter Luladjiev wrote: >>> Should I run with repair flag? >>> >> >> >> Which version of progs do you have ? Because it seems original mode >> (Default) finds the corrupted backref but lowmem doesn't ? >> >> >> ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2020-01-17 9:10 ` Nikolay Borisov @ 2020-01-17 12:04 ` Peter Luladjiev 0 siblings, 0 replies; 75+ messages in thread From: Peter Luladjiev @ 2020-01-17 12:04 UTC (permalink / raw) To: Nikolay Borisov; +Cc: linux-btrfs Thank you very much. Everything seems fine now. On Fri, 17 Jan 2020 at 11:10, Nikolay Borisov <nborisov@suse.com> wrote: > > > > On 17.01.20 г. 10:22 ч., Peter Luladjiev wrote: > > btrfs-progs v5.4 > > As Qu mentioned - with the rootfs unmounted you can run btrfs check > --repair and it should repair that one broken backref. > > > > > On Fri, 17 Jan 2020 at 10:14, Nikolay Borisov <nborisov@suse.com> wrote: > >> > >> > >> > >> On 17.01.20 г. 9:54 ч., Peter Luladjiev wrote: > >>> Should I run with repair flag? > >>> > >> > >> > >> Which version of progs do you have ? Because it seems original mode > >> (Default) finds the corrupted backref but lowmem doesn't ? > >> > >> > >> ^ permalink raw reply [flat|nested] 75+ messages in thread
* read time tree block corruption detected @ 2021-11-22 5:26 x8062 2021-11-22 7:24 ` Nikolay Borisov 0 siblings, 1 reply; 75+ messages in thread From: x8062 @ 2021-11-22 5:26 UTC (permalink / raw) To: linux-btrfs Hello, I got periodic warns in my linux console. in dmesg it is the following pasted text. At https://btrfs.wiki.kernel.org/index.php/Tree-checker I learned it may be a error, so i send the message. Hopefully it could help, Thanks in advance! [ 513.900852] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 [ 513.900869] BTRFS error (device sdb3): block=71928348672 read time tree block corruption detected [ 513.901534] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 [ 513.901550] BTRFS error (device sdb3): block=71928348672 read time tree block corruption detected [ 513.912414] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 [ 513.912433] BTRFS error (device sdb3): block=71928348672 read time tree block corruption detected [ 611.835932] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 [ 611.835950] BTRFS error (device sdb3): block=71928348672 read time tree block corruption detected [ 611.836594] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 [ 611.836603] BTRFS error (device sdb3): block=71928348672 read time tree block corruption detected [ 611.847484] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 [ 611.847500] BTRFS error (device sdb3): block=71928348672 read time tree block corruption detected [ 708.354174] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 [ 708.354186] BTRFS error (device sdb3): block=71928348672 read time tree block corruption detected [ 708.354833] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 [ 708.354841] BTRFS error (device sdb3): block=71928348672 read time tree block corruption detected [ 708.357409] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 [ 708.357420] BTRFS error (device sdb3): block=71928348672 read time tree block corruption detected BTW: my mount option is (rw,relatime,space_cache,commit=90,subvolid=5,subvol=/), I'm using btrfs upon /dev/sdb3 and /dev/sda4, with btrfs raid0 support. $ uname -a Linux XXXP 5.15.3-arch1-1 #1 SMP PREEMPT Thu, 18 Nov 2021 22:23:08 +0000 x86_64 GNU/Linux $ pacman -Qi btrfs-progs Name : btrfs-progs Version : 5.15-1 Description : Btrfs filesystem utilities Architecture : x86_64 URL : https://btrfs.wiki.kernel.org Licenses : GPL2 Groups : None Provides : btrfs-progs-unstable Depends On : glibc util-linux-libs lzo zlib zstd libgcrypt Optional Deps : python: libbtrfsutil python bindings [installed] e2fsprogs: btrfs-convert [installed] reiserfsprogs: btrfs-convert [installed] Required By : None Optional For : None Conflicts With : btrfs-progs-unstable Replaces : btrfs-progs-unstable Installed Size : 4.80 MiB Packager : Tobias Powalowski <tpowa@archlinux.org> Build Date : Sun 07 Nov 2021 02:56:20 AM CST Install Date : Tue 09 Nov 2021 01:10:11 PM CST Install Reason : Explicitly installed Install Script : Yes Validated By : Signature ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-11-22 5:26 x8062 @ 2021-11-22 7:24 ` Nikolay Borisov 2021-11-22 10:07 ` x8062 0 siblings, 1 reply; 75+ messages in thread From: Nikolay Borisov @ 2021-11-22 7:24 UTC (permalink / raw) To: x8062, linux-btrfs On 22.11.21 г. 7:26, x8062 wrote: > Hello, > I got periodic warns in my linux console. in dmesg it is the following pasted text. > At https://btrfs.wiki.kernel.org/index.php/Tree-checker I learned it may be a error, so i send the message. Hopefully it could help, Thanks in advance! > > [ 513.900852] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 > You have faulty ram, since 393 has the 17th bit set to 0 whilst has it set to 1. So your ram is clearly corrupting bits. I advise you run a memtest tool and look for possibly changing the faulty ram module. ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re:Re: read time tree block corruption detected 2021-11-22 7:24 ` Nikolay Borisov @ 2021-11-22 10:07 ` x8062 2021-11-22 10:36 ` Nikolay Borisov 0 siblings, 1 reply; 75+ messages in thread From: x8062 @ 2021-11-22 10:07 UTC (permalink / raw) To: Nikolay Borisov; +Cc: linux-btrfs At 2021-11-22 15:24:38, "Nikolay Borisov" <nborisov@suse.com> wrote: > > >On 22.11.21 г. 7:26, x8062 wrote: >> Hello, >> I got periodic warns in my linux console. in dmesg it is the following pasted text. >> At https://btrfs.wiki.kernel.org/index.php/Tree-checker I learned it may be a error, so i send the message. Hopefully it could help, Thanks in advance! >> >> [ 513.900852] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 >> > > >You have faulty ram, since 393 has the 17th bit set to 0 whilst has it >set to 1. So your ram is clearly corrupting bits. I advise you run a >memtest tool and look for possibly changing the faulty ram module. Thank you, can't believe the ram is not so stable. I'll run a memtest later. ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-11-22 10:07 ` x8062 @ 2021-11-22 10:36 ` Nikolay Borisov 2021-11-23 2:42 ` x8062 0 siblings, 1 reply; 75+ messages in thread From: Nikolay Borisov @ 2021-11-22 10:36 UTC (permalink / raw) To: x8062; +Cc: linux-btrfs On 22.11.21 г. 12:07, x8062 wrote: > At 2021-11-22 15:24:38, "Nikolay Borisov" <nborisov@suse.com> wrote: >> >> >> On 22.11.21 г. 7:26, x8062 wrote: >>> Hello, >>> I got periodic warns in my linux console. in dmesg it is the following pasted text. >>> At https://btrfs.wiki.kernel.org/index.php/Tree-checker I learned it may be a error, so i send the message. Hopefully it could help, Thanks in advance! >>> >>> [ 513.900852] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 >>> >> >> >> You have faulty ram, since 393 has the 17th bit set to 0 whilst has it >> set to 1. So your ram is clearly corrupting bits. I advise you run a >> memtest tool and look for possibly changing the faulty ram module. > > Thank you, can't believe the ram is not so stable. I'll run a memtest later. Actually according to the output this is a read-time corruption. THis means the corrupted data has already been written to disk, likely by an older kernel that didn't have the tree cherk code. So running a memcheck is still useful to prevent future corruption. As far as the corrupted files goes - well its data is corrupted. It can technically be fixed, but you'd have to do it yourself. Or alternatively go back on an older kernel i.e pre- 5.11 and try to copy that particular file (inode 2394634). > ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re:Re: read time tree block corruption detected 2021-11-22 10:36 ` Nikolay Borisov @ 2021-11-23 2:42 ` x8062 2021-11-23 5:56 ` Nikolay Borisov 0 siblings, 1 reply; 75+ messages in thread From: x8062 @ 2021-11-23 2:42 UTC (permalink / raw) To: Nikolay Borisov; +Cc: linux-btrfs At 2021-11-22 18:36:41, "Nikolay Borisov" <nborisov@suse.com> wrote: > > >On 22.11.21 г. 12:07, x8062 wrote: >> At 2021-11-22 15:24:38, "Nikolay Borisov" <nborisov@suse.com> wrote: >>> >>> >>> On 22.11.21 г. 7:26, x8062 wrote: >>>> Hello, >>>> I got periodic warns in my linux console. in dmesg it is the following pasted text. >>>> At https://btrfs.wiki.kernel.org/index.php/Tree-checker I learned it may be a error, so i send the message. Hopefully it could help, Thanks in advance! >>>> >>>> [ 513.900852] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 >>>> >>> >>> >>> You have faulty ram, since 393 has the 17th bit set to 0 whilst has it >>> set to 1. So your ram is clearly corrupting bits. I advise you run a >>> memtest tool and look for possibly changing the faulty ram module. >> >> Thank you, can't believe the ram is not so stable. I'll run a memtest later. > >Actually according to the output this is a read-time corruption. THis >means the corrupted data has already been written to disk, likely by an >older kernel that didn't have the tree cherk code. So running a memcheck >is still useful to prevent future corruption. > >As far as the corrupted files goes - well its data is corrupted. It can >technically be fixed, but you'd have to do it yourself. Or alternatively >go back on an older kernel i.e pre- 5.11 and try to copy that particular >file (inode 2394634). > >> I find some problems here. I use the command "find . -inum 2394634" in the btrfs root dir, but nothing printed. does "root=381" means the subvolume ID=381? but now I don't have such subvolume. I deleted some of the subvolumes a few days ago. this is the current subvolume list(some of the dir name shortened) sudo btrfs subvol list . ID 263 gen 111732 top level 5 path 8007/a ID 354 gen 111729 top level 5 path 8007/b ID 622 gen 111757 top level 5 path f015 ID 1174 gen 111758 top level 5 path cc ID 1326 gen 111757 top level 5 path 8007/c ID 1781 gen 111740 top level 5 path ip ID 1782 gen 111758 top level 5 path og ID 1856 gen 111586 top level 1782 path og/OG/DB/server ID 1858 gen 111757 top level 622 path V6/db ID 1875 gen 111742 top level 5 path sdk ID 1918 gen 111742 top level 5 path 8015 ID 1942 gen 111745 top level 5 path ip6 ID 2007 gen 111758 top level 5 path mnew ID 2114 gen 111751 top level 5 path dd ID 2116 gen 111760 top level 2117 path ds/trunk/20200616 ID 2117 gen 111758 top level 5 path ds ID 2118 gen 111758 top level 2114 path dd/trunk/sourcecode ID 2119 gen 111751 top level 2114 path dd/b/1103 ID 2120 gen 111761 top level 5 path tt ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-11-23 2:42 ` x8062 @ 2021-11-23 5:56 ` Nikolay Borisov 0 siblings, 0 replies; 75+ messages in thread From: Nikolay Borisov @ 2021-11-23 5:56 UTC (permalink / raw) To: x8062; +Cc: linux-btrfs On 23.11.21 г. 4:42, x8062 wrote: > At 2021-11-22 18:36:41, "Nikolay Borisov" <nborisov@suse.com> wrote: >> >> >> On 22.11.21 г. 12:07, x8062 wrote: >>> At 2021-11-22 15:24:38, "Nikolay Borisov" <nborisov@suse.com> wrote: >>>> >>>> >>>> On 22.11.21 г. 7:26, x8062 wrote: >>>>> Hello, >>>>> I got periodic warns in my linux console. in dmesg it is the following pasted text. >>>>> At https://btrfs.wiki.kernel.org/index.php/Tree-checker I learned it may be a error, so i send the message. Hopefully it could help, Thanks in advance! >>>>> >>>>> [ 513.900852] BTRFS critical (device sdb3): corrupt leaf: root=381 block=71928348672 slot=74 ino=2394634 file_offset=0, invalid ram_bytes for uncompressed inline extent, have 393 expect 131465 >>>>> >>>> >>>> >>>> You have faulty ram, since 393 has the 17th bit set to 0 whilst has it >>>> set to 1. So your ram is clearly corrupting bits. I advise you run a >>>> memtest tool and look for possibly changing the faulty ram module. >>> >>> Thank you, can't believe the ram is not so stable. I'll run a memtest later. >> >> Actually according to the output this is a read-time corruption. THis >> means the corrupted data has already been written to disk, likely by an >> older kernel that didn't have the tree cherk code. So running a memcheck >> is still useful to prevent future corruption. >> >> As far as the corrupted files goes - well its data is corrupted. It can >> technically be fixed, but you'd have to do it yourself. Or alternatively >> go back on an older kernel i.e pre- 5.11 and try to copy that particular >> file (inode 2394634). >> >>> > I find some problems here. I use the command "find . -inum 2394634" in the btrfs root dir, but nothing printed. > does "root=381" means the subvolume ID=381? but now I don't have such subvolume. I deleted some of the Yes root is the id of the subvolume, if you have deleted it then the corrupted inode should also be gone. > subvolumes a few days ago. this is the current subvolume list(some of the dir name shortened) > sudo btrfs subvol list . > ID 263 gen 111732 top level 5 path 8007/a > ID 354 gen 111729 top level 5 path 8007/b > ID 622 gen 111757 top level 5 path f015 > ID 1174 gen 111758 top level 5 path cc > ID 1326 gen 111757 top level 5 path 8007/c > ID 1781 gen 111740 top level 5 path ip > ID 1782 gen 111758 top level 5 path og > ID 1856 gen 111586 top level 1782 path og/OG/DB/server > ID 1858 gen 111757 top level 622 path V6/db > ID 1875 gen 111742 top level 5 path sdk > ID 1918 gen 111742 top level 5 path 8015 > ID 1942 gen 111745 top level 5 path ip6 > ID 2007 gen 111758 top level 5 path mnew > ID 2114 gen 111751 top level 5 path dd > ID 2116 gen 111760 top level 2117 path ds/trunk/20200616 > ID 2117 gen 111758 top level 5 path ds > ID 2118 gen 111758 top level 2114 path dd/trunk/sourcecode > ID 2119 gen 111751 top level 2114 path dd/b/1103 > ID 2120 gen 111761 top level 5 path tt > ^ permalink raw reply [flat|nested] 75+ messages in thread
* Read time tree block corruption detected @ 2021-07-17 1:45 pepperpoint 2021-07-17 7:05 ` Qu Wenruo [not found] ` <162650555086.7.16811903270475408953.10183708@simplelogin.co> 0 siblings, 2 replies; 75+ messages in thread From: pepperpoint @ 2021-07-17 1:45 UTC (permalink / raw) To: linux-btrfs Hello, I see this message on dmesg: [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: root=363 block=174113599488 slot=9 ino=7415, invalid nlink: has 2 expect no more than 1 for dir [ 2452.256776] BTRFS error (device dm-0): block=174113599488 read time tree block corruption detected When I run btrfs scrub and btrfs check, no error was detected. I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: Read time tree block corruption detected 2021-07-17 1:45 Read " pepperpoint @ 2021-07-17 7:05 ` Qu Wenruo [not found] ` <162650555086.7.16811903270475408953.10183708@simplelogin.co> 1 sibling, 0 replies; 75+ messages in thread From: Qu Wenruo @ 2021-07-17 7:05 UTC (permalink / raw) To: pepperpoint, linux-btrfs On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: > Hello, > > I see this message on dmesg: > > [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: root=363 block=174113599488 slot=9 ino=7415, invalid nlink: has 2 expect no more than 1 for dir > [ 2452.256776] BTRFS error (device dm-0): block=174113599488 read time tree block corruption detected Please provide the following dump: # btrfs ins dump-tree -b 174113599488 /dev/dm-0 Thanks, Qu > > When I run btrfs scrub and btrfs check, no error was detected. > I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 > > How should I fix this error? > ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <162650555086.7.16811903270475408953.10183708@simplelogin.co>]
* Re: Read time tree block corruption detected [not found] ` <162650555086.7.16811903270475408953.10183708@simplelogin.co> @ 2021-07-17 7:51 ` pepperpoint [not found] ` <162650826457.7.1050455337652772013.10184548@mb.ardentcoding.com> 1 sibling, 0 replies; 75+ messages in thread From: pepperpoint @ 2021-07-17 7:51 UTC (permalink / raw) To: Qu Wenruo; +Cc: linux-btrfs Hi Qu, Please see below for the dump. btrfs-progs v5.12.1 leaf 174113599488 items 18 free space 2008 generation 1330906 owner 363 leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 generation 2063 transid 27726 size 40 nbytes 40 block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 sequence 1501 flags 0x0(none) atime 1386484844.468769570 (2013-12-08 14:40:44) ctime 1386484844.468769570 (2013-12-08 14:40:44) mtime 1386484844.468769570 (2013-12-08 14:40:44) otime 0.0 (1970-01-01 08:00:00) item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 index 12 namelen 1 name: 8 item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 generation 27726 type 0 (inline) inline extent data size 40 ram_bytes 40 compression 0 (none) item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 generation 2542 transid 61261 size 40 nbytes 40 block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 sequence 24769 flags 0x0(none) atime 1394335806.351857522 (2014-03-09 11:30:06) ctime 1394335827.344389955 (2014-03-09 11:30:27) mtime 1394335827.344389955 (2014-03-09 11:30:27) otime 0.0 (1970-01-01 08:00:00) item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 index 13 namelen 1 name: 7 item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 generation 61261 type 0 (inline) inline extent data size 40 ram_bytes 40 compression 0 (none) item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 generation 5754 transid 5767 size 307 nbytes 307 block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 sequence 7 flags 0x0(none) atime 1379834835.428558020 (2013-09-22 15:27:15) ctime 1379834835.428558020 (2013-09-22 15:27:15) mtime 1379834835.428558020 (2013-09-22 15:27:15) otime 0.0 (1970-01-01 08:00:00) item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 index 6 namelen 17 name: dhcpcd-eth0.lease item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 generation 5767 type 0 (inline) inline extent data size 307 ram_bytes 307 compression 0 (none) item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 generation 5904 transid 1330906 size 180 nbytes 0 block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 sequence 177 flags 0x0(none) atime 1483277713.141980592 (2017-01-01 21:35:13) ctime 1563162901.234656246 (2019-07-15 11:55:01) mtime 1406534032.158605559 (2014-07-28 15:53:52) otime 0.0 (1970-01-01 08:00:00) item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 index 28 namelen 9 name: backlight item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 index 3 namelen 9 name: backlight item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 location key (120417 INODE_ITEM 0) type FILE transid 117279 data_len 0 name_len 42 name: pci-0000:00:02.0:backlight:intel_backlight item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 location key (7487 INODE_ITEM 0) type FILE transid 5992 data_len 0 name_len 11 name: acpi_video0 item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 location key (55325 INODE_ITEM 0) type FILE transid 63351 data_len 0 name_len 37 name: platform-VPC2004:00:backlight:ideapad item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 location key (7487 INODE_ITEM 0) type FILE transid 5992 data_len 0 name_len 11 name: acpi_video0 item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 location key (55325 INODE_ITEM 0) type FILE transid 63351 data_len 0 name_len 37 name: platform-VPC2004:00:backlight:ideapad item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 location key (120417 INODE_ITEM 0) type FILE transid 117279 data_len 0 name_len 42 name: pci-0000:00:02.0:backlight:intel_backlight ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: > > > Hello, > > > > I see this message on dmesg: > > > > [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: root=363 block=174113599488 slot=9 ino=7415, invalid nlink: has 2 expect no more than 1 for dir > > > > [ 2452.256776] BTRFS error (device dm-0): block=174113599488 read time tree block corruption detected > > Please provide the following dump: > > btrfs ins dump-tree -b 174113599488 /dev/dm-0 > ============================================= > > Thanks, > > Qu > > > When I run btrfs scrub and btrfs check, no error was detected. > > > > I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 > > > > How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <162650826457.7.1050455337652772013.10184548@mb.ardentcoding.com>]
* Re: Read time tree block corruption detected [not found] ` <162650826457.7.1050455337652772013.10184548@mb.ardentcoding.com> @ 2021-07-17 8:14 ` Qu Wenruo [not found] ` <162650966150.7.11743767259405124657.10185986@simplelogin.co> 1 sibling, 0 replies; 75+ messages in thread From: Qu Wenruo @ 2021-07-17 8:14 UTC (permalink / raw) To: pepperpoint; +Cc: linux-btrfs On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: > Hi Qu, > > Please see below for the dump. > > btrfs-progs v5.12.1 > leaf 174113599488 items 18 free space 2008 generation 1330906 owner 363 > leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 > fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 > chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 > item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 > generation 2063 transid 27726 size 40 nbytes 40 > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > sequence 1501 flags 0x0(none) > atime 1386484844.468769570 (2013-12-08 14:40:44) > ctime 1386484844.468769570 (2013-12-08 14:40:44) > mtime 1386484844.468769570 (2013-12-08 14:40:44) > otime 0.0 (1970-01-01 08:00:00) > item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 > index 12 namelen 1 name: 8 > item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 > generation 27726 type 0 (inline) > inline extent data size 40 ram_bytes 40 compression 0 (none) > item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 > generation 2542 transid 61261 size 40 nbytes 40 > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > sequence 24769 flags 0x0(none) > atime 1394335806.351857522 (2014-03-09 11:30:06) > ctime 1394335827.344389955 (2014-03-09 11:30:27) > mtime 1394335827.344389955 (2014-03-09 11:30:27) > otime 0.0 (1970-01-01 08:00:00) > item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 > index 13 namelen 1 name: 7 > item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 > generation 61261 type 0 (inline) > inline extent data size 40 ram_bytes 40 compression 0 (none) > item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 > generation 5754 transid 5767 size 307 nbytes 307 > block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 > sequence 7 flags 0x0(none) > atime 1379834835.428558020 (2013-09-22 15:27:15) > ctime 1379834835.428558020 (2013-09-22 15:27:15) > mtime 1379834835.428558020 (2013-09-22 15:27:15) > otime 0.0 (1970-01-01 08:00:00) > item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 > index 6 namelen 17 name: dhcpcd-eth0.lease > item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 > generation 5767 type 0 (inline) > inline extent data size 307 ram_bytes 307 compression 0 (none) > item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 > generation 5904 transid 1330906 size 180 nbytes 0 > block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 > sequence 177 flags 0x0(none) > atime 1483277713.141980592 (2017-01-01 21:35:13) > ctime 1563162901.234656246 (2019-07-15 11:55:01) > mtime 1406534032.158605559 (2014-07-28 15:53:52) > otime 0.0 (1970-01-01 08:00:00) This inode is indeed a directory. But it has two hard links, which is definitely something unexpected. Under Linux we shouldn't have any hardlink for directory, as it would easily lead to loops. > item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 > index 28 namelen 9 name: backlight Its parent inode is 260 in the same root, with the name backlight. > item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 > index 3 namelen 9 name: backlight Another hardlink in inode 286, which is definitely a regular thing. Btrfs-progs lacks the ability to detect such problem, we need to enhance it first. But do you have any idea how this directory get created? It looks like the content of sysfs. Thanks, Qu > item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 > location key (120417 INODE_ITEM 0) type FILE > transid 117279 data_len 0 name_len 42 > name: pci-0000:00:02.0:backlight:intel_backlight > item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 > location key (7487 INODE_ITEM 0) type FILE > transid 5992 data_len 0 name_len 11 > name: acpi_video0 > item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 > location key (55325 INODE_ITEM 0) type FILE > transid 63351 data_len 0 name_len 37 > name: platform-VPC2004:00:backlight:ideapad > item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 > location key (7487 INODE_ITEM 0) type FILE > transid 5992 data_len 0 name_len 11 > name: acpi_video0 > item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 > location key (55325 INODE_ITEM 0) type FILE > transid 63351 data_len 0 name_len 37 > name: platform-VPC2004:00:backlight:ideapad > item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 > location key (120417 INODE_ITEM 0) type FILE > transid 117279 data_len 0 name_len 42 > name: pci-0000:00:02.0:backlight:intel_backlight > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >> On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: >> >>> Hello, >>> >>> I see this message on dmesg: >>> >>> [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: root=363 block=174113599488 slot=9 ino=7415, invalid nlink: has 2 expect no more than 1 for dir >>> >>> [ 2452.256776] BTRFS error (device dm-0): block=174113599488 read time tree block corruption detected >> >> Please provide the following dump: >> >> btrfs ins dump-tree -b 174113599488 /dev/dm-0 >> ============================================= >> >> Thanks, >> >> Qu >> >>> When I run btrfs scrub and btrfs check, no error was detected. >>> >>> I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 >>> >>> How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <162650966150.7.11743767259405124657.10185986@simplelogin.co>]
* Re: Read time tree block corruption detected [not found] ` <162650966150.7.11743767259405124657.10185986@simplelogin.co> @ 2021-07-17 8:57 ` pepperpoint 2021-07-17 10:12 ` Qu Wenruo [not found] ` <162651674065.6.7912816287233908759.10188327@simplelogin.co> 0 siblings, 2 replies; 75+ messages in thread From: pepperpoint @ 2021-07-17 8:57 UTC (permalink / raw) To: Qu Wenruo; +Cc: linux-btrfs Hi Qu, I don't know how the directory was created but last month, I used btrfs device add and btrfs device remove to move the filesystem from one partition to another. It failed because of the same error and was advised to use btrfs replace instead. I don't know if the error also happened before I move the file system as I don't have any previous logs. Here is the result when I search for the inodes you mentioned if it helps: # find /run/media/root -inum 260 -exec ls -ldi {} \; 260 -rw-r--r-- 1 root root 36864 Jun 25 06:22 /run/media/root/@vcache/live/snapshot/app-info/cache/en_US.cache 260 drwx------ 1 mongodb mongodb 136 Sep 12 2020 /run/media/root/@vlmongodb/live/snapshot/diagnostic.data 260 -rw-rw---- 1 mysql mysql 50331648 Sep 13 2015 /run/media/root/@vlmysql/live/snapshot/ib_logfile0 260 -rw-r----- 1 root lp 8641 Mar 5 2014 /run/media/root/@vspool/live/snapshot/cups/d00001-001 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/live/snapshot/sys 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/4/snapshot/sys # find /run/media/root -inum 286 -exec ls -ldi {} \; 286 -rw-r--r-- 1 root root 96 Aug 16 2015 /run/media/root/@vcache/live/snapshot/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-4 286 -rw-rw---- 1 mysql mysql 4096 Sep 15 2013 /run/media/root/@vlmysql/live/snapshot/mysql/columns_priv.MYI 286 -rw-r-----+ 1 root systemd-journal 16777216 Jul 4 01:14 /run/media/root/@vlog/live/snapshot/journal/5098dd7845ae46d3ba1826c68a809a7c/user-1000@fbd9f65d0ea349f6b996716280e6c4dd-00000000002314c5-0005c5cb84a3a438.journal Directories with pattern /root/@<dir>/live/snapshot/ are subvolumes and directories with pattern /root/@<dir>/<num>/snapshot/ are snapshots of live. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Saturday, July 17th, 2021 at 4:14 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: > > > Hi Qu, > > > > Please see below for the dump. > > > > btrfs-progs v5.12.1 > > > > leaf 174113599488 items 18 free space 2008 generation 1330906 owner 363 > > > > leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 > > > > fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 > > > > chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 > > > > item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 > > > > generation 2063 transid 27726 size 40 nbytes 40 > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > sequence 1501 flags 0x0(none) > > > > atime 1386484844.468769570 (2013-12-08 14:40:44) > > > > ctime 1386484844.468769570 (2013-12-08 14:40:44) > > > > mtime 1386484844.468769570 (2013-12-08 14:40:44) > > > > otime 0.0 (1970-01-01 08:00:00) > > > > item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 > > > > index 12 namelen 1 name: 8 > > > > item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 > > > > generation 27726 type 0 (inline) > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 > > > > generation 2542 transid 61261 size 40 nbytes 40 > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > sequence 24769 flags 0x0(none) > > > > atime 1394335806.351857522 (2014-03-09 11:30:06) > > > > ctime 1394335827.344389955 (2014-03-09 11:30:27) > > > > mtime 1394335827.344389955 (2014-03-09 11:30:27) > > > > otime 0.0 (1970-01-01 08:00:00) > > > > item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 > > > > index 13 namelen 1 name: 7 > > > > item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 > > > > generation 61261 type 0 (inline) > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 > > > > generation 5754 transid 5767 size 307 nbytes 307 > > > > block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 > > > > sequence 7 flags 0x0(none) > > > > atime 1379834835.428558020 (2013-09-22 15:27:15) > > > > ctime 1379834835.428558020 (2013-09-22 15:27:15) > > > > mtime 1379834835.428558020 (2013-09-22 15:27:15) > > > > otime 0.0 (1970-01-01 08:00:00) > > > > item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 > > > > index 6 namelen 17 name: dhcpcd-eth0.lease > > > > item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 > > > > generation 5767 type 0 (inline) > > > > inline extent data size 307 ram_bytes 307 compression 0 (none) > > > > item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 > > > > generation 5904 transid 1330906 size 180 nbytes 0 > > > > block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 > > > > sequence 177 flags 0x0(none) > > > > atime 1483277713.141980592 (2017-01-01 21:35:13) > > > > ctime 1563162901.234656246 (2019-07-15 11:55:01) > > > > mtime 1406534032.158605559 (2014-07-28 15:53:52) > > > > otime 0.0 (1970-01-01 08:00:00) > > This inode is indeed a directory. > > But it has two hard links, which is definitely something unexpected. > > Under Linux we shouldn't have any hardlink for directory, as it would > > easily lead to loops. > > > item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 > > > > index 28 namelen 9 name: backlight > > Its parent inode is 260 in the same root, with the name backlight. > > > item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 > > > > index 3 namelen 9 name: backlight > > Another hardlink in inode 286, which is definitely a regular thing. > > Btrfs-progs lacks the ability to detect such problem, we need to enhance > > it first. > > But do you have any idea how this directory get created? > > It looks like the content of sysfs. > > Thanks, > > Qu > > > item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 > > > > location key (120417 INODE_ITEM 0) type FILE > > > > transid 117279 data_len 0 name_len 42 > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 > > > > location key (7487 INODE_ITEM 0) type FILE > > > > transid 5992 data_len 0 name_len 11 > > > > name: acpi_video0 > > > > item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 > > > > location key (55325 INODE_ITEM 0) type FILE > > > > transid 63351 data_len 0 name_len 37 > > > > name: platform-VPC2004:00:backlight:ideapad > > > > item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 > > > > location key (7487 INODE_ITEM 0) type FILE > > > > transid 5992 data_len 0 name_len 11 > > > > name: acpi_video0 > > > > item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 > > > > location key (55325 INODE_ITEM 0) type FILE > > > > transid 63351 data_len 0 name_len 37 > > > > name: platform-VPC2004:00:backlight:ideapad > > > > item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 > > > > location key (120417 INODE_ITEM 0) type FILE > > > > transid 117279 data_len 0 name_len 42 > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: > > > > > On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > Hello, > > > > > > > > I see this message on dmesg: > > > > > > > > [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: root=363 block=174113599488 slot=9 ino=7415, invalid nlink: has 2 expect no more than 1 for dir > > > > > > > > [ 2452.256776] BTRFS error (device dm-0): block=174113599488 read time tree block corruption detected > > > > > > Please provide the following dump: > > > > > > btrfs ins dump-tree -b 174113599488 /dev/dm-0 > > > ============================================= > > > > > > Thanks, > > > > > > Qu > > > > > > > When I run btrfs scrub and btrfs check, no error was detected. > > > > > > > > I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 > > > > > > > > How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: Read time tree block corruption detected 2021-07-17 8:57 ` pepperpoint @ 2021-07-17 10:12 ` Qu Wenruo [not found] ` <162651674065.6.7912816287233908759.10188327@simplelogin.co> 1 sibling, 0 replies; 75+ messages in thread From: Qu Wenruo @ 2021-07-17 10:12 UTC (permalink / raw) To: pepperpoint, Qu Wenruo; +Cc: linux-btrfs On 2021/7/17 下午4:57, pepperpoint@mb.ardentcoding.com wrote: > Hi Qu, > > I don't know how the directory was created but last month, I used btrfs device add and btrfs device remove to move the filesystem from one partition to another. It failed because of the same error and was advised to use btrfs replace instead. I don't know if the error also happened before I move the file system as I don't have any previous logs. It definitely happens before you moving the fs. As regular dev replacing/add/move only relocates the metadata, but not touching the fs trees. > > Here is the result when I search for the inodes you mentioned if it helps: > # find /run/media/root -inum 260 -exec ls -ldi {} \; > 260 -rw-r--r-- 1 root root 36864 Jun 25 06:22 /run/media/root/@vcache/live/snapshot/app-info/cache/en_US.cache > 260 drwx------ 1 mongodb mongodb 136 Sep 12 2020 /run/media/root/@vlmongodb/live/snapshot/diagnostic.data > 260 -rw-rw---- 1 mysql mysql 50331648 Sep 13 2015 /run/media/root/@vlmysql/live/snapshot/ib_logfile0 > 260 -rw-r----- 1 root lp 8641 Mar 5 2014 /run/media/root/@vspool/live/snapshot/cups/d00001-001 > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/live/snapshot/sys > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/4/snapshot/sys Since btrfs can have the same inode number inside different subvolumes, you may want to limit the search inside subvolume 363. "-mount" option of find can do that, you only need to locate subvolume 363 by "btrfs subv list". But from these output I guess above two "sys" directory are more possible. Is there any directory named "blaklight" inside those directory? > > # find /run/media/root -inum 286 -exec ls -ldi {} \; > 286 -rw-r--r-- 1 root root 96 Aug 16 2015 /run/media/root/@vcache/live/snapshot/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-4 > 286 -rw-rw---- 1 mysql mysql 4096 Sep 15 2013 /run/media/root/@vlmysql/live/snapshot/mysql/columns_priv.MYI > 286 -rw-r-----+ 1 root systemd-journal 16777216 Jul 4 01:14 /run/media/root/@vlog/live/snapshot/journal/5098dd7845ae46d3ba1826c68a809a7c/user-1000@fbd9f65d0ea349f6b996716280e6c4dd-00000000002314c5-0005c5cb84a3a438.journal This is interesting, it means the inode 286 is not accessible. It can be some orphan inode, but would you dump subvolume 363 then try to locate the inode 286? One example command would be: # btrfs ins dump-tree -t 363 <dev> | grep -A 5 "(286 " Thanks, Qu > > Directories with pattern /root/@<dir>/live/snapshot/ are subvolumes and directories with pattern /root/@<dir>/<num>/snapshot/ are snapshots of live. > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > On Saturday, July 17th, 2021 at 4:14 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >> On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: >> >>> Hi Qu, >>> >>> Please see below for the dump. >>> >>> btrfs-progs v5.12.1 >>> >>> leaf 174113599488 items 18 free space 2008 generation 1330906 owner 363 >>> >>> leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 >>> >>> fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 >>> >>> chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 >>> >>> item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 >>> >>> generation 2063 transid 27726 size 40 nbytes 40 >>> >>> block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 >>> >>> sequence 1501 flags 0x0(none) >>> >>> atime 1386484844.468769570 (2013-12-08 14:40:44) >>> >>> ctime 1386484844.468769570 (2013-12-08 14:40:44) >>> >>> mtime 1386484844.468769570 (2013-12-08 14:40:44) >>> >>> otime 0.0 (1970-01-01 08:00:00) >>> >>> item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 >>> >>> index 12 namelen 1 name: 8 >>> >>> item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 >>> >>> generation 27726 type 0 (inline) >>> >>> inline extent data size 40 ram_bytes 40 compression 0 (none) >>> >>> item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 >>> >>> generation 2542 transid 61261 size 40 nbytes 40 >>> >>> block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 >>> >>> sequence 24769 flags 0x0(none) >>> >>> atime 1394335806.351857522 (2014-03-09 11:30:06) >>> >>> ctime 1394335827.344389955 (2014-03-09 11:30:27) >>> >>> mtime 1394335827.344389955 (2014-03-09 11:30:27) >>> >>> otime 0.0 (1970-01-01 08:00:00) >>> >>> item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 >>> >>> index 13 namelen 1 name: 7 >>> >>> item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 >>> >>> generation 61261 type 0 (inline) >>> >>> inline extent data size 40 ram_bytes 40 compression 0 (none) >>> >>> item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 >>> >>> generation 5754 transid 5767 size 307 nbytes 307 >>> >>> block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 >>> >>> sequence 7 flags 0x0(none) >>> >>> atime 1379834835.428558020 (2013-09-22 15:27:15) >>> >>> ctime 1379834835.428558020 (2013-09-22 15:27:15) >>> >>> mtime 1379834835.428558020 (2013-09-22 15:27:15) >>> >>> otime 0.0 (1970-01-01 08:00:00) >>> >>> item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 >>> >>> index 6 namelen 17 name: dhcpcd-eth0.lease >>> >>> item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 >>> >>> generation 5767 type 0 (inline) >>> >>> inline extent data size 307 ram_bytes 307 compression 0 (none) >>> >>> item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 >>> >>> generation 5904 transid 1330906 size 180 nbytes 0 >>> >>> block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 >>> >>> sequence 177 flags 0x0(none) >>> >>> atime 1483277713.141980592 (2017-01-01 21:35:13) >>> >>> ctime 1563162901.234656246 (2019-07-15 11:55:01) >>> >>> mtime 1406534032.158605559 (2014-07-28 15:53:52) >>> >>> otime 0.0 (1970-01-01 08:00:00) >> >> This inode is indeed a directory. >> >> But it has two hard links, which is definitely something unexpected. >> >> Under Linux we shouldn't have any hardlink for directory, as it would >> >> easily lead to loops. >> >>> item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 >>> >>> index 28 namelen 9 name: backlight >> >> Its parent inode is 260 in the same root, with the name backlight. >> >>> item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 >>> >>> index 3 namelen 9 name: backlight >> >> Another hardlink in inode 286, which is definitely a regular thing. >> >> Btrfs-progs lacks the ability to detect such problem, we need to enhance >> >> it first. >> >> But do you have any idea how this directory get created? >> >> It looks like the content of sysfs. >> >> Thanks, >> >> Qu >> >>> item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 >>> >>> location key (120417 INODE_ITEM 0) type FILE >>> >>> transid 117279 data_len 0 name_len 42 >>> >>> name: pci-0000:00:02.0:backlight:intel_backlight >>> >>> item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 >>> >>> location key (7487 INODE_ITEM 0) type FILE >>> >>> transid 5992 data_len 0 name_len 11 >>> >>> name: acpi_video0 >>> >>> item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 >>> >>> location key (55325 INODE_ITEM 0) type FILE >>> >>> transid 63351 data_len 0 name_len 37 >>> >>> name: platform-VPC2004:00:backlight:ideapad >>> >>> item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 >>> >>> location key (7487 INODE_ITEM 0) type FILE >>> >>> transid 5992 data_len 0 name_len 11 >>> >>> name: acpi_video0 >>> >>> item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 >>> >>> location key (55325 INODE_ITEM 0) type FILE >>> >>> transid 63351 data_len 0 name_len 37 >>> >>> name: platform-VPC2004:00:backlight:ideapad >>> >>> item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 >>> >>> location key (120417 INODE_ITEM 0) type FILE >>> >>> transid 117279 data_len 0 name_len 42 >>> >>> name: pci-0000:00:02.0:backlight:intel_backlight >>> >>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>> >>> On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: >>> >>>> On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: >>>> >>>>> Hello, >>>>> >>>>> I see this message on dmesg: >>>>> >>>>> [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: root=363 block=174113599488 slot=9 ino=7415, invalid nlink: has 2 expect no more than 1 for dir >>>>> >>>>> [ 2452.256776] BTRFS error (device dm-0): block=174113599488 read time tree block corruption detected >>>> >>>> Please provide the following dump: >>>> >>>> btrfs ins dump-tree -b 174113599488 /dev/dm-0 >>>> ============================================= >>>> >>>> Thanks, >>>> >>>> Qu >>>> >>>>> When I run btrfs scrub and btrfs check, no error was detected. >>>>> >>>>> I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 >>>>> >>>>> How should I fix this error? > ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <162651674065.6.7912816287233908759.10188327@simplelogin.co>]
* Re: Read time tree block corruption detected [not found] ` <162651674065.6.7912816287233908759.10188327@simplelogin.co> @ 2021-07-17 10:34 ` pepperpoint [not found] ` <162651809235.7.7061042874033963922.10188873@mb.ardentcoding.com> 1 sibling, 0 replies; 75+ messages in thread From: pepperpoint @ 2021-07-17 10:34 UTC (permalink / raw) To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs Hi Qu, Unfortunately I cannot find subvolume 363 # btrfs subvolume list /run/media/root ID 361 gen 1814826 top level 584 path @/live/snapshot ID 364 gen 1814414 top level 5 path @vtmp/live/snapshot ID 369 gen 1814414 top level 5 path @vlmachines/live/snapshot ID 493 gen 1814414 top level 5 path @vlportables/live/snapshot ID 579 gen 1814828 top level 5 path @vlog/live/snapshot ID 580 gen 1814414 top level 5 path @vcache/live/snapshot ID 581 gen 1814414 top level 5 path @vlmongodb/live/snapshot ID 582 gen 1814414 top level 5 path @vlmysql/live/snapshot ID 583 gen 1814414 top level 5 path @vspool/live/snapshot ID 584 gen 1814414 top level 5 path @ ID 598 gen 1813420 top level 584 path @/4/snapshot # btrfs ins dump-tree -t 363 /dev/dm-0 | grep -A 5 "(286 " parent transid verify failed on 174170742784 wanted 1789655 found 1812621 parent transid verify failed on 174170742784 wanted 1789655 found 1812621 parent transid verify failed on 174170742784 wanted 1789655 found 1812621 Ignoring transid failure ERROR: child eb corrupted: parent bytenr=174170738688 item=0 parent level=2 child bytenr=174170742784 child level=0 ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Saturday, July 17th, 2021 at 6:12 PM, Qu Wenruo <wqu@suse.com> wrote: > On 2021/7/17 下午4:57, pepperpoint@mb.ardentcoding.com wrote: > > > Hi Qu, > > > > I don't know how the directory was created but last month, I used btrfs device add and btrfs device remove to move the filesystem from one partition to another. It failed because of the same error and was advised to use btrfs replace instead. I don't know if the error also happened before I move the file system as I don't have any previous logs. > > It definitely happens before you moving the fs. > > As regular dev replacing/add/move only relocates the metadata, but not > > touching the fs trees. > > > Here is the result when I search for the inodes you mentioned if it helps: > > > > find /run/media/root -inum 260 -exec ls -ldi {} \; > > ================================================== > > > > 260 -rw-r--r-- 1 root root 36864 Jun 25 06:22 /run/media/root/@vcache/live/snapshot/app-info/cache/en_US.cache > > > > 260 drwx------ 1 mongodb mongodb 136 Sep 12 2020 /run/media/root/@vlmongodb/live/snapshot/diagnostic.data > > > > 260 -rw-rw---- 1 mysql mysql 50331648 Sep 13 2015 /run/media/root/@vlmysql/live/snapshot/ib_logfile0 > > > > 260 -rw-r----- 1 root lp 8641 Mar 5 2014 /run/media/root/@vspool/live/snapshot/cups/d00001-001 > > > > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/live/snapshot/sys > > > > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/4/snapshot/sys > > Since btrfs can have the same inode number inside different subvolumes, > > you may want to limit the search inside subvolume 363. > > "-mount" option of find can do that, you only need to locate subvolume > > 363 by "btrfs subv list". > > But from these output I guess above two "sys" directory are more possible. > > Is there any directory named "blaklight" inside those directory? > > > find /run/media/root -inum 286 -exec ls -ldi {} \; > > ================================================== > > > > 286 -rw-r--r-- 1 root root 96 Aug 16 2015 /run/media/root/@vcache/live/snapshot/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-4 > > > > 286 -rw-rw---- 1 mysql mysql 4096 Sep 15 2013 /run/media/root/@vlmysql/live/snapshot/mysql/columns_priv.MYI > > > > 286 -rw-r-----+ 1 root systemd-journal 16777216 Jul 4 01:14 /run/media/root/@vlog/live/snapshot/journal/5098dd7845ae46d3ba1826c68a809a7c/user-1000@fbd9f65d0ea349f6b996716280e6c4dd-00000000002314c5-0005c5cb84a3a438.journal > > This is interesting, it means the inode 286 is not accessible. > > It can be some orphan inode, but would you dump subvolume 363 then try > > to locate the inode 286? > > One example command would be: > > btrfs ins dump-tree -t 363 <dev> | grep -A 5 "(286 " > ==================================================== > > Thanks, > > Qu > > > Directories with pattern /root/@<dir>/live/snapshot/ are subvolumes and directories with pattern /root/@<dir>/<num>/snapshot/ are snapshots of live. > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > On Saturday, July 17th, 2021 at 4:14 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: > > > > > On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > Hi Qu, > > > > > > > > Please see below for the dump. > > > > > > > > btrfs-progs v5.12.1 > > > > > > > > leaf 174113599488 items 18 free space 2008 generation 1330906 owner 363 > > > > > > > > leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 > > > > > > > > fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 > > > > > > > > chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 > > > > > > > > item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 > > > > > > > > generation 2063 transid 27726 size 40 nbytes 40 > > > > > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > > > > > sequence 1501 flags 0x0(none) > > > > > > > > atime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > ctime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > mtime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 > > > > > > > > index 12 namelen 1 name: 8 > > > > > > > > item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 > > > > > > > > generation 27726 type 0 (inline) > > > > > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > > > > > item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 > > > > > > > > generation 2542 transid 61261 size 40 nbytes 40 > > > > > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > > > > > sequence 24769 flags 0x0(none) > > > > > > > > atime 1394335806.351857522 (2014-03-09 11:30:06) > > > > > > > > ctime 1394335827.344389955 (2014-03-09 11:30:27) > > > > > > > > mtime 1394335827.344389955 (2014-03-09 11:30:27) > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 > > > > > > > > index 13 namelen 1 name: 7 > > > > > > > > item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 > > > > > > > > generation 61261 type 0 (inline) > > > > > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > > > > > item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 > > > > > > > > generation 5754 transid 5767 size 307 nbytes 307 > > > > > > > > block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 > > > > > > > > sequence 7 flags 0x0(none) > > > > > > > > atime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > ctime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > mtime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 > > > > > > > > index 6 namelen 17 name: dhcpcd-eth0.lease > > > > > > > > item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 > > > > > > > > generation 5767 type 0 (inline) > > > > > > > > inline extent data size 307 ram_bytes 307 compression 0 (none) > > > > > > > > item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 > > > > > > > > generation 5904 transid 1330906 size 180 nbytes 0 > > > > > > > > block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 > > > > > > > > sequence 177 flags 0x0(none) > > > > > > > > atime 1483277713.141980592 (2017-01-01 21:35:13) > > > > > > > > ctime 1563162901.234656246 (2019-07-15 11:55:01) > > > > > > > > mtime 1406534032.158605559 (2014-07-28 15:53:52) > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > This inode is indeed a directory. > > > > > > But it has two hard links, which is definitely something unexpected. > > > > > > Under Linux we shouldn't have any hardlink for directory, as it would > > > > > > easily lead to loops. > > > > > > > item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 > > > > > > > > index 28 namelen 9 name: backlight > > > > > > Its parent inode is 260 in the same root, with the name backlight. > > > > > > > item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 > > > > > > > > index 3 namelen 9 name: backlight > > > > > > Another hardlink in inode 286, which is definitely a regular thing. > > > > > > Btrfs-progs lacks the ability to detect such problem, we need to enhance > > > > > > it first. > > > > > > But do you have any idea how this directory get created? > > > > > > It looks like the content of sysfs. > > > > > > Thanks, > > > > > > Qu > > > > > > > item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 > > > > > > > > location key (120417 INODE_ITEM 0) type FILE > > > > > > > > transid 117279 data_len 0 name_len 42 > > > > > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > > > > > item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 > > > > > > > > location key (7487 INODE_ITEM 0) type FILE > > > > > > > > transid 5992 data_len 0 name_len 11 > > > > > > > > name: acpi_video0 > > > > > > > > item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 > > > > > > > > location key (55325 INODE_ITEM 0) type FILE > > > > > > > > transid 63351 data_len 0 name_len 37 > > > > > > > > name: platform-VPC2004:00:backlight:ideapad > > > > > > > > item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 > > > > > > > > location key (7487 INODE_ITEM 0) type FILE > > > > > > > > transid 5992 data_len 0 name_len 11 > > > > > > > > name: acpi_video0 > > > > > > > > item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 > > > > > > > > location key (55325 INODE_ITEM 0) type FILE > > > > > > > > transid 63351 data_len 0 name_len 37 > > > > > > > > name: platform-VPC2004:00:backlight:ideapad > > > > > > > > item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 > > > > > > > > location key (120417 INODE_ITEM 0) type FILE > > > > > > > > transid 117279 data_len 0 name_len 42 > > > > > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: > > > > > > > > > On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > Hello, > > > > > > > > > > > > I see this message on dmesg: > > > > > > > > > > > > [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: root=363 block=174113599488 slot=9 ino=7415, invalid nlink: has 2 expect no more than 1 for dir > > > > > > > > > > > > [ 2452.256776] BTRFS error (device dm-0): block=174113599488 read time tree block corruption detected > > > > > > > > > > Please provide the following dump: > > > > > > > > > > btrfs ins dump-tree -b 174113599488 /dev/dm-0 > > > > > ============================================= > > > > > > > > > > Thanks, > > > > > > > > > > Qu > > > > > > > > > > > When I run btrfs scrub and btrfs check, no error was detected. > > > > > > > > > > > > I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 > > > > > > > > > > > > How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <162651809235.7.7061042874033963922.10188873@mb.ardentcoding.com>]
* Re: Read time tree block corruption detected [not found] ` <162651809235.7.7061042874033963922.10188873@mb.ardentcoding.com> @ 2021-07-17 10:48 ` Qu Wenruo [not found] ` <162651892663.6.17938009695497100557.10189169@simplelogin.co> 1 sibling, 0 replies; 75+ messages in thread From: Qu Wenruo @ 2021-07-17 10:48 UTC (permalink / raw) To: pepperpoint, Qu Wenruo; +Cc: linux-btrfs On 2021/7/17 下午6:34, pepperpoint@mb.ardentcoding.com wrote: > Hi Qu, > > Unfortunately I cannot find subvolume 363 > # btrfs subvolume list /run/media/root > ID 361 gen 1814826 top level 584 path @/live/snapshot > ID 364 gen 1814414 top level 5 path @vtmp/live/snapshot > ID 369 gen 1814414 top level 5 path @vlmachines/live/snapshot > ID 493 gen 1814414 top level 5 path @vlportables/live/snapshot > ID 579 gen 1814828 top level 5 path @vlog/live/snapshot > ID 580 gen 1814414 top level 5 path @vcache/live/snapshot > ID 581 gen 1814414 top level 5 path @vlmongodb/live/snapshot > ID 582 gen 1814414 top level 5 path @vlmysql/live/snapshot > ID 583 gen 1814414 top level 5 path @vspool/live/snapshot > ID 584 gen 1814414 top level 5 path @ > ID 598 gen 1813420 top level 584 path @/4/snapshot Maybe 363 is some subvolume get deleted and later snapshot of it still exists. This will be harder to debug. Can you take a btrfs-image dump of your filesystem? (needs to be taken with the fs unmounted). The dumped image will contain your metadata, including file names and directory structures, but no data inside those files. Although btrfs-image has "-s" option to mask the filenames, but considering the filename in this case is useful to locate the inode, I guess it's better to take the image without any "-s" option. > > # btrfs ins dump-tree -t 363 /dev/dm-0 | grep -A 5 "(286 " > parent transid verify failed on 174170742784 wanted 1789655 found 1812621 > parent transid verify failed on 174170742784 wanted 1789655 found 1812621 > parent transid verify failed on 174170742784 wanted 1789655 found 1812621 > Ignoring transid failure > ERROR: child eb corrupted: parent bytenr=174170738688 item=0 parent level=2 child bytenr=174170742784 child level=0 This transid mismatch may be a problem when running dump-tree on mounted fs, since you mentioned btrfs check reported no error, there shouldn't be a transid mismatch error. Anyway, if you can upload the btrfs-image dump, it would be much easier for us to debug and find out what's really going. Thanks, Qu > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > On Saturday, July 17th, 2021 at 6:12 PM, Qu Wenruo <wqu@suse.com> wrote: > >> On 2021/7/17 下午4:57, pepperpoint@mb.ardentcoding.com wrote: >> >>> Hi Qu, >>> >>> I don't know how the directory was created but last month, I used btrfs device add and btrfs device remove to move the filesystem from one partition to another. It failed because of the same error and was advised to use btrfs replace instead. I don't know if the error also happened before I move the file system as I don't have any previous logs. >> >> It definitely happens before you moving the fs. >> >> As regular dev replacing/add/move only relocates the metadata, but not >> >> touching the fs trees. >> >>> Here is the result when I search for the inodes you mentioned if it helps: >>> >>> find /run/media/root -inum 260 -exec ls -ldi {} \; >>> ================================================== >>> >>> 260 -rw-r--r-- 1 root root 36864 Jun 25 06:22 /run/media/root/@vcache/live/snapshot/app-info/cache/en_US.cache >>> >>> 260 drwx------ 1 mongodb mongodb 136 Sep 12 2020 /run/media/root/@vlmongodb/live/snapshot/diagnostic.data >>> >>> 260 -rw-rw---- 1 mysql mysql 50331648 Sep 13 2015 /run/media/root/@vlmysql/live/snapshot/ib_logfile0 >>> >>> 260 -rw-r----- 1 root lp 8641 Mar 5 2014 /run/media/root/@vspool/live/snapshot/cups/d00001-001 >>> >>> 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/live/snapshot/sys >>> >>> 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/4/snapshot/sys >> >> Since btrfs can have the same inode number inside different subvolumes, >> >> you may want to limit the search inside subvolume 363. >> >> "-mount" option of find can do that, you only need to locate subvolume >> >> 363 by "btrfs subv list". >> >> But from these output I guess above two "sys" directory are more possible. >> >> Is there any directory named "blaklight" inside those directory? >> >>> find /run/media/root -inum 286 -exec ls -ldi {} \; >>> ================================================== >>> >>> 286 -rw-r--r-- 1 root root 96 Aug 16 2015 /run/media/root/@vcache/live/snapshot/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-4 >>> >>> 286 -rw-rw---- 1 mysql mysql 4096 Sep 15 2013 /run/media/root/@vlmysql/live/snapshot/mysql/columns_priv.MYI >>> >>> 286 -rw-r-----+ 1 root systemd-journal 16777216 Jul 4 01:14 /run/media/root/@vlog/live/snapshot/journal/5098dd7845ae46d3ba1826c68a809a7c/user-1000@fbd9f65d0ea349f6b996716280e6c4dd-00000000002314c5-0005c5cb84a3a438.journal >> >> This is interesting, it means the inode 286 is not accessible. >> >> It can be some orphan inode, but would you dump subvolume 363 then try >> >> to locate the inode 286? >> >> One example command would be: >> >> btrfs ins dump-tree -t 363 <dev> | grep -A 5 "(286 " >> ==================================================== >> >> Thanks, >> >> Qu >> >>> Directories with pattern /root/@<dir>/live/snapshot/ are subvolumes and directories with pattern /root/@<dir>/<num>/snapshot/ are snapshots of live. >>> >>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>> >>> On Saturday, July 17th, 2021 at 4:14 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: >>> >>>> On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: >>>> >>>>> Hi Qu, >>>>> >>>>> Please see below for the dump. >>>>> >>>>> btrfs-progs v5.12.1 >>>>> >>>>> leaf 174113599488 items 18 free space 2008 generation 1330906 owner 363 >>>>> >>>>> leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 >>>>> >>>>> fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 >>>>> >>>>> chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 >>>>> >>>>> item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 >>>>> >>>>> generation 2063 transid 27726 size 40 nbytes 40 >>>>> >>>>> block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 >>>>> >>>>> sequence 1501 flags 0x0(none) >>>>> >>>>> atime 1386484844.468769570 (2013-12-08 14:40:44) >>>>> >>>>> ctime 1386484844.468769570 (2013-12-08 14:40:44) >>>>> >>>>> mtime 1386484844.468769570 (2013-12-08 14:40:44) >>>>> >>>>> otime 0.0 (1970-01-01 08:00:00) >>>>> >>>>> item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 >>>>> >>>>> index 12 namelen 1 name: 8 >>>>> >>>>> item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 >>>>> >>>>> generation 27726 type 0 (inline) >>>>> >>>>> inline extent data size 40 ram_bytes 40 compression 0 (none) >>>>> >>>>> item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 >>>>> >>>>> generation 2542 transid 61261 size 40 nbytes 40 >>>>> >>>>> block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 >>>>> >>>>> sequence 24769 flags 0x0(none) >>>>> >>>>> atime 1394335806.351857522 (2014-03-09 11:30:06) >>>>> >>>>> ctime 1394335827.344389955 (2014-03-09 11:30:27) >>>>> >>>>> mtime 1394335827.344389955 (2014-03-09 11:30:27) >>>>> >>>>> otime 0.0 (1970-01-01 08:00:00) >>>>> >>>>> item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 >>>>> >>>>> index 13 namelen 1 name: 7 >>>>> >>>>> item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 >>>>> >>>>> generation 61261 type 0 (inline) >>>>> >>>>> inline extent data size 40 ram_bytes 40 compression 0 (none) >>>>> >>>>> item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 >>>>> >>>>> generation 5754 transid 5767 size 307 nbytes 307 >>>>> >>>>> block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 >>>>> >>>>> sequence 7 flags 0x0(none) >>>>> >>>>> atime 1379834835.428558020 (2013-09-22 15:27:15) >>>>> >>>>> ctime 1379834835.428558020 (2013-09-22 15:27:15) >>>>> >>>>> mtime 1379834835.428558020 (2013-09-22 15:27:15) >>>>> >>>>> otime 0.0 (1970-01-01 08:00:00) >>>>> >>>>> item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 >>>>> >>>>> index 6 namelen 17 name: dhcpcd-eth0.lease >>>>> >>>>> item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 >>>>> >>>>> generation 5767 type 0 (inline) >>>>> >>>>> inline extent data size 307 ram_bytes 307 compression 0 (none) >>>>> >>>>> item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 >>>>> >>>>> generation 5904 transid 1330906 size 180 nbytes 0 >>>>> >>>>> block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 >>>>> >>>>> sequence 177 flags 0x0(none) >>>>> >>>>> atime 1483277713.141980592 (2017-01-01 21:35:13) >>>>> >>>>> ctime 1563162901.234656246 (2019-07-15 11:55:01) >>>>> >>>>> mtime 1406534032.158605559 (2014-07-28 15:53:52) >>>>> >>>>> otime 0.0 (1970-01-01 08:00:00) >>>> >>>> This inode is indeed a directory. >>>> >>>> But it has two hard links, which is definitely something unexpected. >>>> >>>> Under Linux we shouldn't have any hardlink for directory, as it would >>>> >>>> easily lead to loops. >>>> >>>>> item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 >>>>> >>>>> index 28 namelen 9 name: backlight >>>> >>>> Its parent inode is 260 in the same root, with the name backlight. >>>> >>>>> item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 >>>>> >>>>> index 3 namelen 9 name: backlight >>>> >>>> Another hardlink in inode 286, which is definitely a regular thing. >>>> >>>> Btrfs-progs lacks the ability to detect such problem, we need to enhance >>>> >>>> it first. >>>> >>>> But do you have any idea how this directory get created? >>>> >>>> It looks like the content of sysfs. >>>> >>>> Thanks, >>>> >>>> Qu >>>> >>>>> item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 >>>>> >>>>> location key (120417 INODE_ITEM 0) type FILE >>>>> >>>>> transid 117279 data_len 0 name_len 42 >>>>> >>>>> name: pci-0000:00:02.0:backlight:intel_backlight >>>>> >>>>> item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 >>>>> >>>>> location key (7487 INODE_ITEM 0) type FILE >>>>> >>>>> transid 5992 data_len 0 name_len 11 >>>>> >>>>> name: acpi_video0 >>>>> >>>>> item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 >>>>> >>>>> location key (55325 INODE_ITEM 0) type FILE >>>>> >>>>> transid 63351 data_len 0 name_len 37 >>>>> >>>>> name: platform-VPC2004:00:backlight:ideapad >>>>> >>>>> item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 >>>>> >>>>> location key (7487 INODE_ITEM 0) type FILE >>>>> >>>>> transid 5992 data_len 0 name_len 11 >>>>> >>>>> name: acpi_video0 >>>>> >>>>> item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 >>>>> >>>>> location key (55325 INODE_ITEM 0) type FILE >>>>> >>>>> transid 63351 data_len 0 name_len 37 >>>>> >>>>> name: platform-VPC2004:00:backlight:ideapad >>>>> >>>>> item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 >>>>> >>>>> location key (120417 INODE_ITEM 0) type FILE >>>>> >>>>> transid 117279 data_len 0 name_len 42 >>>>> >>>>> name: pci-0000:00:02.0:backlight:intel_backlight >>>>> >>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>> >>>>> On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: >>>>> >>>>>> On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> I see this message on dmesg: >>>>>>> >>>>>>> [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: root=363 block=174113599488 slot=9 ino=7415, invalid nlink: has 2 expect no more than 1 for dir >>>>>>> >>>>>>> [ 2452.256776] BTRFS error (device dm-0): block=174113599488 read time tree block corruption detected >>>>>> >>>>>> Please provide the following dump: >>>>>> >>>>>> btrfs ins dump-tree -b 174113599488 /dev/dm-0 >>>>>> ============================================= >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Qu >>>>>> >>>>>>> When I run btrfs scrub and btrfs check, no error was detected. >>>>>>> >>>>>>> I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 >>>>>>> >>>>>>> How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <162651892663.6.17938009695497100557.10189169@simplelogin.co>]
* Re: Read time tree block corruption detected [not found] ` <162651892663.6.17938009695497100557.10189169@simplelogin.co> @ 2021-07-17 12:51 ` pepperpoint [not found] ` <CQeY09U34m7SrT6nTAlkSQrbLJtmyKF1tDfuGDtKUkwJqujMI_nZU4MpGiU4F_Q1U3Lk1aWD1mFCT5SlsOsOcILWECflIw7EhVQTQpy_1Ts=@email.ardentcoding.com> 1 sibling, 0 replies; 75+ messages in thread From: pepperpoint @ 2021-07-17 12:51 UTC (permalink / raw) To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs Hi Qu, I run btrfs ins dump-tree -t 363 unmounted but the same error appears. Rerunning btrfs check does not show any error. Regards, Lester ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Saturday, July 17th, 2021 at 6:48 PM, Qu Wenruo - <quwenruo.btrfs@gmx.com> wrote: > On 2021/7/17 下午6:34, pepperpoint@mb.ardentcoding.com wrote: > > > Hi Qu, > > > > Unfortunately I cannot find subvolume 363 > > > > btrfs subvolume list /run/media/root > > ==================================== > > > > ID 361 gen 1814826 top level 584 path @/live/snapshot > > > > ID 364 gen 1814414 top level 5 path @vtmp/live/snapshot > > > > ID 369 gen 1814414 top level 5 path @vlmachines/live/snapshot > > > > ID 493 gen 1814414 top level 5 path @vlportables/live/snapshot > > > > ID 579 gen 1814828 top level 5 path @vlog/live/snapshot > > > > ID 580 gen 1814414 top level 5 path @vcache/live/snapshot > > > > ID 581 gen 1814414 top level 5 path @vlmongodb/live/snapshot > > > > ID 582 gen 1814414 top level 5 path @vlmysql/live/snapshot > > > > ID 583 gen 1814414 top level 5 path @vspool/live/snapshot > > > > ID 584 gen 1814414 top level 5 path @ > > > > ID 598 gen 1813420 top level 584 path @/4/snapshot > > Maybe 363 is some subvolume get deleted and later snapshot of it still > > exists. > > This will be harder to debug. > > Can you take a btrfs-image dump of your filesystem? (needs to be taken > > with the fs unmounted). > > The dumped image will contain your metadata, including file names and > > directory structures, but no data inside those files. > > Although btrfs-image has "-s" option to mask the filenames, but > > considering the filename in this case is useful to locate the inode, I > > guess it's better to take the image without any "-s" option. > > > btrfs ins dump-tree -t 363 /dev/dm-0 | grep -A 5 "(286 " > > ======================================================== > > > > parent transid verify failed on 174170742784 wanted 1789655 found 1812621 > > > > parent transid verify failed on 174170742784 wanted 1789655 found 1812621 > > > > parent transid verify failed on 174170742784 wanted 1789655 found 1812621 > > > > Ignoring transid failure > > > > ERROR: child eb corrupted: parent bytenr=174170738688 item=0 parent level=2 child bytenr=174170742784 child level=0 > > This transid mismatch may be a problem when running dump-tree on mounted > > fs, since you mentioned btrfs check reported no error, there shouldn't > > be a transid mismatch error. > > Anyway, if you can upload the btrfs-image dump, it would be much easier > > for us to debug and find out what's really going. > > Thanks, > > Qu > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > On Saturday, July 17th, 2021 at 6:12 PM, Qu Wenruo wqu@suse.com wrote: > > > > > On 2021/7/17 下午4:57, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > Hi Qu, > > > > > > > > I don't know how the directory was created but last month, I used btrfs device add and btrfs device remove to move the filesystem from one partition to another. It failed because of the same error and was advised to use btrfs replace instead. I don't know if the error also happened before I move the file system as I don't have any previous logs. > > > > > > It definitely happens before you moving the fs. > > > > > > As regular dev replacing/add/move only relocates the metadata, but not > > > > > > touching the fs trees. > > > > > > > Here is the result when I search for the inodes you mentioned if it helps: > > > > > > > > find /run/media/root -inum 260 -exec ls -ldi {} \; > > > > ================================================== > > > > > > > > 260 -rw-r--r-- 1 root root 36864 Jun 25 06:22 /run/media/root/@vcache/live/snapshot/app-info/cache/en_US.cache > > > > > > > > 260 drwx------ 1 mongodb mongodb 136 Sep 12 2020 /run/media/root/@vlmongodb/live/snapshot/diagnostic.data > > > > > > > > 260 -rw-rw---- 1 mysql mysql 50331648 Sep 13 2015 /run/media/root/@vlmysql/live/snapshot/ib_logfile0 > > > > > > > > 260 -rw-r----- 1 root lp 8641 Mar 5 2014 /run/media/root/@vspool/live/snapshot/cups/d00001-001 > > > > > > > > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/live/snapshot/sys > > > > > > > > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/4/snapshot/sys > > > > > > Since btrfs can have the same inode number inside different subvolumes, > > > > > > you may want to limit the search inside subvolume 363. > > > > > > "-mount" option of find can do that, you only need to locate subvolume > > > > > > 363 by "btrfs subv list". > > > > > > But from these output I guess above two "sys" directory are more possible. > > > > > > Is there any directory named "blaklight" inside those directory? > > > > > > > find /run/media/root -inum 286 -exec ls -ldi {} \; > > > > ================================================== > > > > > > > > 286 -rw-r--r-- 1 root root 96 Aug 16 2015 /run/media/root/@vcache/live/snapshot/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-4 > > > > > > > > 286 -rw-rw---- 1 mysql mysql 4096 Sep 15 2013 /run/media/root/@vlmysql/live/snapshot/mysql/columns_priv.MYI > > > > > > > > 286 -rw-r-----+ 1 root systemd-journal 16777216 Jul 4 01:14 /run/media/root/@vlog/live/snapshot/journal/5098dd7845ae46d3ba1826c68a809a7c/user-1000@fbd9f65d0ea349f6b996716280e6c4dd-00000000002314c5-0005c5cb84a3a438.journal > > > > > > This is interesting, it means the inode 286 is not accessible. > > > > > > It can be some orphan inode, but would you dump subvolume 363 then try > > > > > > to locate the inode 286? > > > > > > One example command would be: > > > > > > btrfs ins dump-tree -t 363 <dev> | grep -A 5 "(286 " > > > ==================================================== > > > > > > Thanks, > > > > > > Qu > > > > > > > Directories with pattern /root/@<dir>/live/snapshot/ are subvolumes and directories with pattern /root/@<dir>/<num>/snapshot/ are snapshots of live. > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > On Saturday, July 17th, 2021 at 4:14 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: > > > > > > > > > On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > Please see below for the dump. > > > > > > > > > > > > btrfs-progs v5.12.1 > > > > > > > > > > > > leaf 174113599488 items 18 free space 2008 generation 1330906 owner 363 > > > > > > > > > > > > leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 > > > > > > > > > > > > fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 > > > > > > > > > > > > chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 > > > > > > > > > > > > item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 > > > > > > > > > > > > generation 2063 transid 27726 size 40 nbytes 40 > > > > > > > > > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > > > > > > > > > sequence 1501 flags 0x0(none) > > > > > > > > > > > > atime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > ctime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > mtime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 > > > > > > > > > > > > index 12 namelen 1 name: 8 > > > > > > > > > > > > item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 > > > > > > > > > > > > generation 27726 type 0 (inline) > > > > > > > > > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > > > > > > > > > item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 > > > > > > > > > > > > generation 2542 transid 61261 size 40 nbytes 40 > > > > > > > > > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > > > > > > > > > sequence 24769 flags 0x0(none) > > > > > > > > > > > > atime 1394335806.351857522 (2014-03-09 11:30:06) > > > > > > > > > > > > ctime 1394335827.344389955 (2014-03-09 11:30:27) > > > > > > > > > > > > mtime 1394335827.344389955 (2014-03-09 11:30:27) > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 > > > > > > > > > > > > index 13 namelen 1 name: 7 > > > > > > > > > > > > item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 > > > > > > > > > > > > generation 61261 type 0 (inline) > > > > > > > > > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > > > > > > > > > item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 > > > > > > > > > > > > generation 5754 transid 5767 size 307 nbytes 307 > > > > > > > > > > > > block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 > > > > > > > > > > > > sequence 7 flags 0x0(none) > > > > > > > > > > > > atime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > ctime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > mtime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 > > > > > > > > > > > > index 6 namelen 17 name: dhcpcd-eth0.lease > > > > > > > > > > > > item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 > > > > > > > > > > > > generation 5767 type 0 (inline) > > > > > > > > > > > > inline extent data size 307 ram_bytes 307 compression 0 (none) > > > > > > > > > > > > item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 > > > > > > > > > > > > generation 5904 transid 1330906 size 180 nbytes 0 > > > > > > > > > > > > block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 > > > > > > > > > > > > sequence 177 flags 0x0(none) > > > > > > > > > > > > atime 1483277713.141980592 (2017-01-01 21:35:13) > > > > > > > > > > > > ctime 1563162901.234656246 (2019-07-15 11:55:01) > > > > > > > > > > > > mtime 1406534032.158605559 (2014-07-28 15:53:52) > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > This inode is indeed a directory. > > > > > > > > > > But it has two hard links, which is definitely something unexpected. > > > > > > > > > > Under Linux we shouldn't have any hardlink for directory, as it would > > > > > > > > > > easily lead to loops. > > > > > > > > > > > item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 > > > > > > > > > > > > index 28 namelen 9 name: backlight > > > > > > > > > > Its parent inode is 260 in the same root, with the name backlight. > > > > > > > > > > > item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 > > > > > > > > > > > > index 3 namelen 9 name: backlight > > > > > > > > > > Another hardlink in inode 286, which is definitely a regular thing. > > > > > > > > > > Btrfs-progs lacks the ability to detect such problem, we need to enhance > > > > > > > > > > it first. > > > > > > > > > > But do you have any idea how this directory get created? > > > > > > > > > > It looks like the content of sysfs. > > > > > > > > > > Thanks, > > > > > > > > > > Qu > > > > > > > > > > > item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 > > > > > > > > > > > > location key (120417 INODE_ITEM 0) type FILE > > > > > > > > > > > > transid 117279 data_len 0 name_len 42 > > > > > > > > > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > > > > > > > > > item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 > > > > > > > > > > > > location key (7487 INODE_ITEM 0) type FILE > > > > > > > > > > > > transid 5992 data_len 0 name_len 11 > > > > > > > > > > > > name: acpi_video0 > > > > > > > > > > > > item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 > > > > > > > > > > > > location key (55325 INODE_ITEM 0) type FILE > > > > > > > > > > > > transid 63351 data_len 0 name_len 37 > > > > > > > > > > > > name: platform-VPC2004:00:backlight:ideapad > > > > > > > > > > > > item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 > > > > > > > > > > > > location key (7487 INODE_ITEM 0) type FILE > > > > > > > > > > > > transid 5992 data_len 0 name_len 11 > > > > > > > > > > > > name: acpi_video0 > > > > > > > > > > > > item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 > > > > > > > > > > > > location key (55325 INODE_ITEM 0) type FILE > > > > > > > > > > > > transid 63351 data_len 0 name_len 37 > > > > > > > > > > > > name: platform-VPC2004:00:backlight:ideapad > > > > > > > > > > > > item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 > > > > > > > > > > > > location key (120417 INODE_ITEM 0) type FILE > > > > > > > > > > > > transid 117279 data_len 0 name_len 42 > > > > > > > > > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: > > > > > > > > > > > > > On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > I see this message on dmesg: > > > > > > > > > > > > > > > > [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: root=363 block=174113599488 slot=9 ino=7415, invalid nlink: has 2 expect no more than 1 for dir > > > > > > > > > > > > > > > > [ 2452.256776] BTRFS error (device dm-0): block=174113599488 read time tree block corruption detected > > > > > > > > > > > > > > Please provide the following dump: > > > > > > > > > > > > > > btrfs ins dump-tree -b 174113599488 /dev/dm-0 > > > > > > > ============================================= > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > When I run btrfs scrub and btrfs check, no error was detected. > > > > > > > > > > > > > > > > I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 > > > > > > > > > > > > > > > > How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <CQeY09U34m7SrT6nTAlkSQrbLJtmyKF1tDfuGDtKUkwJqujMI_nZU4MpGiU4F_Q1U3Lk1aWD1mFCT5SlsOsOcILWECflIw7EhVQTQpy_1Ts=@email.ardentcoding.com>]
* Re: Read time tree block corruption detected [not found] ` <CQeY09U34m7SrT6nTAlkSQrbLJtmyKF1tDfuGDtKUkwJqujMI_nZU4MpGiU4F_Q1U3Lk1aWD1mFCT5SlsOsOcILWECflIw7EhVQTQpy_1Ts=@email.ardentcoding.com> @ 2021-07-18 5:26 ` pepperpoint 2021-07-18 7:15 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: pepperpoint @ 2021-07-18 5:26 UTC (permalink / raw) To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs Hi Qu, May I know if there are any leads on this? What should I do for now? Regards, Lester ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Saturday, July 17th, 2021 at 8:51 PM, pepperpoint@mb.ardentcoding.com wrote: > Hi Qu, > > I run btrfs ins dump-tree -t 363 unmounted but the same error appears. Rerunning btrfs check does not show any error. > > Regards, > > Lester > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > On Saturday, July 17th, 2021 at 6:48 PM, Qu Wenruo - quwenruo.btrfs@gmx.com wrote: > > > On 2021/7/17 下午6:34, pepperpoint@mb.ardentcoding.com wrote: > > > > > Hi Qu, > > > > > > Unfortunately I cannot find subvolume 363 > > > > > > btrfs subvolume list /run/media/root > > > ==================================== > > > > > > ID 361 gen 1814826 top level 584 path @/live/snapshot > > > > > > ID 364 gen 1814414 top level 5 path @vtmp/live/snapshot > > > > > > ID 369 gen 1814414 top level 5 path @vlmachines/live/snapshot > > > > > > ID 493 gen 1814414 top level 5 path @vlportables/live/snapshot > > > > > > ID 579 gen 1814828 top level 5 path @vlog/live/snapshot > > > > > > ID 580 gen 1814414 top level 5 path @vcache/live/snapshot > > > > > > ID 581 gen 1814414 top level 5 path @vlmongodb/live/snapshot > > > > > > ID 582 gen 1814414 top level 5 path @vlmysql/live/snapshot > > > > > > ID 583 gen 1814414 top level 5 path @vspool/live/snapshot > > > > > > ID 584 gen 1814414 top level 5 path @ > > > > > > ID 598 gen 1813420 top level 584 path @/4/snapshot > > > > Maybe 363 is some subvolume get deleted and later snapshot of it still > > > > exists. > > > > This will be harder to debug. > > > > Can you take a btrfs-image dump of your filesystem? (needs to be taken > > > > with the fs unmounted). > > > > The dumped image will contain your metadata, including file names and > > > > directory structures, but no data inside those files. > > > > Although btrfs-image has "-s" option to mask the filenames, but > > > > considering the filename in this case is useful to locate the inode, I > > > > guess it's better to take the image without any "-s" option. > > > > > btrfs ins dump-tree -t 363 /dev/dm-0 | grep -A 5 "(286 " > > > ======================================================== > > > > > > parent transid verify failed on 174170742784 wanted 1789655 found 1812621 > > > > > > parent transid verify failed on 174170742784 wanted 1789655 found 1812621 > > > > > > parent transid verify failed on 174170742784 wanted 1789655 found 1812621 > > > > > > Ignoring transid failure > > > > > > ERROR: child eb corrupted: parent bytenr=174170738688 item=0 parent level=2 child bytenr=174170742784 child level=0 > > > > This transid mismatch may be a problem when running dump-tree on mounted > > > > fs, since you mentioned btrfs check reported no error, there shouldn't > > > > be a transid mismatch error. > > > > Anyway, if you can upload the btrfs-image dump, it would be much easier > > > > for us to debug and find out what's really going. > > > > Thanks, > > > > Qu > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > On Saturday, July 17th, 2021 at 6:12 PM, Qu Wenruo wqu@suse.com wrote: > > > > > > > On 2021/7/17 下午4:57, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > Hi Qu, > > > > > > > > > > I don't know how the directory was created but last month, I used btrfs device add and btrfs device remove to move the filesystem from one partition to another. It failed because of the same error and was advised to use btrfs replace instead. I don't know if the error also happened before I move the file system as I don't have any previous logs. > > > > > > > > It definitely happens before you moving the fs. > > > > > > > > As regular dev replacing/add/move only relocates the metadata, but not > > > > > > > > touching the fs trees. > > > > > > > > > Here is the result when I search for the inodes you mentioned if it helps: > > > > > > > > > > find /run/media/root -inum 260 -exec ls -ldi {} \; > > > > > ================================================== > > > > > > > > > > 260 -rw-r--r-- 1 root root 36864 Jun 25 06:22 /run/media/root/@vcache/live/snapshot/app-info/cache/en_US.cache > > > > > > > > > > 260 drwx------ 1 mongodb mongodb 136 Sep 12 2020 /run/media/root/@vlmongodb/live/snapshot/diagnostic.data > > > > > > > > > > 260 -rw-rw---- 1 mysql mysql 50331648 Sep 13 2015 /run/media/root/@vlmysql/live/snapshot/ib_logfile0 > > > > > > > > > > 260 -rw-r----- 1 root lp 8641 Mar 5 2014 /run/media/root/@vspool/live/snapshot/cups/d00001-001 > > > > > > > > > > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/live/snapshot/sys > > > > > > > > > > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/4/snapshot/sys > > > > > > > > Since btrfs can have the same inode number inside different subvolumes, > > > > > > > > you may want to limit the search inside subvolume 363. > > > > > > > > "-mount" option of find can do that, you only need to locate subvolume > > > > > > > > 363 by "btrfs subv list". > > > > > > > > But from these output I guess above two "sys" directory are more possible. > > > > > > > > Is there any directory named "blaklight" inside those directory? > > > > > > > > > find /run/media/root -inum 286 -exec ls -ldi {} \; > > > > > ================================================== > > > > > > > > > > 286 -rw-r--r-- 1 root root 96 Aug 16 2015 /run/media/root/@vcache/live/snapshot/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-4 > > > > > > > > > > 286 -rw-rw---- 1 mysql mysql 4096 Sep 15 2013 /run/media/root/@vlmysql/live/snapshot/mysql/columns_priv.MYI > > > > > > > > > > 286 -rw-r-----+ 1 root systemd-journal 16777216 Jul 4 01:14 /run/media/root/@vlog/live/snapshot/journal/5098dd7845ae46d3ba1826c68a809a7c/user-1000@fbd9f65d0ea349f6b996716280e6c4dd-00000000002314c5-0005c5cb84a3a438.journal > > > > > > > > This is interesting, it means the inode 286 is not accessible. > > > > > > > > It can be some orphan inode, but would you dump subvolume 363 then try > > > > > > > > to locate the inode 286? > > > > > > > > One example command would be: > > > > > > > > btrfs ins dump-tree -t 363 <dev> | grep -A 5 "(286 " > > > > ==================================================== > > > > > > > > Thanks, > > > > > > > > Qu > > > > > > > > > Directories with pattern /root/@<dir>/live/snapshot/ are subvolumes and directories with pattern /root/@<dir>/<num>/snapshot/ are snapshots of live. > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > On Saturday, July 17th, 2021 at 4:14 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: > > > > > > > > > > > On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > > > Please see below for the dump. > > > > > > > > > > > > > > btrfs-progs v5.12.1 > > > > > > > > > > > > > > leaf 174113599488 items 18 free space 2008 generation 1330906 owner 363 > > > > > > > > > > > > > > leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 > > > > > > > > > > > > > > fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 > > > > > > > > > > > > > > chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 > > > > > > > > > > > > > > item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 > > > > > > > > > > > > > > generation 2063 transid 27726 size 40 nbytes 40 > > > > > > > > > > > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > > > > > > > > > > > sequence 1501 flags 0x0(none) > > > > > > > > > > > > > > atime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > > > ctime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > > > mtime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 > > > > > > > > > > > > > > index 12 namelen 1 name: 8 > > > > > > > > > > > > > > item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 > > > > > > > > > > > > > > generation 27726 type 0 (inline) > > > > > > > > > > > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > > > > > > > > > > > item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 > > > > > > > > > > > > > > generation 2542 transid 61261 size 40 nbytes 40 > > > > > > > > > > > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > > > > > > > > > > > sequence 24769 flags 0x0(none) > > > > > > > > > > > > > > atime 1394335806.351857522 (2014-03-09 11:30:06) > > > > > > > > > > > > > > ctime 1394335827.344389955 (2014-03-09 11:30:27) > > > > > > > > > > > > > > mtime 1394335827.344389955 (2014-03-09 11:30:27) > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 > > > > > > > > > > > > > > index 13 namelen 1 name: 7 > > > > > > > > > > > > > > item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 > > > > > > > > > > > > > > generation 61261 type 0 (inline) > > > > > > > > > > > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > > > > > > > > > > > item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 > > > > > > > > > > > > > > generation 5754 transid 5767 size 307 nbytes 307 > > > > > > > > > > > > > > block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 > > > > > > > > > > > > > > sequence 7 flags 0x0(none) > > > > > > > > > > > > > > atime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > > > ctime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > > > mtime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 > > > > > > > > > > > > > > index 6 namelen 17 name: dhcpcd-eth0.lease > > > > > > > > > > > > > > item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 > > > > > > > > > > > > > > generation 5767 type 0 (inline) > > > > > > > > > > > > > > inline extent data size 307 ram_bytes 307 compression 0 (none) > > > > > > > > > > > > > > item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 > > > > > > > > > > > > > > generation 5904 transid 1330906 size 180 nbytes 0 > > > > > > > > > > > > > > block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 > > > > > > > > > > > > > > sequence 177 flags 0x0(none) > > > > > > > > > > > > > > atime 1483277713.141980592 (2017-01-01 21:35:13) > > > > > > > > > > > > > > ctime 1563162901.234656246 (2019-07-15 11:55:01) > > > > > > > > > > > > > > mtime 1406534032.158605559 (2014-07-28 15:53:52) > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > This inode is indeed a directory. > > > > > > > > > > > > But it has two hard links, which is definitely something unexpected. > > > > > > > > > > > > Under Linux we shouldn't have any hardlink for directory, as it would > > > > > > > > > > > > easily lead to loops. > > > > > > > > > > > > > item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 > > > > > > > > > > > > > > index 28 namelen 9 name: backlight > > > > > > > > > > > > Its parent inode is 260 in the same root, with the name backlight. > > > > > > > > > > > > > item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 > > > > > > > > > > > > > > index 3 namelen 9 name: backlight > > > > > > > > > > > > Another hardlink in inode 286, which is definitely a regular thing. > > > > > > > > > > > > Btrfs-progs lacks the ability to detect such problem, we need to enhance > > > > > > > > > > > > it first. > > > > > > > > > > > > But do you have any idea how this directory get created? > > > > > > > > > > > > It looks like the content of sysfs. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Qu > > > > > > > > > > > > > item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 > > > > > > > > > > > > > > location key (120417 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > transid 117279 data_len 0 name_len 42 > > > > > > > > > > > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > > > > > > > > > > > item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 > > > > > > > > > > > > > > location key (7487 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > transid 5992 data_len 0 name_len 11 > > > > > > > > > > > > > > name: acpi_video0 > > > > > > > > > > > > > > item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 > > > > > > > > > > > > > > location key (55325 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > transid 63351 data_len 0 name_len 37 > > > > > > > > > > > > > > name: platform-VPC2004:00:backlight:ideapad > > > > > > > > > > > > > > item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 > > > > > > > > > > > > > > location key (7487 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > transid 5992 data_len 0 name_len 11 > > > > > > > > > > > > > > name: acpi_video0 > > > > > > > > > > > > > > item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 > > > > > > > > > > > > > > location key (55325 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > transid 63351 data_len 0 name_len 37 > > > > > > > > > > > > > > name: platform-VPC2004:00:backlight:ideapad > > > > > > > > > > > > > > item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 > > > > > > > > > > > > > > location key (120417 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > transid 117279 data_len 0 name_len 42 > > > > > > > > > > > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > > > On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: > > > > > > > > > > > > > > > On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > I see this message on dmesg: > > > > > > > > > > > > > > > > > > [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: root=363 block=174113599488 slot=9 ino=7415, invalid nlink: has 2 expect no more than 1 for dir > > > > > > > > > > > > > > > > > > [ 2452.256776] BTRFS error (device dm-0): block=174113599488 read time tree block corruption detected > > > > > > > > > > > > > > > > Please provide the following dump: > > > > > > > > > > > > > > > > btrfs ins dump-tree -b 174113599488 /dev/dm-0 > > > > > > > > ============================================= > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > > > When I run btrfs scrub and btrfs check, no error was detected. > > > > > > > > > > > > > > > > > > I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 > > > > > > > > > > > > > > > > > > How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: Read time tree block corruption detected 2021-07-18 5:26 ` pepperpoint @ 2021-07-18 7:15 ` Qu Wenruo [not found] ` <162659327011.8.12718249358254503695.10218325@simplelogin.co> 0 siblings, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2021-07-18 7:15 UTC (permalink / raw) To: pepperpoint; +Cc: Qu Wenruo, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 15135 bytes --] On 2021/7/18 下午1:26, pepperpoint@mb.ardentcoding.com wrote: > Hi Qu, > > May I know if there are any leads on this? What should I do for now? Sorry for the late reply. With the image dump, it's much easier to find what's going wrong. - About root 363 It's an orphan root, thus user can't access it directly. Furthermore, it's being dropped, thus "btrfs ins dump-tree -t 363" reports transid error, as part of the tree has already been dropped, and this is expected. So far your fs is still OK, except that reported error. - About the offending tree block The offending tree block only belongs to the delete subvolume 363, thus it should be delete soon. But unfortunately due to the corrupted content, it's unable to be deleted. For now, if you can re-compile btrfs module, we can workaround the problem by temporarily disable read-time tree-checker so that the deletion can continue, and after the root 363 get fully deleted, the problem should be gone. Or you can use older kernel, any kernel <= v5.1 should not have the enhanced check, thus can continue with the subvolume deletion. If you want to go through the re-compile path, the needed diff is attached Thanks, Qu > > Regards, > Lester > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > On Saturday, July 17th, 2021 at 8:51 PM, pepperpoint@mb.ardentcoding.com wrote: > >> Hi Qu, >> >> I run btrfs ins dump-tree -t 363 unmounted but the same error appears. Rerunning btrfs check does not show any error. >> >> Regards, >> >> Lester >> >> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >> >> On Saturday, July 17th, 2021 at 6:48 PM, Qu Wenruo - quwenruo.btrfs@gmx.com wrote: >> >>> On 2021/7/17 下午6:34, pepperpoint@mb.ardentcoding.com wrote: >>> >>>> Hi Qu, >>>> >>>> Unfortunately I cannot find subvolume 363 >>>> >>>> btrfs subvolume list /run/media/root >>>> ==================================== >>>> >>>> ID 361 gen 1814826 top level 584 path @/live/snapshot >>>> >>>> ID 364 gen 1814414 top level 5 path @vtmp/live/snapshot >>>> >>>> ID 369 gen 1814414 top level 5 path @vlmachines/live/snapshot >>>> >>>> ID 493 gen 1814414 top level 5 path @vlportables/live/snapshot >>>> >>>> ID 579 gen 1814828 top level 5 path @vlog/live/snapshot >>>> >>>> ID 580 gen 1814414 top level 5 path @vcache/live/snapshot >>>> >>>> ID 581 gen 1814414 top level 5 path @vlmongodb/live/snapshot >>>> >>>> ID 582 gen 1814414 top level 5 path @vlmysql/live/snapshot >>>> >>>> ID 583 gen 1814414 top level 5 path @vspool/live/snapshot >>>> >>>> ID 584 gen 1814414 top level 5 path @ >>>> >>>> ID 598 gen 1813420 top level 584 path @/4/snapshot >>> >>> Maybe 363 is some subvolume get deleted and later snapshot of it still >>> >>> exists. >>> >>> This will be harder to debug. >>> >>> Can you take a btrfs-image dump of your filesystem? (needs to be taken >>> >>> with the fs unmounted). >>> >>> The dumped image will contain your metadata, including file names and >>> >>> directory structures, but no data inside those files. >>> >>> Although btrfs-image has "-s" option to mask the filenames, but >>> >>> considering the filename in this case is useful to locate the inode, I >>> >>> guess it's better to take the image without any "-s" option. >>> >>>> btrfs ins dump-tree -t 363 /dev/dm-0 | grep -A 5 "(286 " >>>> ======================================================== >>>> >>>> parent transid verify failed on 174170742784 wanted 1789655 found 1812621 >>>> >>>> parent transid verify failed on 174170742784 wanted 1789655 found 1812621 >>>> >>>> parent transid verify failed on 174170742784 wanted 1789655 found 1812621 >>>> >>>> Ignoring transid failure >>>> >>>> ERROR: child eb corrupted: parent bytenr=174170738688 item=0 parent level=2 child bytenr=174170742784 child level=0 >>> >>> This transid mismatch may be a problem when running dump-tree on mounted >>> >>> fs, since you mentioned btrfs check reported no error, there shouldn't >>> >>> be a transid mismatch error. >>> >>> Anyway, if you can upload the btrfs-image dump, it would be much easier >>> >>> for us to debug and find out what's really going. >>> >>> Thanks, >>> >>> Qu >>> >>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>> >>>> On Saturday, July 17th, 2021 at 6:12 PM, Qu Wenruo wqu@suse.com wrote: >>>> >>>>> On 2021/7/17 下午4:57, pepperpoint@mb.ardentcoding.com wrote: >>>>> >>>>>> Hi Qu, >>>>>> >>>>>> I don't know how the directory was created but last month, I used btrfs device add and btrfs device remove to move the filesystem from one partition to another. It failed because of the same error and was advised to use btrfs replace instead. I don't know if the error also happened before I move the file system as I don't have any previous logs. >>>>> >>>>> It definitely happens before you moving the fs. >>>>> >>>>> As regular dev replacing/add/move only relocates the metadata, but not >>>>> >>>>> touching the fs trees. >>>>> >>>>>> Here is the result when I search for the inodes you mentioned if it helps: >>>>>> >>>>>> find /run/media/root -inum 260 -exec ls -ldi {} \; >>>>>> ================================================== >>>>>> >>>>>> 260 -rw-r--r-- 1 root root 36864 Jun 25 06:22 /run/media/root/@vcache/live/snapshot/app-info/cache/en_US.cache >>>>>> >>>>>> 260 drwx------ 1 mongodb mongodb 136 Sep 12 2020 /run/media/root/@vlmongodb/live/snapshot/diagnostic.data >>>>>> >>>>>> 260 -rw-rw---- 1 mysql mysql 50331648 Sep 13 2015 /run/media/root/@vlmysql/live/snapshot/ib_logfile0 >>>>>> >>>>>> 260 -rw-r----- 1 root lp 8641 Mar 5 2014 /run/media/root/@vspool/live/snapshot/cups/d00001-001 >>>>>> >>>>>> 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/live/snapshot/sys >>>>>> >>>>>> 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 /run/media/root/@/4/snapshot/sys >>>>> >>>>> Since btrfs can have the same inode number inside different subvolumes, >>>>> >>>>> you may want to limit the search inside subvolume 363. >>>>> >>>>> "-mount" option of find can do that, you only need to locate subvolume >>>>> >>>>> 363 by "btrfs subv list". >>>>> >>>>> But from these output I guess above two "sys" directory are more possible. >>>>> >>>>> Is there any directory named "blaklight" inside those directory? >>>>> >>>>>> find /run/media/root -inum 286 -exec ls -ldi {} \; >>>>>> ================================================== >>>>>> >>>>>> 286 -rw-r--r-- 1 root root 96 Aug 16 2015 /run/media/root/@vcache/live/snapshot/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-4 >>>>>> >>>>>> 286 -rw-rw---- 1 mysql mysql 4096 Sep 15 2013 /run/media/root/@vlmysql/live/snapshot/mysql/columns_priv.MYI >>>>>> >>>>>> 286 -rw-r-----+ 1 root systemd-journal 16777216 Jul 4 01:14 /run/media/root/@vlog/live/snapshot/journal/5098dd7845ae46d3ba1826c68a809a7c/user-1000@fbd9f65d0ea349f6b996716280e6c4dd-00000000002314c5-0005c5cb84a3a438.journal >>>>> >>>>> This is interesting, it means the inode 286 is not accessible. >>>>> >>>>> It can be some orphan inode, but would you dump subvolume 363 then try >>>>> >>>>> to locate the inode 286? >>>>> >>>>> One example command would be: >>>>> >>>>> btrfs ins dump-tree -t 363 <dev> | grep -A 5 "(286 " >>>>> ==================================================== >>>>> >>>>> Thanks, >>>>> >>>>> Qu >>>>> >>>>>> Directories with pattern /root/@<dir>/live/snapshot/ are subvolumes and directories with pattern /root/@<dir>/<num>/snapshot/ are snapshots of live. >>>>>> >>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>>> >>>>>> On Saturday, July 17th, 2021 at 4:14 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: >>>>>> >>>>>>> On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: >>>>>>> >>>>>>>> Hi Qu, >>>>>>>> >>>>>>>> Please see below for the dump. >>>>>>>> >>>>>>>> btrfs-progs v5.12.1 >>>>>>>> >>>>>>>> leaf 174113599488 items 18 free space 2008 generation 1330906 owner 363 >>>>>>>> >>>>>>>> leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 >>>>>>>> >>>>>>>> fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 >>>>>>>> >>>>>>>> chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 >>>>>>>> >>>>>>>> item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 >>>>>>>> >>>>>>>> generation 2063 transid 27726 size 40 nbytes 40 >>>>>>>> >>>>>>>> block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 >>>>>>>> >>>>>>>> sequence 1501 flags 0x0(none) >>>>>>>> >>>>>>>> atime 1386484844.468769570 (2013-12-08 14:40:44) >>>>>>>> >>>>>>>> ctime 1386484844.468769570 (2013-12-08 14:40:44) >>>>>>>> >>>>>>>> mtime 1386484844.468769570 (2013-12-08 14:40:44) >>>>>>>> >>>>>>>> otime 0.0 (1970-01-01 08:00:00) >>>>>>>> >>>>>>>> item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 >>>>>>>> >>>>>>>> index 12 namelen 1 name: 8 >>>>>>>> >>>>>>>> item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 >>>>>>>> >>>>>>>> generation 27726 type 0 (inline) >>>>>>>> >>>>>>>> inline extent data size 40 ram_bytes 40 compression 0 (none) >>>>>>>> >>>>>>>> item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 >>>>>>>> >>>>>>>> generation 2542 transid 61261 size 40 nbytes 40 >>>>>>>> >>>>>>>> block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 >>>>>>>> >>>>>>>> sequence 24769 flags 0x0(none) >>>>>>>> >>>>>>>> atime 1394335806.351857522 (2014-03-09 11:30:06) >>>>>>>> >>>>>>>> ctime 1394335827.344389955 (2014-03-09 11:30:27) >>>>>>>> >>>>>>>> mtime 1394335827.344389955 (2014-03-09 11:30:27) >>>>>>>> >>>>>>>> otime 0.0 (1970-01-01 08:00:00) >>>>>>>> >>>>>>>> item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 >>>>>>>> >>>>>>>> index 13 namelen 1 name: 7 >>>>>>>> >>>>>>>> item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 >>>>>>>> >>>>>>>> generation 61261 type 0 (inline) >>>>>>>> >>>>>>>> inline extent data size 40 ram_bytes 40 compression 0 (none) >>>>>>>> >>>>>>>> item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 >>>>>>>> >>>>>>>> generation 5754 transid 5767 size 307 nbytes 307 >>>>>>>> >>>>>>>> block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 >>>>>>>> >>>>>>>> sequence 7 flags 0x0(none) >>>>>>>> >>>>>>>> atime 1379834835.428558020 (2013-09-22 15:27:15) >>>>>>>> >>>>>>>> ctime 1379834835.428558020 (2013-09-22 15:27:15) >>>>>>>> >>>>>>>> mtime 1379834835.428558020 (2013-09-22 15:27:15) >>>>>>>> >>>>>>>> otime 0.0 (1970-01-01 08:00:00) >>>>>>>> >>>>>>>> item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 >>>>>>>> >>>>>>>> index 6 namelen 17 name: dhcpcd-eth0.lease >>>>>>>> >>>>>>>> item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 >>>>>>>> >>>>>>>> generation 5767 type 0 (inline) >>>>>>>> >>>>>>>> inline extent data size 307 ram_bytes 307 compression 0 (none) >>>>>>>> >>>>>>>> item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 >>>>>>>> >>>>>>>> generation 5904 transid 1330906 size 180 nbytes 0 >>>>>>>> >>>>>>>> block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 >>>>>>>> >>>>>>>> sequence 177 flags 0x0(none) >>>>>>>> >>>>>>>> atime 1483277713.141980592 (2017-01-01 21:35:13) >>>>>>>> >>>>>>>> ctime 1563162901.234656246 (2019-07-15 11:55:01) >>>>>>>> >>>>>>>> mtime 1406534032.158605559 (2014-07-28 15:53:52) >>>>>>>> >>>>>>>> otime 0.0 (1970-01-01 08:00:00) >>>>>>> >>>>>>> This inode is indeed a directory. >>>>>>> >>>>>>> But it has two hard links, which is definitely something unexpected. >>>>>>> >>>>>>> Under Linux we shouldn't have any hardlink for directory, as it would >>>>>>> >>>>>>> easily lead to loops. >>>>>>> >>>>>>>> item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 >>>>>>>> >>>>>>>> index 28 namelen 9 name: backlight >>>>>>> >>>>>>> Its parent inode is 260 in the same root, with the name backlight. >>>>>>> >>>>>>>> item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 >>>>>>>> >>>>>>>> index 3 namelen 9 name: backlight >>>>>>> >>>>>>> Another hardlink in inode 286, which is definitely a regular thing. >>>>>>> >>>>>>> Btrfs-progs lacks the ability to detect such problem, we need to enhance >>>>>>> >>>>>>> it first. >>>>>>> >>>>>>> But do you have any idea how this directory get created? >>>>>>> >>>>>>> It looks like the content of sysfs. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Qu >>>>>>> >>>>>>>> item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 >>>>>>>> >>>>>>>> location key (120417 INODE_ITEM 0) type FILE >>>>>>>> >>>>>>>> transid 117279 data_len 0 name_len 42 >>>>>>>> >>>>>>>> name: pci-0000:00:02.0:backlight:intel_backlight >>>>>>>> >>>>>>>> item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 >>>>>>>> >>>>>>>> location key (7487 INODE_ITEM 0) type FILE >>>>>>>> >>>>>>>> transid 5992 data_len 0 name_len 11 >>>>>>>> >>>>>>>> name: acpi_video0 >>>>>>>> >>>>>>>> item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 >>>>>>>> >>>>>>>> location key (55325 INODE_ITEM 0) type FILE >>>>>>>> >>>>>>>> transid 63351 data_len 0 name_len 37 >>>>>>>> >>>>>>>> name: platform-VPC2004:00:backlight:ideapad >>>>>>>> >>>>>>>> item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 >>>>>>>> >>>>>>>> location key (7487 INODE_ITEM 0) type FILE >>>>>>>> >>>>>>>> transid 5992 data_len 0 name_len 11 >>>>>>>> >>>>>>>> name: acpi_video0 >>>>>>>> >>>>>>>> item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 >>>>>>>> >>>>>>>> location key (55325 INODE_ITEM 0) type FILE >>>>>>>> >>>>>>>> transid 63351 data_len 0 name_len 37 >>>>>>>> >>>>>>>> name: platform-VPC2004:00:backlight:ideapad >>>>>>>> >>>>>>>> item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 >>>>>>>> >>>>>>>> location key (120417 INODE_ITEM 0) type FILE >>>>>>>> >>>>>>>> transid 117279 data_len 0 name_len 42 >>>>>>>> >>>>>>>> name: pci-0000:00:02.0:backlight:intel_backlight >>>>>>>> >>>>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>>>>> >>>>>>>> On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: >>>>>>>> >>>>>>>>> On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: >>>>>>>>> >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> I see this message on dmesg: >>>>>>>>>> >>>>>>>>>> [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: root=363 block=174113599488 slot=9 ino=7415, invalid nlink: has 2 expect no more than 1 for dir >>>>>>>>>> >>>>>>>>>> [ 2452.256776] BTRFS error (device dm-0): block=174113599488 read time tree block corruption detected >>>>>>>>> >>>>>>>>> Please provide the following dump: >>>>>>>>> >>>>>>>>> btrfs ins dump-tree -b 174113599488 /dev/dm-0 >>>>>>>>> ============================================= >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Qu >>>>>>>>> >>>>>>>>>> When I run btrfs scrub and btrfs check, no error was detected. >>>>>>>>>> >>>>>>>>>> I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 >>>>>>>>>> >>>>>>>>>> How should I fix this error? [-- Attachment #2: diff --] [-- Type: text/plain, Size: 660 bytes --] diff --git a/fs/btrfs/tree-checker.c b/fs/btrfs/tree-checker.c index a8b2e0d2c025..99b4309d6db6 100644 --- a/fs/btrfs/tree-checker.c +++ b/fs/btrfs/tree-checker.c @@ -1048,12 +1048,14 @@ static int check_inode_item(struct extent_buffer *leaf, return -EUCLEAN; } } + /* if (unlikely(S_ISDIR(mode) && btrfs_inode_nlink(leaf, iitem) > 1)) { inode_item_err(leaf, slot, "invalid nlink: has %u expect no more than 1 for dir", btrfs_inode_nlink(leaf, iitem)); return -EUCLEAN; } + */ if (unlikely(btrfs_inode_flags(leaf, iitem) & ~BTRFS_INODE_FLAG_MASK)) { inode_item_err(leaf, slot, "unknown flags detected: 0x%llx", ^ permalink raw reply related [flat|nested] 75+ messages in thread
[parent not found: <162659327011.8.12718249358254503695.10218325@simplelogin.co>]
* Re: Read time tree block corruption detected [not found] ` <162659327011.8.12718249358254503695.10218325@simplelogin.co> @ 2021-07-18 8:46 ` pepperpoint [not found] ` <162659797656.6.14385932343235367381.10220447@mb.ardentcoding.com> 1 sibling, 0 replies; 75+ messages in thread From: pepperpoint @ 2021-07-18 8:46 UTC (permalink / raw) To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs Hi Qu, When I find which directory some of the filenames are located, they are under /var/lib. I had subvolume for /var which was created probably around 2018. I don't remember how I created this but I probably use rsync to copy the files from existing /var or created a snapshot of root and delete other files that is not under /var. Around June, I tried to move the filesystem to another partition through btrfs device add and btrfs device remove but failed due to that error and was advised to use btrfs replace instead. Then at the beginning of this month, I reorganized it merging most of the /var content back to root and created subvolume for /var/lib/mysql and /var/lib/mongodb. I encountered an error when I copy some of the files through cp --reflink but I failed for /var/lib/mysql so I created a snapshot from /var and remove the extra files. This is also the time I saw the errors in the log. Before that, the errors was not in the log. Regards, Lester ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Sunday, July 18th, 2021 at 3:27 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > Hi, > > BTW, it's really important for us to know how the directory is hardlinked. > > Thus I salvaged the filenames found in the half-dropped root 363. > > Since it may contain confidential info, I send the filename list to you > > off-list. > > If you can remind what the root 363 is used for, and any possible > > operations which may be involved in that subvolume, it's better to reply > > it to the mail list so that we can get some clue on the root cause. > > Thanks, > > Qu > > On 2021/7/18 下午3:15, Qu Wenruo wrote: > > > On 2021/7/18 下午1:26, pepperpoint@mb.ardentcoding.com wrote: > > > > > Hi Qu, > > > > > > May I know if there are any leads on this? What should I do for now? > > > > Sorry for the late reply. > > > > With the image dump, it's much easier to find what's going wrong. > > > > - About root 363 > > > > It's an orphan root, thus user can't access it directly. > > > > Furthermore, it's being dropped, thus "btrfs ins dump-tree -t 363" > > > > reports transid error, as part of the tree has already been dropped, > > > > and this is expected. > > > > So far your fs is still OK, except that reported error. > > > > - About the offending tree block > > > > The offending tree block only belongs to the delete subvolume 363, > > > > thus it should be delete soon. > > > > But unfortunately due to the corrupted content, it's unable to be > > > > deleted. > > > > For now, if you can re-compile btrfs module, we can workaround the > > > > problem by temporarily disable read-time tree-checker so that the > > > > deletion can continue, and after the root 363 get fully deleted, the > > > > problem should be gone. > > > > Or you can use older kernel, any kernel <= v5.1 should not have the > > > > enhanced check, thus can continue with the subvolume deletion. > > > > If you want to go through the re-compile path, the needed diff is > > > > attached > > > > Thanks, > > > > Qu > > > > > Regards, > > > > > > Lester > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > On Saturday, July 17th, 2021 at 8:51 PM, > > > > > > pepperpoint@mb.ardentcoding.com wrote: > > > > > > > Hi Qu, > > > > > > > > I run btrfs ins dump-tree -t 363 unmounted but the same error > > > > > > > > appears. Rerunning btrfs check does not show any error. > > > > > > > > Regards, > > > > > > > > Lester > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > On Saturday, July 17th, 2021 at 6:48 PM, Qu Wenruo - > > > > > > > > quwenruo.btrfs@gmx.com wrote: > > > > > > > > > On 2021/7/17 下午6:34, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > Unfortunately I cannot find subvolume 363 > > > > > > > > > > > > btrfs subvolume list /run/media/root > > > > > > ==================================== > > > > > > > > > > > > ID 361 gen 1814826 top level 584 path @/live/snapshot > > > > > > > > > > > > ID 364 gen 1814414 top level 5 path @vtmp/live/snapshot > > > > > > > > > > > > ID 369 gen 1814414 top level 5 path @vlmachines/live/snapshot > > > > > > > > > > > > ID 493 gen 1814414 top level 5 path @vlportables/live/snapshot > > > > > > > > > > > > ID 579 gen 1814828 top level 5 path @vlog/live/snapshot > > > > > > > > > > > > ID 580 gen 1814414 top level 5 path @vcache/live/snapshot > > > > > > > > > > > > ID 581 gen 1814414 top level 5 path @vlmongodb/live/snapshot > > > > > > > > > > > > ID 582 gen 1814414 top level 5 path @vlmysql/live/snapshot > > > > > > > > > > > > ID 583 gen 1814414 top level 5 path @vspool/live/snapshot > > > > > > > > > > > > ID 584 gen 1814414 top level 5 path @ > > > > > > > > > > > > ID 598 gen 1813420 top level 584 path @/4/snapshot > > > > > > > > > > Maybe 363 is some subvolume get deleted and later snapshot of it still > > > > > > > > > > exists. > > > > > > > > > > This will be harder to debug. > > > > > > > > > > Can you take a btrfs-image dump of your filesystem? (needs to be taken > > > > > > > > > > with the fs unmounted). > > > > > > > > > > The dumped image will contain your metadata, including file names and > > > > > > > > > > directory structures, but no data inside those files. > > > > > > > > > > Although btrfs-image has "-s" option to mask the filenames, but > > > > > > > > > > considering the filename in this case is useful to locate the inode, I > > > > > > > > > > guess it's better to take the image without any "-s" option. > > > > > > > > > > > btrfs ins dump-tree -t 363 /dev/dm-0 | grep -A 5 "(286 " > > > > > > ======================================================== > > > > > > > > > > > > parent transid verify failed on 174170742784 wanted 1789655 found > > > > > > > > > > > > 1812621 > > > > > > > > > > > > parent transid verify failed on 174170742784 wanted 1789655 found > > > > > > > > > > > > 1812621 > > > > > > > > > > > > parent transid verify failed on 174170742784 wanted 1789655 found > > > > > > > > > > > > 1812621 > > > > > > > > > > > > Ignoring transid failure > > > > > > > > > > > > ERROR: child eb corrupted: parent bytenr=174170738688 item=0 parent > > > > > > > > > > > > level=2 child bytenr=174170742784 child level=0 > > > > > > > > > > This transid mismatch may be a problem when running dump-tree on > > > > > > > > > > mounted > > > > > > > > > > fs, since you mentioned btrfs check reported no error, there shouldn't > > > > > > > > > > be a transid mismatch error. > > > > > > > > > > Anyway, if you can upload the btrfs-image dump, it would be much easier > > > > > > > > > > for us to debug and find out what's really going. > > > > > > > > > > Thanks, > > > > > > > > > > Qu > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > On Saturday, July 17th, 2021 at 6:12 PM, Qu Wenruo wqu@suse.com wrote: > > > > > > > > > > > > > On 2021/7/17 下午4:57, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > > > > > I don't know how the directory was created but last month, I used > > > > > > > > > > > > > > > > btrfs device add and btrfs device remove to move the filesystem > > > > > > > > > > > > > > > > from one partition to another. It failed because of the same > > > > > > > > > > > > > > > > error and was advised to use btrfs replace instead. I don't know > > > > > > > > > > > > > > > > if the error also happened before I move the file system as I > > > > > > > > > > > > > > > > don't have any previous logs. > > > > > > > > > > > > > > It definitely happens before you moving the fs. > > > > > > > > > > > > > > As regular dev replacing/add/move only relocates the metadata, but > > > > > > > > > > > > > > not > > > > > > > > > > > > > > touching the fs trees. > > > > > > > > > > > > > > > Here is the result when I search for the inodes you mentioned if > > > > > > > > > > > > > > > > it helps: > > > > > > > > > > > > > > > > find /run/media/root -inum 260 -exec ls -ldi {} \; > > > > > > > > ================================================== > > > > > > > > > > > > > > > > 260 -rw-r--r-- 1 root root 36864 Jun 25 06:22 > > > > > > > > > > > > > > > > /run/media/root/@vcache/live/snapshot/app-info/cache/en_US.cache > > > > > > > > > > > > > > > > 260 drwx------ 1 mongodb mongodb 136 Sep 12 2020 > > > > > > > > > > > > > > > > /run/media/root/@vlmongodb/live/snapshot/diagnostic.data > > > > > > > > > > > > > > > > 260 -rw-rw---- 1 mysql mysql 50331648 Sep 13 2015 > > > > > > > > > > > > > > > > /run/media/root/@vlmysql/live/snapshot/ib_logfile0 > > > > > > > > > > > > > > > > 260 -rw-r----- 1 root lp 8641 Mar 5 2014 > > > > > > > > > > > > > > > > /run/media/root/@vspool/live/snapshot/cups/d00001-001 > > > > > > > > > > > > > > > > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 > > > > > > > > > > > > > > > > /run/media/root/@/live/snapshot/sys > > > > > > > > > > > > > > > > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 > > > > > > > > > > > > > > > > /run/media/root/@/4/snapshot/sys > > > > > > > > > > > > > > Since btrfs can have the same inode number inside different > > > > > > > > > > > > > > subvolumes, > > > > > > > > > > > > > > you may want to limit the search inside subvolume 363. > > > > > > > > > > > > > > "-mount" option of find can do that, you only need to locate > > > > > > > > > > > > > > subvolume > > > > > > > > > > > > > > 363 by "btrfs subv list". > > > > > > > > > > > > > > But from these output I guess above two "sys" directory are more > > > > > > > > > > > > > > possible. > > > > > > > > > > > > > > Is there any directory named "blaklight" inside those directory? > > > > > > > > > > > > > > > find /run/media/root -inum 286 -exec ls -ldi {} \; > > > > > > > > ================================================== > > > > > > > > > > > > > > > > 286 -rw-r--r-- 1 root root 96 Aug 16 2015 > > > > > > > > > > > > > > > > /run/media/root/@vcache/live/snapshot/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-4 > > > > > > > > > > > > > > > > 286 -rw-rw---- 1 mysql mysql 4096 Sep 15 2013 > > > > > > > > > > > > > > > > /run/media/root/@vlmysql/live/snapshot/mysql/columns_priv.MYI > > > > > > > > > > > > > > > > 286 -rw-r-----+ 1 root systemd-journal 16777216 Jul 4 01:14 > > > > > > > > > > > > > > > > /run/media/root/@vlog/live/snapshot/journal/5098dd7845ae46d3ba1826c68a809a7c/user-1000@fbd9f65d0ea349f6b996716280e6c4dd-00000000002314c5-0005c5cb84a3a438.journal > > > > > > > > > > > > > > This is interesting, it means the inode 286 is not accessible. > > > > > > > > > > > > > > It can be some orphan inode, but would you dump subvolume 363 then > > > > > > > > > > > > > > try > > > > > > > > > > > > > > to locate the inode 286? > > > > > > > > > > > > > > One example command would be: > > > > > > > > > > > > > > btrfs ins dump-tree -t 363 <dev> | grep -A 5 "(286 " > > > > > > > ==================================================== > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > Directories with pattern /root/@<dir>/live/snapshot/ are > > > > > > > > > > > > > > > > subvolumes and directories with pattern > > > > > > > > > > > > > > > > /root/@<dir>/<num>/snapshot/ are snapshots of live. > > > > > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > > > > > On Saturday, July 17th, 2021 at 4:14 PM, Qu Wenruo > > > > > > > > > > > > > > > > quwenruo.btrfs@gmx.com wrote: > > > > > > > > > > > > > > > > > On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > > > > > > > > > Please see below for the dump. > > > > > > > > > > > > > > > > > > > > btrfs-progs v5.12.1 > > > > > > > > > > > > > > > > > > > > leaf 174113599488 items 18 free space 2008 generation 1330906 > > > > > > > > > > > > > > > > > > > > owner 363 > > > > > > > > > > > > > > > > > > > > leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 > > > > > > > > > > > > > > > > > > > > fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 > > > > > > > > > > > > > > > > > > > > chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 > > > > > > > > > > > > > > > > > > > > item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 > > > > > > > > > > > > > > > > > > > > generation 2063 transid 27726 size 40 nbytes 40 > > > > > > > > > > > > > > > > > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > > > > > > > > > > > > > > > > > sequence 1501 flags 0x0(none) > > > > > > > > > > > > > > > > > > > > atime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > > > > > > > > > ctime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > > > > > > > > > mtime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > > > > > > > item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 > > > > > > > > > > > > > > > > > > > > index 12 namelen 1 name: 8 > > > > > > > > > > > > > > > > > > > > item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 > > > > > > > > > > > > > > > > > > > > generation 27726 type 0 (inline) > > > > > > > > > > > > > > > > > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > > > > > > > > > > > > > > > > > item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 > > > > > > > > > > > > > > > > > > > > generation 2542 transid 61261 size 40 nbytes 40 > > > > > > > > > > > > > > > > > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > > > > > > > > > > > > > > > > > sequence 24769 flags 0x0(none) > > > > > > > > > > > > > > > > > > > > atime 1394335806.351857522 (2014-03-09 11:30:06) > > > > > > > > > > > > > > > > > > > > ctime 1394335827.344389955 (2014-03-09 11:30:27) > > > > > > > > > > > > > > > > > > > > mtime 1394335827.344389955 (2014-03-09 11:30:27) > > > > > > > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > > > > > > > item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 > > > > > > > > > > > > > > > > > > > > index 13 namelen 1 name: 7 > > > > > > > > > > > > > > > > > > > > item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 > > > > > > > > > > > > > > > > > > > > generation 61261 type 0 (inline) > > > > > > > > > > > > > > > > > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > > > > > > > > > > > > > > > > > item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 > > > > > > > > > > > > > > > > > > > > generation 5754 transid 5767 size 307 nbytes 307 > > > > > > > > > > > > > > > > > > > > block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 > > > > > > > > > > > > > > > > > > > > sequence 7 flags 0x0(none) > > > > > > > > > > > > > > > > > > > > atime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > > > > > > > > > ctime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > > > > > > > > > mtime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > > > > > > > item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 > > > > > > > > > > > > > > > > > > > > index 6 namelen 17 name: dhcpcd-eth0.lease > > > > > > > > > > > > > > > > > > > > item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 > > > > > > > > > > > > > > > > > > > > generation 5767 type 0 (inline) > > > > > > > > > > > > > > > > > > > > inline extent data size 307 ram_bytes 307 compression 0 (none) > > > > > > > > > > > > > > > > > > > > item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 > > > > > > > > > > > > > > > > > > > > generation 5904 transid 1330906 size 180 nbytes 0 > > > > > > > > > > > > > > > > > > > > block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 > > > > > > > > > > > > > > > > > > > > sequence 177 flags 0x0(none) > > > > > > > > > > > > > > > > > > > > atime 1483277713.141980592 (2017-01-01 21:35:13) > > > > > > > > > > > > > > > > > > > > ctime 1563162901.234656246 (2019-07-15 11:55:01) > > > > > > > > > > > > > > > > > > > > mtime 1406534032.158605559 (2014-07-28 15:53:52) > > > > > > > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > > > > > This inode is indeed a directory. > > > > > > > > > > > > > > > > > > But it has two hard links, which is definitely something > > > > > > > > > > > > > > > > > > unexpected. > > > > > > > > > > > > > > > > > > Under Linux we shouldn't have any hardlink for directory, as it > > > > > > > > > > > > > > > > > > would > > > > > > > > > > > > > > > > > > easily lead to loops. > > > > > > > > > > > > > > > > > > > item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 > > > > > > > > > > > > > > > > > > > > index 28 namelen 9 name: backlight > > > > > > > > > > > > > > > > > > Its parent inode is 260 in the same root, with the name backlight. > > > > > > > > > > > > > > > > > > > item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 > > > > > > > > > > > > > > > > > > > > index 3 namelen 9 name: backlight > > > > > > > > > > > > > > > > > > Another hardlink in inode 286, which is definitely a regular thing. > > > > > > > > > > > > > > > > > > Btrfs-progs lacks the ability to detect such problem, we need to > > > > > > > > > > > > > > > > > > enhance > > > > > > > > > > > > > > > > > > it first. > > > > > > > > > > > > > > > > > > But do you have any idea how this directory get created? > > > > > > > > > > > > > > > > > > It looks like the content of sysfs. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > > > > > item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 > > > > > > > > > > > > > > > > > > > > location key (120417 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > transid 117279 data_len 0 name_len 42 > > > > > > > > > > > > > > > > > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > > > > > > > > > > > > > > > > > item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 > > > > > > > > > > > > > > > > > > > > location key (7487 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > transid 5992 data_len 0 name_len 11 > > > > > > > > > > > > > > > > > > > > name: acpi_video0 > > > > > > > > > > > > > > > > > > > > item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 > > > > > > > > > > > > > > > > > > > > location key (55325 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > transid 63351 data_len 0 name_len 37 > > > > > > > > > > > > > > > > > > > > name: platform-VPC2004:00:backlight:ideapad > > > > > > > > > > > > > > > > > > > > item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 > > > > > > > > > > > > > > > > > > > > location key (7487 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > transid 5992 data_len 0 name_len 11 > > > > > > > > > > > > > > > > > > > > name: acpi_video0 > > > > > > > > > > > > > > > > > > > > item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 > > > > > > > > > > > > > > > > > > > > location key (55325 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > transid 63351 data_len 0 name_len 37 > > > > > > > > > > > > > > > > > > > > name: platform-VPC2004:00:backlight:ideapad > > > > > > > > > > > > > > > > > > > > item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 > > > > > > > > > > > > > > > > > > > > location key (120417 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > transid 117279 data_len 0 name_len 42 > > > > > > > > > > > > > > > > > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > > > > > > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > > > > > > > > > On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo > > > > > > > > > > > > > > > > > > > > quwenruo.btrfs@gmx.com wrote: > > > > > > > > > > > > > > > > > > > > > On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > > > > > > > I see this message on dmesg: > > > > > > > > > > > > > > > > > > > > > > > > [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: > > > > > > > > > > > > > > > > > > > > > > > > root=363 block=174113599488 slot=9 ino=7415, invalid nlink: > > > > > > > > > > > > > > > > > > > > > > > > has 2 expect no more than 1 for dir > > > > > > > > > > > > > > > > > > > > > > > > [ 2452.256776] BTRFS error (device dm-0): block=174113599488 > > > > > > > > > > > > > > > > > > > > > > > > read time tree block corruption detected > > > > > > > > > > > > > > > > > > > > > > Please provide the following dump: > > > > > > > > > > > > > > > > > > > > > > btrfs ins dump-tree -b 174113599488 /dev/dm-0 > > > > > > > > > > > ============================================= > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > > > > > > > > > When I run btrfs scrub and btrfs check, no error was detected. > > > > > > > > > > > > > > > > > > > > > > > > I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 > > > > > > > > > > > > > > > > > > > > > > > > How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <162659797656.6.14385932343235367381.10220447@mb.ardentcoding.com>]
* Re: Read time tree block corruption detected [not found] ` <162659797656.6.14385932343235367381.10220447@mb.ardentcoding.com> @ 2021-07-18 9:32 ` Qu Wenruo [not found] ` <162660074747.7.3300740266059717894.10221270@simplelogin.co> 1 sibling, 0 replies; 75+ messages in thread From: Qu Wenruo @ 2021-07-18 9:32 UTC (permalink / raw) To: pepperpoint; +Cc: Qu Wenruo, linux-btrfs On 2021/7/18 下午4:46, pepperpoint@mb.ardentcoding.com wrote: > Hi Qu, > > When I find which directory some of the filenames are located, they are under /var/lib. > I had subvolume for /var which was created probably around 2018. Then it's possible by somehow we allowed that hardlink to directory. Not sure if it's a bug in VFS layer or in btrfs itself. But around 2019 (aka, v5.2 kernel), that check for refs of directory is introduced and at the same time, write-time tree checker is introduced. This means if the bug happens after v5.2 kernel, it will be rejected before submitting to disk. So the problem definitely happens before the install of v5.2 kernel. > > I don't remember how I created this but I probably use rsync to copy the files from existing /var > or created a snapshot of root and delete other files that is not under /var. But that's still pretty weird. > > Around June, I tried to move the filesystem to another partition through btrfs device add and btrfs device remove > but failed due to that error and was advised to use btrfs replace instead. > > Then at the beginning of this month, I reorganized it merging most of the /var content back to root > and created subvolume for /var/lib/mysql and /var/lib/mongodb. I encountered an error when > I copy some of the files through cp --reflink but I failed for /var/lib/mysql so I created a snapshot > from /var and remove the extra files. > > This is also the time I saw the errors in the log. Before that, the errors was not in the log. At least, we should prevent such problem from reaching disk. If you reverted to older LTS kernel, using Arch Linux Archive, it would be possible to continue deleting the subvolume and solve the problem. After the root 363 get fully deleted, you can verify that tree block get deleted by the following command: # btrfs ins dump-tree -t extent <device> | grep 174113599488 -A 3 Which should show no output. Thanks, Qu > > Regards, > Lester > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > On Sunday, July 18th, 2021 at 3:27 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >> Hi, >> >> BTW, it's really important for us to know how the directory is hardlinked. >> >> Thus I salvaged the filenames found in the half-dropped root 363. >> >> Since it may contain confidential info, I send the filename list to you >> >> off-list. >> >> If you can remind what the root 363 is used for, and any possible >> >> operations which may be involved in that subvolume, it's better to reply >> >> it to the mail list so that we can get some clue on the root cause. >> >> Thanks, >> >> Qu >> >> On 2021/7/18 下午3:15, Qu Wenruo wrote: >> >>> On 2021/7/18 下午1:26, pepperpoint@mb.ardentcoding.com wrote: >>> >>>> Hi Qu, >>>> >>>> May I know if there are any leads on this? What should I do for now? >>> >>> Sorry for the late reply. >>> >>> With the image dump, it's much easier to find what's going wrong. >>> >>> - About root 363 >>> >>> It's an orphan root, thus user can't access it directly. >>> >>> Furthermore, it's being dropped, thus "btrfs ins dump-tree -t 363" >>> >>> reports transid error, as part of the tree has already been dropped, >>> >>> and this is expected. >>> >>> So far your fs is still OK, except that reported error. >>> >>> - About the offending tree block >>> >>> The offending tree block only belongs to the delete subvolume 363, >>> >>> thus it should be delete soon. >>> >>> But unfortunately due to the corrupted content, it's unable to be >>> >>> deleted. >>> >>> For now, if you can re-compile btrfs module, we can workaround the >>> >>> problem by temporarily disable read-time tree-checker so that the >>> >>> deletion can continue, and after the root 363 get fully deleted, the >>> >>> problem should be gone. >>> >>> Or you can use older kernel, any kernel <= v5.1 should not have the >>> >>> enhanced check, thus can continue with the subvolume deletion. >>> >>> If you want to go through the re-compile path, the needed diff is >>> >>> attached >>> >>> Thanks, >>> >>> Qu >>> >>>> Regards, >>>> >>>> Lester >>>> >>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>> >>>> On Saturday, July 17th, 2021 at 8:51 PM, >>>> >>>> pepperpoint@mb.ardentcoding.com wrote: >>>> >>>>> Hi Qu, >>>>> >>>>> I run btrfs ins dump-tree -t 363 unmounted but the same error >>>>> >>>>> appears. Rerunning btrfs check does not show any error. >>>>> >>>>> Regards, >>>>> >>>>> Lester >>>>> >>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>> >>>>> On Saturday, July 17th, 2021 at 6:48 PM, Qu Wenruo - >>>>> >>>>> quwenruo.btrfs@gmx.com wrote: >>>>> >>>>>> On 2021/7/17 下午6:34, pepperpoint@mb.ardentcoding.com wrote: >>>>>> >>>>>>> Hi Qu, >>>>>>> >>>>>>> Unfortunately I cannot find subvolume 363 >>>>>>> >>>>>>> btrfs subvolume list /run/media/root >>>>>>> ==================================== >>>>>>> >>>>>>> ID 361 gen 1814826 top level 584 path @/live/snapshot >>>>>>> >>>>>>> ID 364 gen 1814414 top level 5 path @vtmp/live/snapshot >>>>>>> >>>>>>> ID 369 gen 1814414 top level 5 path @vlmachines/live/snapshot >>>>>>> >>>>>>> ID 493 gen 1814414 top level 5 path @vlportables/live/snapshot >>>>>>> >>>>>>> ID 579 gen 1814828 top level 5 path @vlog/live/snapshot >>>>>>> >>>>>>> ID 580 gen 1814414 top level 5 path @vcache/live/snapshot >>>>>>> >>>>>>> ID 581 gen 1814414 top level 5 path @vlmongodb/live/snapshot >>>>>>> >>>>>>> ID 582 gen 1814414 top level 5 path @vlmysql/live/snapshot >>>>>>> >>>>>>> ID 583 gen 1814414 top level 5 path @vspool/live/snapshot >>>>>>> >>>>>>> ID 584 gen 1814414 top level 5 path @ >>>>>>> >>>>>>> ID 598 gen 1813420 top level 584 path @/4/snapshot >>>>>> >>>>>> Maybe 363 is some subvolume get deleted and later snapshot of it still >>>>>> >>>>>> exists. >>>>>> >>>>>> This will be harder to debug. >>>>>> >>>>>> Can you take a btrfs-image dump of your filesystem? (needs to be taken >>>>>> >>>>>> with the fs unmounted). >>>>>> >>>>>> The dumped image will contain your metadata, including file names and >>>>>> >>>>>> directory structures, but no data inside those files. >>>>>> >>>>>> Although btrfs-image has "-s" option to mask the filenames, but >>>>>> >>>>>> considering the filename in this case is useful to locate the inode, I >>>>>> >>>>>> guess it's better to take the image without any "-s" option. >>>>>> >>>>>>> btrfs ins dump-tree -t 363 /dev/dm-0 | grep -A 5 "(286 " >>>>>>> ======================================================== >>>>>>> >>>>>>> parent transid verify failed on 174170742784 wanted 1789655 found >>>>>>> >>>>>>> 1812621 >>>>>>> >>>>>>> parent transid verify failed on 174170742784 wanted 1789655 found >>>>>>> >>>>>>> 1812621 >>>>>>> >>>>>>> parent transid verify failed on 174170742784 wanted 1789655 found >>>>>>> >>>>>>> 1812621 >>>>>>> >>>>>>> Ignoring transid failure >>>>>>> >>>>>>> ERROR: child eb corrupted: parent bytenr=174170738688 item=0 parent >>>>>>> >>>>>>> level=2 child bytenr=174170742784 child level=0 >>>>>> >>>>>> This transid mismatch may be a problem when running dump-tree on >>>>>> >>>>>> mounted >>>>>> >>>>>> fs, since you mentioned btrfs check reported no error, there shouldn't >>>>>> >>>>>> be a transid mismatch error. >>>>>> >>>>>> Anyway, if you can upload the btrfs-image dump, it would be much easier >>>>>> >>>>>> for us to debug and find out what's really going. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Qu >>>>>> >>>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>>>> >>>>>>> On Saturday, July 17th, 2021 at 6:12 PM, Qu Wenruo wqu@suse.com wrote: >>>>>>> >>>>>>>> On 2021/7/17 下午4:57, pepperpoint@mb.ardentcoding.com wrote: >>>>>>>> >>>>>>>>> Hi Qu, >>>>>>>>> >>>>>>>>> I don't know how the directory was created but last month, I used >>>>>>>>> >>>>>>>>> btrfs device add and btrfs device remove to move the filesystem >>>>>>>>> >>>>>>>>> from one partition to another. It failed because of the same >>>>>>>>> >>>>>>>>> error and was advised to use btrfs replace instead. I don't know >>>>>>>>> >>>>>>>>> if the error also happened before I move the file system as I >>>>>>>>> >>>>>>>>> don't have any previous logs. >>>>>>>> >>>>>>>> It definitely happens before you moving the fs. >>>>>>>> >>>>>>>> As regular dev replacing/add/move only relocates the metadata, but >>>>>>>> >>>>>>>> not >>>>>>>> >>>>>>>> touching the fs trees. >>>>>>>> >>>>>>>>> Here is the result when I search for the inodes you mentioned if >>>>>>>>> >>>>>>>>> it helps: >>>>>>>>> >>>>>>>>> find /run/media/root -inum 260 -exec ls -ldi {} \; >>>>>>>>> ================================================== >>>>>>>>> >>>>>>>>> 260 -rw-r--r-- 1 root root 36864 Jun 25 06:22 >>>>>>>>> >>>>>>>>> /run/media/root/@vcache/live/snapshot/app-info/cache/en_US.cache >>>>>>>>> >>>>>>>>> 260 drwx------ 1 mongodb mongodb 136 Sep 12 2020 >>>>>>>>> >>>>>>>>> /run/media/root/@vlmongodb/live/snapshot/diagnostic.data >>>>>>>>> >>>>>>>>> 260 -rw-rw---- 1 mysql mysql 50331648 Sep 13 2015 >>>>>>>>> >>>>>>>>> /run/media/root/@vlmysql/live/snapshot/ib_logfile0 >>>>>>>>> >>>>>>>>> 260 -rw-r----- 1 root lp 8641 Mar 5 2014 >>>>>>>>> >>>>>>>>> /run/media/root/@vspool/live/snapshot/cups/d00001-001 >>>>>>>>> >>>>>>>>> 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 >>>>>>>>> >>>>>>>>> /run/media/root/@/live/snapshot/sys >>>>>>>>> >>>>>>>>> 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 >>>>>>>>> >>>>>>>>> /run/media/root/@/4/snapshot/sys >>>>>>>> >>>>>>>> Since btrfs can have the same inode number inside different >>>>>>>> >>>>>>>> subvolumes, >>>>>>>> >>>>>>>> you may want to limit the search inside subvolume 363. >>>>>>>> >>>>>>>> "-mount" option of find can do that, you only need to locate >>>>>>>> >>>>>>>> subvolume >>>>>>>> >>>>>>>> 363 by "btrfs subv list". >>>>>>>> >>>>>>>> But from these output I guess above two "sys" directory are more >>>>>>>> >>>>>>>> possible. >>>>>>>> >>>>>>>> Is there any directory named "blaklight" inside those directory? >>>>>>>> >>>>>>>>> find /run/media/root -inum 286 -exec ls -ldi {} \; >>>>>>>>> ================================================== >>>>>>>>> >>>>>>>>> 286 -rw-r--r-- 1 root root 96 Aug 16 2015 >>>>>>>>> >>>>>>>>> /run/media/root/@vcache/live/snapshot/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-4 >>>>>>>>> >>>>>>>>> 286 -rw-rw---- 1 mysql mysql 4096 Sep 15 2013 >>>>>>>>> >>>>>>>>> /run/media/root/@vlmysql/live/snapshot/mysql/columns_priv.MYI >>>>>>>>> >>>>>>>>> 286 -rw-r-----+ 1 root systemd-journal 16777216 Jul 4 01:14 >>>>>>>>> >>>>>>>>> /run/media/root/@vlog/live/snapshot/journal/5098dd7845ae46d3ba1826c68a809a7c/user-1000@fbd9f65d0ea349f6b996716280e6c4dd-00000000002314c5-0005c5cb84a3a438.journal >>>>>>>> >>>>>>>> This is interesting, it means the inode 286 is not accessible. >>>>>>>> >>>>>>>> It can be some orphan inode, but would you dump subvolume 363 then >>>>>>>> >>>>>>>> try >>>>>>>> >>>>>>>> to locate the inode 286? >>>>>>>> >>>>>>>> One example command would be: >>>>>>>> >>>>>>>> btrfs ins dump-tree -t 363 <dev> | grep -A 5 "(286 " >>>>>>>> ==================================================== >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Qu >>>>>>>> >>>>>>>>> Directories with pattern /root/@<dir>/live/snapshot/ are >>>>>>>>> >>>>>>>>> subvolumes and directories with pattern >>>>>>>>> >>>>>>>>> /root/@<dir>/<num>/snapshot/ are snapshots of live. >>>>>>>>> >>>>>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>>>>>> >>>>>>>>> On Saturday, July 17th, 2021 at 4:14 PM, Qu Wenruo >>>>>>>>> >>>>>>>>> quwenruo.btrfs@gmx.com wrote: >>>>>>>>> >>>>>>>>>> On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: >>>>>>>>>> >>>>>>>>>>> Hi Qu, >>>>>>>>>>> >>>>>>>>>>> Please see below for the dump. >>>>>>>>>>> >>>>>>>>>>> btrfs-progs v5.12.1 >>>>>>>>>>> >>>>>>>>>>> leaf 174113599488 items 18 free space 2008 generation 1330906 >>>>>>>>>>> >>>>>>>>>>> owner 363 >>>>>>>>>>> >>>>>>>>>>> leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 >>>>>>>>>>> >>>>>>>>>>> fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 >>>>>>>>>>> >>>>>>>>>>> chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 >>>>>>>>>>> >>>>>>>>>>> item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 >>>>>>>>>>> >>>>>>>>>>> generation 2063 transid 27726 size 40 nbytes 40 >>>>>>>>>>> >>>>>>>>>>> block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 >>>>>>>>>>> >>>>>>>>>>> sequence 1501 flags 0x0(none) >>>>>>>>>>> >>>>>>>>>>> atime 1386484844.468769570 (2013-12-08 14:40:44) >>>>>>>>>>> >>>>>>>>>>> ctime 1386484844.468769570 (2013-12-08 14:40:44) >>>>>>>>>>> >>>>>>>>>>> mtime 1386484844.468769570 (2013-12-08 14:40:44) >>>>>>>>>>> >>>>>>>>>>> otime 0.0 (1970-01-01 08:00:00) >>>>>>>>>>> >>>>>>>>>>> item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 >>>>>>>>>>> >>>>>>>>>>> index 12 namelen 1 name: 8 >>>>>>>>>>> >>>>>>>>>>> item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 >>>>>>>>>>> >>>>>>>>>>> generation 27726 type 0 (inline) >>>>>>>>>>> >>>>>>>>>>> inline extent data size 40 ram_bytes 40 compression 0 (none) >>>>>>>>>>> >>>>>>>>>>> item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 >>>>>>>>>>> >>>>>>>>>>> generation 2542 transid 61261 size 40 nbytes 40 >>>>>>>>>>> >>>>>>>>>>> block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 >>>>>>>>>>> >>>>>>>>>>> sequence 24769 flags 0x0(none) >>>>>>>>>>> >>>>>>>>>>> atime 1394335806.351857522 (2014-03-09 11:30:06) >>>>>>>>>>> >>>>>>>>>>> ctime 1394335827.344389955 (2014-03-09 11:30:27) >>>>>>>>>>> >>>>>>>>>>> mtime 1394335827.344389955 (2014-03-09 11:30:27) >>>>>>>>>>> >>>>>>>>>>> otime 0.0 (1970-01-01 08:00:00) >>>>>>>>>>> >>>>>>>>>>> item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 >>>>>>>>>>> >>>>>>>>>>> index 13 namelen 1 name: 7 >>>>>>>>>>> >>>>>>>>>>> item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 >>>>>>>>>>> >>>>>>>>>>> generation 61261 type 0 (inline) >>>>>>>>>>> >>>>>>>>>>> inline extent data size 40 ram_bytes 40 compression 0 (none) >>>>>>>>>>> >>>>>>>>>>> item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 >>>>>>>>>>> >>>>>>>>>>> generation 5754 transid 5767 size 307 nbytes 307 >>>>>>>>>>> >>>>>>>>>>> block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 >>>>>>>>>>> >>>>>>>>>>> sequence 7 flags 0x0(none) >>>>>>>>>>> >>>>>>>>>>> atime 1379834835.428558020 (2013-09-22 15:27:15) >>>>>>>>>>> >>>>>>>>>>> ctime 1379834835.428558020 (2013-09-22 15:27:15) >>>>>>>>>>> >>>>>>>>>>> mtime 1379834835.428558020 (2013-09-22 15:27:15) >>>>>>>>>>> >>>>>>>>>>> otime 0.0 (1970-01-01 08:00:00) >>>>>>>>>>> >>>>>>>>>>> item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 >>>>>>>>>>> >>>>>>>>>>> index 6 namelen 17 name: dhcpcd-eth0.lease >>>>>>>>>>> >>>>>>>>>>> item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 >>>>>>>>>>> >>>>>>>>>>> generation 5767 type 0 (inline) >>>>>>>>>>> >>>>>>>>>>> inline extent data size 307 ram_bytes 307 compression 0 (none) >>>>>>>>>>> >>>>>>>>>>> item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 >>>>>>>>>>> >>>>>>>>>>> generation 5904 transid 1330906 size 180 nbytes 0 >>>>>>>>>>> >>>>>>>>>>> block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 >>>>>>>>>>> >>>>>>>>>>> sequence 177 flags 0x0(none) >>>>>>>>>>> >>>>>>>>>>> atime 1483277713.141980592 (2017-01-01 21:35:13) >>>>>>>>>>> >>>>>>>>>>> ctime 1563162901.234656246 (2019-07-15 11:55:01) >>>>>>>>>>> >>>>>>>>>>> mtime 1406534032.158605559 (2014-07-28 15:53:52) >>>>>>>>>>> >>>>>>>>>>> otime 0.0 (1970-01-01 08:00:00) >>>>>>>>>> >>>>>>>>>> This inode is indeed a directory. >>>>>>>>>> >>>>>>>>>> But it has two hard links, which is definitely something >>>>>>>>>> >>>>>>>>>> unexpected. >>>>>>>>>> >>>>>>>>>> Under Linux we shouldn't have any hardlink for directory, as it >>>>>>>>>> >>>>>>>>>> would >>>>>>>>>> >>>>>>>>>> easily lead to loops. >>>>>>>>>> >>>>>>>>>>> item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 >>>>>>>>>>> >>>>>>>>>>> index 28 namelen 9 name: backlight >>>>>>>>>> >>>>>>>>>> Its parent inode is 260 in the same root, with the name backlight. >>>>>>>>>> >>>>>>>>>>> item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 >>>>>>>>>>> >>>>>>>>>>> index 3 namelen 9 name: backlight >>>>>>>>>> >>>>>>>>>> Another hardlink in inode 286, which is definitely a regular thing. >>>>>>>>>> >>>>>>>>>> Btrfs-progs lacks the ability to detect such problem, we need to >>>>>>>>>> >>>>>>>>>> enhance >>>>>>>>>> >>>>>>>>>> it first. >>>>>>>>>> >>>>>>>>>> But do you have any idea how this directory get created? >>>>>>>>>> >>>>>>>>>> It looks like the content of sysfs. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Qu >>>>>>>>>> >>>>>>>>>>> item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 >>>>>>>>>>> >>>>>>>>>>> location key (120417 INODE_ITEM 0) type FILE >>>>>>>>>>> >>>>>>>>>>> transid 117279 data_len 0 name_len 42 >>>>>>>>>>> >>>>>>>>>>> name: pci-0000:00:02.0:backlight:intel_backlight >>>>>>>>>>> >>>>>>>>>>> item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 >>>>>>>>>>> >>>>>>>>>>> location key (7487 INODE_ITEM 0) type FILE >>>>>>>>>>> >>>>>>>>>>> transid 5992 data_len 0 name_len 11 >>>>>>>>>>> >>>>>>>>>>> name: acpi_video0 >>>>>>>>>>> >>>>>>>>>>> item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 >>>>>>>>>>> >>>>>>>>>>> location key (55325 INODE_ITEM 0) type FILE >>>>>>>>>>> >>>>>>>>>>> transid 63351 data_len 0 name_len 37 >>>>>>>>>>> >>>>>>>>>>> name: platform-VPC2004:00:backlight:ideapad >>>>>>>>>>> >>>>>>>>>>> item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 >>>>>>>>>>> >>>>>>>>>>> location key (7487 INODE_ITEM 0) type FILE >>>>>>>>>>> >>>>>>>>>>> transid 5992 data_len 0 name_len 11 >>>>>>>>>>> >>>>>>>>>>> name: acpi_video0 >>>>>>>>>>> >>>>>>>>>>> item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 >>>>>>>>>>> >>>>>>>>>>> location key (55325 INODE_ITEM 0) type FILE >>>>>>>>>>> >>>>>>>>>>> transid 63351 data_len 0 name_len 37 >>>>>>>>>>> >>>>>>>>>>> name: platform-VPC2004:00:backlight:ideapad >>>>>>>>>>> >>>>>>>>>>> item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 >>>>>>>>>>> >>>>>>>>>>> location key (120417 INODE_ITEM 0) type FILE >>>>>>>>>>> >>>>>>>>>>> transid 117279 data_len 0 name_len 42 >>>>>>>>>>> >>>>>>>>>>> name: pci-0000:00:02.0:backlight:intel_backlight >>>>>>>>>>> >>>>>>>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>>>>>>>> >>>>>>>>>>> On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo >>>>>>>>>>> >>>>>>>>>>> quwenruo.btrfs@gmx.com wrote: >>>>>>>>>>> >>>>>>>>>>>> On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hello, >>>>>>>>>>>>> >>>>>>>>>>>>> I see this message on dmesg: >>>>>>>>>>>>> >>>>>>>>>>>>> [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: >>>>>>>>>>>>> >>>>>>>>>>>>> root=363 block=174113599488 slot=9 ino=7415, invalid nlink: >>>>>>>>>>>>> >>>>>>>>>>>>> has 2 expect no more than 1 for dir >>>>>>>>>>>>> >>>>>>>>>>>>> [ 2452.256776] BTRFS error (device dm-0): block=174113599488 >>>>>>>>>>>>> >>>>>>>>>>>>> read time tree block corruption detected >>>>>>>>>>>> >>>>>>>>>>>> Please provide the following dump: >>>>>>>>>>>> >>>>>>>>>>>> btrfs ins dump-tree -b 174113599488 /dev/dm-0 >>>>>>>>>>>> ============================================= >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Qu >>>>>>>>>>>> >>>>>>>>>>>>> When I run btrfs scrub and btrfs check, no error was detected. >>>>>>>>>>>>> >>>>>>>>>>>>> I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 >>>>>>>>>>>>> >>>>>>>>>>>>> How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <162660074747.7.3300740266059717894.10221270@simplelogin.co>]
* Re: Read time tree block corruption detected [not found] ` <162660074747.7.3300740266059717894.10221270@simplelogin.co> @ 2021-07-18 10:34 ` pepperpoint [not found] ` <162660447733.8.9782212603273165785.10222524@mb.ardentcoding.com> 1 sibling, 0 replies; 75+ messages in thread From: pepperpoint @ 2021-07-18 10:34 UTC (permalink / raw) To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs Hi Qu, I boot an ISO with Linux 5.1.15, mount the filesystem, wait for a while and restart. This command did not output anything when I boot the system and run it: btrfs ins dump-tree -t extent /dev/dm-0 | grep 174113599488 -A 3 Checking the logs, I do not see the error anymore from the time I boot the system. I also ran btrfs scrub just in case and detected no errors. I think the filesystem is now fixed? Regards, Lester ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Sunday, July 18th, 2021 at 5:32 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > On 2021/7/18 下午4:46, pepperpoint@mb.ardentcoding.com wrote: > > > Hi Qu, > > > > When I find which directory some of the filenames are located, they are under /var/lib. > > > > I had subvolume for /var which was created probably around 2018. > > Then it's possible by somehow we allowed that hardlink to directory. > > Not sure if it's a bug in VFS layer or in btrfs itself. > > But around 2019 (aka, v5.2 kernel), that check for refs of directory is > > introduced and at the same time, write-time tree checker is introduced. > > This means if the bug happens after v5.2 kernel, it will be rejected > > before submitting to disk. > > So the problem definitely happens before the install of v5.2 kernel. > > > I don't remember how I created this but I probably use rsync to copy the files from existing /var > > > > or created a snapshot of root and delete other files that is not under /var. > > But that's still pretty weird. > > > Around June, I tried to move the filesystem to another partition through btrfs device add and btrfs device remove > > > > but failed due to that error and was advised to use btrfs replace instead. > > > > Then at the beginning of this month, I reorganized it merging most of the /var content back to root > > > > and created subvolume for /var/lib/mysql and /var/lib/mongodb. I encountered an error when > > > > I copy some of the files through cp --reflink but I failed for /var/lib/mysql so I created a snapshot > > > > from /var and remove the extra files. > > > > This is also the time I saw the errors in the log. Before that, the errors was not in the log. > > At least, we should prevent such problem from reaching disk. > > If you reverted to older LTS kernel, using Arch Linux Archive, it would > > be possible to continue deleting the subvolume and solve the problem. > > After the root 363 get fully deleted, you can verify that tree block get > > deleted by the following command: > > btrfs ins dump-tree -t extent <device> | grep 174113599488 -A 3 > =============================================================== > > Which should show no output. > > Thanks, > > Qu > > > Regards, > > > > Lester > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > On Sunday, July 18th, 2021 at 3:27 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: > > > > > Hi, > > > > > > BTW, it's really important for us to know how the directory is hardlinked. > > > > > > Thus I salvaged the filenames found in the half-dropped root 363. > > > > > > Since it may contain confidential info, I send the filename list to you > > > > > > off-list. > > > > > > If you can remind what the root 363 is used for, and any possible > > > > > > operations which may be involved in that subvolume, it's better to reply > > > > > > it to the mail list so that we can get some clue on the root cause. > > > > > > Thanks, > > > > > > Qu > > > > > > On 2021/7/18 下午3:15, Qu Wenruo wrote: > > > > > > > On 2021/7/18 下午1:26, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > Hi Qu, > > > > > > > > > > May I know if there are any leads on this? What should I do for now? > > > > > > > > Sorry for the late reply. > > > > > > > > With the image dump, it's much easier to find what's going wrong. > > > > > > > > - About root 363 > > > > > > > > It's an orphan root, thus user can't access it directly. > > > > > > > > > > > > Furthermore, it's being dropped, thus "btrfs ins dump-tree -t 363" > > > > > > > > reports transid error, as part of the tree has already been dropped, > > > > > > > > and this is expected. > > > > > > > > So far your fs is still OK, except that reported error. > > > > > > > > - About the offending tree block > > > > > > > > The offending tree block only belongs to the delete subvolume 363, > > > > > > > > thus it should be delete soon. > > > > > > > > > > > > But unfortunately due to the corrupted content, it's unable to be > > > > > > > > deleted. > > > > > > > > For now, if you can re-compile btrfs module, we can workaround the > > > > > > > > problem by temporarily disable read-time tree-checker so that the > > > > > > > > deletion can continue, and after the root 363 get fully deleted, the > > > > > > > > problem should be gone. > > > > > > > > Or you can use older kernel, any kernel <= v5.1 should not have the > > > > > > > > enhanced check, thus can continue with the subvolume deletion. > > > > > > > > If you want to go through the re-compile path, the needed diff is > > > > > > > > attached > > > > > > > > Thanks, > > > > > > > > Qu > > > > > > > > > Regards, > > > > > > > > > > Lester > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > On Saturday, July 17th, 2021 at 8:51 PM, > > > > > > > > > > pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > I run btrfs ins dump-tree -t 363 unmounted but the same error > > > > > > > > > > > > appears. Rerunning btrfs check does not show any error. > > > > > > > > > > > > Regards, > > > > > > > > > > > > Lester > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > On Saturday, July 17th, 2021 at 6:48 PM, Qu Wenruo - > > > > > > > > > > > > quwenruo.btrfs@gmx.com wrote: > > > > > > > > > > > > > On 2021/7/17 下午6:34, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > > > > > Unfortunately I cannot find subvolume 363 > > > > > > > > > > > > > > > > btrfs subvolume list /run/media/root > > > > > > > > ==================================== > > > > > > > > > > > > > > > > ID 361 gen 1814826 top level 584 path @/live/snapshot > > > > > > > > > > > > > > > > ID 364 gen 1814414 top level 5 path @vtmp/live/snapshot > > > > > > > > > > > > > > > > ID 369 gen 1814414 top level 5 path @vlmachines/live/snapshot > > > > > > > > > > > > > > > > ID 493 gen 1814414 top level 5 path @vlportables/live/snapshot > > > > > > > > > > > > > > > > ID 579 gen 1814828 top level 5 path @vlog/live/snapshot > > > > > > > > > > > > > > > > ID 580 gen 1814414 top level 5 path @vcache/live/snapshot > > > > > > > > > > > > > > > > ID 581 gen 1814414 top level 5 path @vlmongodb/live/snapshot > > > > > > > > > > > > > > > > ID 582 gen 1814414 top level 5 path @vlmysql/live/snapshot > > > > > > > > > > > > > > > > ID 583 gen 1814414 top level 5 path @vspool/live/snapshot > > > > > > > > > > > > > > > > ID 584 gen 1814414 top level 5 path @ > > > > > > > > > > > > > > > > ID 598 gen 1813420 top level 584 path @/4/snapshot > > > > > > > > > > > > > > Maybe 363 is some subvolume get deleted and later snapshot of it still > > > > > > > > > > > > > > exists. > > > > > > > > > > > > > > This will be harder to debug. > > > > > > > > > > > > > > Can you take a btrfs-image dump of your filesystem? (needs to be taken > > > > > > > > > > > > > > with the fs unmounted). > > > > > > > > > > > > > > The dumped image will contain your metadata, including file names and > > > > > > > > > > > > > > directory structures, but no data inside those files. > > > > > > > > > > > > > > Although btrfs-image has "-s" option to mask the filenames, but > > > > > > > > > > > > > > considering the filename in this case is useful to locate the inode, I > > > > > > > > > > > > > > guess it's better to take the image without any "-s" option. > > > > > > > > > > > > > > > btrfs ins dump-tree -t 363 /dev/dm-0 | grep -A 5 "(286 " > > > > > > > > ======================================================== > > > > > > > > > > > > > > > > parent transid verify failed on 174170742784 wanted 1789655 found > > > > > > > > > > > > > > > > 1812621 > > > > > > > > > > > > > > > > parent transid verify failed on 174170742784 wanted 1789655 found > > > > > > > > > > > > > > > > 1812621 > > > > > > > > > > > > > > > > parent transid verify failed on 174170742784 wanted 1789655 found > > > > > > > > > > > > > > > > 1812621 > > > > > > > > > > > > > > > > Ignoring transid failure > > > > > > > > > > > > > > > > ERROR: child eb corrupted: parent bytenr=174170738688 item=0 parent > > > > > > > > > > > > > > > > level=2 child bytenr=174170742784 child level=0 > > > > > > > > > > > > > > This transid mismatch may be a problem when running dump-tree on > > > > > > > > > > > > > > mounted > > > > > > > > > > > > > > fs, since you mentioned btrfs check reported no error, there shouldn't > > > > > > > > > > > > > > be a transid mismatch error. > > > > > > > > > > > > > > Anyway, if you can upload the btrfs-image dump, it would be much easier > > > > > > > > > > > > > > for us to debug and find out what's really going. > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > > > > > On Saturday, July 17th, 2021 at 6:12 PM, Qu Wenruo wqu@suse.com wrote: > > > > > > > > > > > > > > > > > On 2021/7/17 下午4:57, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > > > > > > > > > I don't know how the directory was created but last month, I used > > > > > > > > > > > > > > > > > > > > btrfs device add and btrfs device remove to move the filesystem > > > > > > > > > > > > > > > > > > > > from one partition to another. It failed because of the same > > > > > > > > > > > > > > > > > > > > error and was advised to use btrfs replace instead. I don't know > > > > > > > > > > > > > > > > > > > > if the error also happened before I move the file system as I > > > > > > > > > > > > > > > > > > > > don't have any previous logs. > > > > > > > > > > > > > > > > > > It definitely happens before you moving the fs. > > > > > > > > > > > > > > > > > > As regular dev replacing/add/move only relocates the metadata, but > > > > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > > touching the fs trees. > > > > > > > > > > > > > > > > > > > Here is the result when I search for the inodes you mentioned if > > > > > > > > > > > > > > > > > > > > it helps: > > > > > > > > > > > > > > > > > > > > find /run/media/root -inum 260 -exec ls -ldi {} \; > > > > > > > > > > ================================================== > > > > > > > > > > > > > > > > > > > > 260 -rw-r--r-- 1 root root 36864 Jun 25 06:22 > > > > > > > > > > > > > > > > > > > > /run/media/root/@vcache/live/snapshot/app-info/cache/en_US.cache > > > > > > > > > > > > > > > > > > > > 260 drwx------ 1 mongodb mongodb 136 Sep 12 2020 > > > > > > > > > > > > > > > > > > > > /run/media/root/@vlmongodb/live/snapshot/diagnostic.data > > > > > > > > > > > > > > > > > > > > 260 -rw-rw---- 1 mysql mysql 50331648 Sep 13 2015 > > > > > > > > > > > > > > > > > > > > /run/media/root/@vlmysql/live/snapshot/ib_logfile0 > > > > > > > > > > > > > > > > > > > > 260 -rw-r----- 1 root lp 8641 Mar 5 2014 > > > > > > > > > > > > > > > > > > > > /run/media/root/@vspool/live/snapshot/cups/d00001-001 > > > > > > > > > > > > > > > > > > > > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 > > > > > > > > > > > > > > > > > > > > /run/media/root/@/live/snapshot/sys > > > > > > > > > > > > > > > > > > > > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 > > > > > > > > > > > > > > > > > > > > /run/media/root/@/4/snapshot/sys > > > > > > > > > > > > > > > > > > Since btrfs can have the same inode number inside different > > > > > > > > > > > > > > > > > > subvolumes, > > > > > > > > > > > > > > > > > > you may want to limit the search inside subvolume 363. > > > > > > > > > > > > > > > > > > "-mount" option of find can do that, you only need to locate > > > > > > > > > > > > > > > > > > subvolume > > > > > > > > > > > > > > > > > > 363 by "btrfs subv list". > > > > > > > > > > > > > > > > > > But from these output I guess above two "sys" directory are more > > > > > > > > > > > > > > > > > > possible. > > > > > > > > > > > > > > > > > > Is there any directory named "blaklight" inside those directory? > > > > > > > > > > > > > > > > > > > find /run/media/root -inum 286 -exec ls -ldi {} \; > > > > > > > > > > ================================================== > > > > > > > > > > > > > > > > > > > > 286 -rw-r--r-- 1 root root 96 Aug 16 2015 > > > > > > > > > > > > > > > > > > > > /run/media/root/@vcache/live/snapshot/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-4 > > > > > > > > > > > > > > > > > > > > 286 -rw-rw---- 1 mysql mysql 4096 Sep 15 2013 > > > > > > > > > > > > > > > > > > > > /run/media/root/@vlmysql/live/snapshot/mysql/columns_priv.MYI > > > > > > > > > > > > > > > > > > > > 286 -rw-r-----+ 1 root systemd-journal 16777216 Jul 4 01:14 > > > > > > > > > > > > > > > > > > > > /run/media/root/@vlog/live/snapshot/journal/5098dd7845ae46d3ba1826c68a809a7c/user-1000@fbd9f65d0ea349f6b996716280e6c4dd-00000000002314c5-0005c5cb84a3a438.journal > > > > > > > > > > > > > > > > > > This is interesting, it means the inode 286 is not accessible. > > > > > > > > > > > > > > > > > > It can be some orphan inode, but would you dump subvolume 363 then > > > > > > > > > > > > > > > > > > try > > > > > > > > > > > > > > > > > > to locate the inode 286? > > > > > > > > > > > > > > > > > > One example command would be: > > > > > > > > > > > > > > > > > > btrfs ins dump-tree -t 363 <dev> | grep -A 5 "(286 " > > > > > > > > > ==================================================== > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > > > > > Directories with pattern /root/@<dir>/live/snapshot/ are > > > > > > > > > > > > > > > > > > > > subvolumes and directories with pattern > > > > > > > > > > > > > > > > > > > > /root/@<dir>/<num>/snapshot/ are snapshots of live. > > > > > > > > > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > > > > > > > > > On Saturday, July 17th, 2021 at 4:14 PM, Qu Wenruo > > > > > > > > > > > > > > > > > > > > quwenruo.btrfs@gmx.com wrote: > > > > > > > > > > > > > > > > > > > > > On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > > > > > > > > > > > > > Please see below for the dump. > > > > > > > > > > > > > > > > > > > > > > > > btrfs-progs v5.12.1 > > > > > > > > > > > > > > > > > > > > > > > > leaf 174113599488 items 18 free space 2008 generation 1330906 > > > > > > > > > > > > > > > > > > > > > > > > owner 363 > > > > > > > > > > > > > > > > > > > > > > > > leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 > > > > > > > > > > > > > > > > > > > > > > > > fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 > > > > > > > > > > > > > > > > > > > > > > > > chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 > > > > > > > > > > > > > > > > > > > > > > > > item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 > > > > > > > > > > > > > > > > > > > > > > > > generation 2063 transid 27726 size 40 nbytes 40 > > > > > > > > > > > > > > > > > > > > > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > > > > > > > > > > > > > > > > > > > > > sequence 1501 flags 0x0(none) > > > > > > > > > > > > > > > > > > > > > > > > atime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > > > > > > > > > > > > > ctime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > > > > > > > > > > > > > mtime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > > > > > > > > > > > item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 > > > > > > > > > > > > > > > > > > > > > > > > index 12 namelen 1 name: 8 > > > > > > > > > > > > > > > > > > > > > > > > item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 > > > > > > > > > > > > > > > > > > > > > > > > generation 27726 type 0 (inline) > > > > > > > > > > > > > > > > > > > > > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > > > > > > > > > > > > > > > > > > > > > item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 > > > > > > > > > > > > > > > > > > > > > > > > generation 2542 transid 61261 size 40 nbytes 40 > > > > > > > > > > > > > > > > > > > > > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > > > > > > > > > > > > > > > > > > > > > sequence 24769 flags 0x0(none) > > > > > > > > > > > > > > > > > > > > > > > > atime 1394335806.351857522 (2014-03-09 11:30:06) > > > > > > > > > > > > > > > > > > > > > > > > ctime 1394335827.344389955 (2014-03-09 11:30:27) > > > > > > > > > > > > > > > > > > > > > > > > mtime 1394335827.344389955 (2014-03-09 11:30:27) > > > > > > > > > > > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > > > > > > > > > > > item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 > > > > > > > > > > > > > > > > > > > > > > > > index 13 namelen 1 name: 7 > > > > > > > > > > > > > > > > > > > > > > > > item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 > > > > > > > > > > > > > > > > > > > > > > > > generation 61261 type 0 (inline) > > > > > > > > > > > > > > > > > > > > > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > > > > > > > > > > > > > > > > > > > > > item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 > > > > > > > > > > > > > > > > > > > > > > > > generation 5754 transid 5767 size 307 nbytes 307 > > > > > > > > > > > > > > > > > > > > > > > > block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 > > > > > > > > > > > > > > > > > > > > > > > > sequence 7 flags 0x0(none) > > > > > > > > > > > > > > > > > > > > > > > > atime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > > > > > > > > > > > > > ctime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > > > > > > > > > > > > > mtime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > > > > > > > > > > > item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 > > > > > > > > > > > > > > > > > > > > > > > > index 6 namelen 17 name: dhcpcd-eth0.lease > > > > > > > > > > > > > > > > > > > > > > > > item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 > > > > > > > > > > > > > > > > > > > > > > > > generation 5767 type 0 (inline) > > > > > > > > > > > > > > > > > > > > > > > > inline extent data size 307 ram_bytes 307 compression 0 (none) > > > > > > > > > > > > > > > > > > > > > > > > item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 > > > > > > > > > > > > > > > > > > > > > > > > generation 5904 transid 1330906 size 180 nbytes 0 > > > > > > > > > > > > > > > > > > > > > > > > block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 > > > > > > > > > > > > > > > > > > > > > > > > sequence 177 flags 0x0(none) > > > > > > > > > > > > > > > > > > > > > > > > atime 1483277713.141980592 (2017-01-01 21:35:13) > > > > > > > > > > > > > > > > > > > > > > > > ctime 1563162901.234656246 (2019-07-15 11:55:01) > > > > > > > > > > > > > > > > > > > > > > > > mtime 1406534032.158605559 (2014-07-28 15:53:52) > > > > > > > > > > > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > > > > > > > > > This inode is indeed a directory. > > > > > > > > > > > > > > > > > > > > > > But it has two hard links, which is definitely something > > > > > > > > > > > > > > > > > > > > > > unexpected. > > > > > > > > > > > > > > > > > > > > > > Under Linux we shouldn't have any hardlink for directory, as it > > > > > > > > > > > > > > > > > > > > > > would > > > > > > > > > > > > > > > > > > > > > > easily lead to loops. > > > > > > > > > > > > > > > > > > > > > > > item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 > > > > > > > > > > > > > > > > > > > > > > > > index 28 namelen 9 name: backlight > > > > > > > > > > > > > > > > > > > > > > Its parent inode is 260 in the same root, with the name backlight. > > > > > > > > > > > > > > > > > > > > > > > item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 > > > > > > > > > > > > > > > > > > > > > > > > index 3 namelen 9 name: backlight > > > > > > > > > > > > > > > > > > > > > > Another hardlink in inode 286, which is definitely a regular thing. > > > > > > > > > > > > > > > > > > > > > > Btrfs-progs lacks the ability to detect such problem, we need to > > > > > > > > > > > > > > > > > > > > > > enhance > > > > > > > > > > > > > > > > > > > > > > it first. > > > > > > > > > > > > > > > > > > > > > > But do you have any idea how this directory get created? > > > > > > > > > > > > > > > > > > > > > > It looks like the content of sysfs. > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > > > > > > > > > item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 > > > > > > > > > > > > > > > > > > > > > > > > location key (120417 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > > > > > transid 117279 data_len 0 name_len 42 > > > > > > > > > > > > > > > > > > > > > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > > > > > > > > > > > > > > > > > > > > > item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 > > > > > > > > > > > > > > > > > > > > > > > > location key (7487 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > > > > > transid 5992 data_len 0 name_len 11 > > > > > > > > > > > > > > > > > > > > > > > > name: acpi_video0 > > > > > > > > > > > > > > > > > > > > > > > > item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 > > > > > > > > > > > > > > > > > > > > > > > > location key (55325 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > > > > > transid 63351 data_len 0 name_len 37 > > > > > > > > > > > > > > > > > > > > > > > > name: platform-VPC2004:00:backlight:ideapad > > > > > > > > > > > > > > > > > > > > > > > > item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 > > > > > > > > > > > > > > > > > > > > > > > > location key (7487 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > > > > > transid 5992 data_len 0 name_len 11 > > > > > > > > > > > > > > > > > > > > > > > > name: acpi_video0 > > > > > > > > > > > > > > > > > > > > > > > > item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 > > > > > > > > > > > > > > > > > > > > > > > > location key (55325 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > > > > > transid 63351 data_len 0 name_len 37 > > > > > > > > > > > > > > > > > > > > > > > > name: platform-VPC2004:00:backlight:ideapad > > > > > > > > > > > > > > > > > > > > > > > > item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 > > > > > > > > > > > > > > > > > > > > > > > > location key (120417 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > > > > > transid 117279 data_len 0 name_len 42 > > > > > > > > > > > > > > > > > > > > > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > > > > > > > > > > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > > > > > > > > > > > > > On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo > > > > > > > > > > > > > > > > > > > > > > > > quwenruo.btrfs@gmx.com wrote: > > > > > > > > > > > > > > > > > > > > > > > > > On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > > > > > > > > > > > I see this message on dmesg: > > > > > > > > > > > > > > > > > > > > > > > > > > > > [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: > > > > > > > > > > > > > > > > > > > > > > > > > > > > root=363 block=174113599488 slot=9 ino=7415, invalid nlink: > > > > > > > > > > > > > > > > > > > > > > > > > > > > has 2 expect no more than 1 for dir > > > > > > > > > > > > > > > > > > > > > > > > > > > > [ 2452.256776] BTRFS error (device dm-0): block=174113599488 > > > > > > > > > > > > > > > > > > > > > > > > > > > > read time tree block corruption detected > > > > > > > > > > > > > > > > > > > > > > > > > > Please provide the following dump: > > > > > > > > > > > > > > > > > > > > > > > > > > btrfs ins dump-tree -b 174113599488 /dev/dm-0 > > > > > > > > > > > > > ============================================= > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > > > > > > > > > > > > > When I run btrfs scrub and btrfs check, no error was detected. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <162660447733.8.9782212603273165785.10222524@mb.ardentcoding.com>]
* Re: Read time tree block corruption detected [not found] ` <162660447733.8.9782212603273165785.10222524@mb.ardentcoding.com> @ 2021-07-18 11:13 ` Qu Wenruo [not found] ` <162660684635.8.12423097770824212671.10223516@simplelogin.co> 1 sibling, 0 replies; 75+ messages in thread From: Qu Wenruo @ 2021-07-18 11:13 UTC (permalink / raw) To: pepperpoint; +Cc: Qu Wenruo, linux-btrfs On 2021/7/18 下午6:34, pepperpoint@mb.ardentcoding.com wrote: > Hi Qu, > > I boot an ISO with Linux 5.1.15, mount the filesystem, wait for a while and restart. > > This command did not output anything when I boot the system and run it: > btrfs ins dump-tree -t extent /dev/dm-0 | grep 174113599488 -A 3 > > Checking the logs, I do not see the error anymore from the time I boot the system. I also ran btrfs scrub just in case and detected no errors. > > I think the filesystem is now fixed? Yep, as expected. If you want to be extra extra sure, you can try another command (no need to unmount the fs): # btrfs ins dump-tree -t root /dev/dm-0 | grep "(363 ROOT" It should has no output. I'll later submit a patch to enhance btrfs-progs to detect such problem. Thanks, Qu > > Regards, > Lester > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > On Sunday, July 18th, 2021 at 5:32 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >> On 2021/7/18 下午4:46, pepperpoint@mb.ardentcoding.com wrote: >> >>> Hi Qu, >>> >>> When I find which directory some of the filenames are located, they are under /var/lib. >>> >>> I had subvolume for /var which was created probably around 2018. >> >> Then it's possible by somehow we allowed that hardlink to directory. >> >> Not sure if it's a bug in VFS layer or in btrfs itself. >> >> But around 2019 (aka, v5.2 kernel), that check for refs of directory is >> >> introduced and at the same time, write-time tree checker is introduced. >> >> This means if the bug happens after v5.2 kernel, it will be rejected >> >> before submitting to disk. >> >> So the problem definitely happens before the install of v5.2 kernel. >> >>> I don't remember how I created this but I probably use rsync to copy the files from existing /var >>> >>> or created a snapshot of root and delete other files that is not under /var. >> >> But that's still pretty weird. >> >>> Around June, I tried to move the filesystem to another partition through btrfs device add and btrfs device remove >>> >>> but failed due to that error and was advised to use btrfs replace instead. >>> >>> Then at the beginning of this month, I reorganized it merging most of the /var content back to root >>> >>> and created subvolume for /var/lib/mysql and /var/lib/mongodb. I encountered an error when >>> >>> I copy some of the files through cp --reflink but I failed for /var/lib/mysql so I created a snapshot >>> >>> from /var and remove the extra files. >>> >>> This is also the time I saw the errors in the log. Before that, the errors was not in the log. >> >> At least, we should prevent such problem from reaching disk. >> >> If you reverted to older LTS kernel, using Arch Linux Archive, it would >> >> be possible to continue deleting the subvolume and solve the problem. >> >> After the root 363 get fully deleted, you can verify that tree block get >> >> deleted by the following command: >> >> btrfs ins dump-tree -t extent <device> | grep 174113599488 -A 3 >> =============================================================== >> >> Which should show no output. >> >> Thanks, >> >> Qu >> >>> Regards, >>> >>> Lester >>> >>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>> >>> On Sunday, July 18th, 2021 at 3:27 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: >>> >>>> Hi, >>>> >>>> BTW, it's really important for us to know how the directory is hardlinked. >>>> >>>> Thus I salvaged the filenames found in the half-dropped root 363. >>>> >>>> Since it may contain confidential info, I send the filename list to you >>>> >>>> off-list. >>>> >>>> If you can remind what the root 363 is used for, and any possible >>>> >>>> operations which may be involved in that subvolume, it's better to reply >>>> >>>> it to the mail list so that we can get some clue on the root cause. >>>> >>>> Thanks, >>>> >>>> Qu >>>> >>>> On 2021/7/18 下午3:15, Qu Wenruo wrote: >>>> >>>>> On 2021/7/18 下午1:26, pepperpoint@mb.ardentcoding.com wrote: >>>>> >>>>>> Hi Qu, >>>>>> >>>>>> May I know if there are any leads on this? What should I do for now? >>>>> >>>>> Sorry for the late reply. >>>>> >>>>> With the image dump, it's much easier to find what's going wrong. >>>>> >>>>> - About root 363 >>>>> >>>>> It's an orphan root, thus user can't access it directly. >>>>> >>>>> >>>>> Furthermore, it's being dropped, thus "btrfs ins dump-tree -t 363" >>>>> >>>>> reports transid error, as part of the tree has already been dropped, >>>>> >>>>> and this is expected. >>>>> >>>>> So far your fs is still OK, except that reported error. >>>>> >>>>> - About the offending tree block >>>>> >>>>> The offending tree block only belongs to the delete subvolume 363, >>>>> >>>>> thus it should be delete soon. >>>>> >>>>> >>>>> But unfortunately due to the corrupted content, it's unable to be >>>>> >>>>> deleted. >>>>> >>>>> For now, if you can re-compile btrfs module, we can workaround the >>>>> >>>>> problem by temporarily disable read-time tree-checker so that the >>>>> >>>>> deletion can continue, and after the root 363 get fully deleted, the >>>>> >>>>> problem should be gone. >>>>> >>>>> Or you can use older kernel, any kernel <= v5.1 should not have the >>>>> >>>>> enhanced check, thus can continue with the subvolume deletion. >>>>> >>>>> If you want to go through the re-compile path, the needed diff is >>>>> >>>>> attached >>>>> >>>>> Thanks, >>>>> >>>>> Qu >>>>> >>>>>> Regards, >>>>>> >>>>>> Lester >>>>>> >>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>>> >>>>>> On Saturday, July 17th, 2021 at 8:51 PM, >>>>>> >>>>>> pepperpoint@mb.ardentcoding.com wrote: >>>>>> >>>>>>> Hi Qu, >>>>>>> >>>>>>> I run btrfs ins dump-tree -t 363 unmounted but the same error >>>>>>> >>>>>>> appears. Rerunning btrfs check does not show any error. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Lester >>>>>>> >>>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>>>> >>>>>>> On Saturday, July 17th, 2021 at 6:48 PM, Qu Wenruo - >>>>>>> >>>>>>> quwenruo.btrfs@gmx.com wrote: >>>>>>> >>>>>>>> On 2021/7/17 下午6:34, pepperpoint@mb.ardentcoding.com wrote: >>>>>>>> >>>>>>>>> Hi Qu, >>>>>>>>> >>>>>>>>> Unfortunately I cannot find subvolume 363 >>>>>>>>> >>>>>>>>> btrfs subvolume list /run/media/root >>>>>>>>> ==================================== >>>>>>>>> >>>>>>>>> ID 361 gen 1814826 top level 584 path @/live/snapshot >>>>>>>>> >>>>>>>>> ID 364 gen 1814414 top level 5 path @vtmp/live/snapshot >>>>>>>>> >>>>>>>>> ID 369 gen 1814414 top level 5 path @vlmachines/live/snapshot >>>>>>>>> >>>>>>>>> ID 493 gen 1814414 top level 5 path @vlportables/live/snapshot >>>>>>>>> >>>>>>>>> ID 579 gen 1814828 top level 5 path @vlog/live/snapshot >>>>>>>>> >>>>>>>>> ID 580 gen 1814414 top level 5 path @vcache/live/snapshot >>>>>>>>> >>>>>>>>> ID 581 gen 1814414 top level 5 path @vlmongodb/live/snapshot >>>>>>>>> >>>>>>>>> ID 582 gen 1814414 top level 5 path @vlmysql/live/snapshot >>>>>>>>> >>>>>>>>> ID 583 gen 1814414 top level 5 path @vspool/live/snapshot >>>>>>>>> >>>>>>>>> ID 584 gen 1814414 top level 5 path @ >>>>>>>>> >>>>>>>>> ID 598 gen 1813420 top level 584 path @/4/snapshot >>>>>>>> >>>>>>>> Maybe 363 is some subvolume get deleted and later snapshot of it still >>>>>>>> >>>>>>>> exists. >>>>>>>> >>>>>>>> This will be harder to debug. >>>>>>>> >>>>>>>> Can you take a btrfs-image dump of your filesystem? (needs to be taken >>>>>>>> >>>>>>>> with the fs unmounted). >>>>>>>> >>>>>>>> The dumped image will contain your metadata, including file names and >>>>>>>> >>>>>>>> directory structures, but no data inside those files. >>>>>>>> >>>>>>>> Although btrfs-image has "-s" option to mask the filenames, but >>>>>>>> >>>>>>>> considering the filename in this case is useful to locate the inode, I >>>>>>>> >>>>>>>> guess it's better to take the image without any "-s" option. >>>>>>>> >>>>>>>>> btrfs ins dump-tree -t 363 /dev/dm-0 | grep -A 5 "(286 " >>>>>>>>> ======================================================== >>>>>>>>> >>>>>>>>> parent transid verify failed on 174170742784 wanted 1789655 found >>>>>>>>> >>>>>>>>> 1812621 >>>>>>>>> >>>>>>>>> parent transid verify failed on 174170742784 wanted 1789655 found >>>>>>>>> >>>>>>>>> 1812621 >>>>>>>>> >>>>>>>>> parent transid verify failed on 174170742784 wanted 1789655 found >>>>>>>>> >>>>>>>>> 1812621 >>>>>>>>> >>>>>>>>> Ignoring transid failure >>>>>>>>> >>>>>>>>> ERROR: child eb corrupted: parent bytenr=174170738688 item=0 parent >>>>>>>>> >>>>>>>>> level=2 child bytenr=174170742784 child level=0 >>>>>>>> >>>>>>>> This transid mismatch may be a problem when running dump-tree on >>>>>>>> >>>>>>>> mounted >>>>>>>> >>>>>>>> fs, since you mentioned btrfs check reported no error, there shouldn't >>>>>>>> >>>>>>>> be a transid mismatch error. >>>>>>>> >>>>>>>> Anyway, if you can upload the btrfs-image dump, it would be much easier >>>>>>>> >>>>>>>> for us to debug and find out what's really going. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Qu >>>>>>>> >>>>>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>>>>>> >>>>>>>>> On Saturday, July 17th, 2021 at 6:12 PM, Qu Wenruo wqu@suse.com wrote: >>>>>>>>> >>>>>>>>>> On 2021/7/17 下午4:57, pepperpoint@mb.ardentcoding.com wrote: >>>>>>>>>> >>>>>>>>>>> Hi Qu, >>>>>>>>>>> >>>>>>>>>>> I don't know how the directory was created but last month, I used >>>>>>>>>>> >>>>>>>>>>> btrfs device add and btrfs device remove to move the filesystem >>>>>>>>>>> >>>>>>>>>>> from one partition to another. It failed because of the same >>>>>>>>>>> >>>>>>>>>>> error and was advised to use btrfs replace instead. I don't know >>>>>>>>>>> >>>>>>>>>>> if the error also happened before I move the file system as I >>>>>>>>>>> >>>>>>>>>>> don't have any previous logs. >>>>>>>>>> >>>>>>>>>> It definitely happens before you moving the fs. >>>>>>>>>> >>>>>>>>>> As regular dev replacing/add/move only relocates the metadata, but >>>>>>>>>> >>>>>>>>>> not >>>>>>>>>> >>>>>>>>>> touching the fs trees. >>>>>>>>>> >>>>>>>>>>> Here is the result when I search for the inodes you mentioned if >>>>>>>>>>> >>>>>>>>>>> it helps: >>>>>>>>>>> >>>>>>>>>>> find /run/media/root -inum 260 -exec ls -ldi {} \; >>>>>>>>>>> ================================================== >>>>>>>>>>> >>>>>>>>>>> 260 -rw-r--r-- 1 root root 36864 Jun 25 06:22 >>>>>>>>>>> >>>>>>>>>>> /run/media/root/@vcache/live/snapshot/app-info/cache/en_US.cache >>>>>>>>>>> >>>>>>>>>>> 260 drwx------ 1 mongodb mongodb 136 Sep 12 2020 >>>>>>>>>>> >>>>>>>>>>> /run/media/root/@vlmongodb/live/snapshot/diagnostic.data >>>>>>>>>>> >>>>>>>>>>> 260 -rw-rw---- 1 mysql mysql 50331648 Sep 13 2015 >>>>>>>>>>> >>>>>>>>>>> /run/media/root/@vlmysql/live/snapshot/ib_logfile0 >>>>>>>>>>> >>>>>>>>>>> 260 -rw-r----- 1 root lp 8641 Mar 5 2014 >>>>>>>>>>> >>>>>>>>>>> /run/media/root/@vspool/live/snapshot/cups/d00001-001 >>>>>>>>>>> >>>>>>>>>>> 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 >>>>>>>>>>> >>>>>>>>>>> /run/media/root/@/live/snapshot/sys >>>>>>>>>>> >>>>>>>>>>> 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 >>>>>>>>>>> >>>>>>>>>>> /run/media/root/@/4/snapshot/sys >>>>>>>>>> >>>>>>>>>> Since btrfs can have the same inode number inside different >>>>>>>>>> >>>>>>>>>> subvolumes, >>>>>>>>>> >>>>>>>>>> you may want to limit the search inside subvolume 363. >>>>>>>>>> >>>>>>>>>> "-mount" option of find can do that, you only need to locate >>>>>>>>>> >>>>>>>>>> subvolume >>>>>>>>>> >>>>>>>>>> 363 by "btrfs subv list". >>>>>>>>>> >>>>>>>>>> But from these output I guess above two "sys" directory are more >>>>>>>>>> >>>>>>>>>> possible. >>>>>>>>>> >>>>>>>>>> Is there any directory named "blaklight" inside those directory? >>>>>>>>>> >>>>>>>>>>> find /run/media/root -inum 286 -exec ls -ldi {} \; >>>>>>>>>>> ================================================== >>>>>>>>>>> >>>>>>>>>>> 286 -rw-r--r-- 1 root root 96 Aug 16 2015 >>>>>>>>>>> >>>>>>>>>>> /run/media/root/@vcache/live/snapshot/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-4 >>>>>>>>>>> >>>>>>>>>>> 286 -rw-rw---- 1 mysql mysql 4096 Sep 15 2013 >>>>>>>>>>> >>>>>>>>>>> /run/media/root/@vlmysql/live/snapshot/mysql/columns_priv.MYI >>>>>>>>>>> >>>>>>>>>>> 286 -rw-r-----+ 1 root systemd-journal 16777216 Jul 4 01:14 >>>>>>>>>>> >>>>>>>>>>> /run/media/root/@vlog/live/snapshot/journal/5098dd7845ae46d3ba1826c68a809a7c/user-1000@fbd9f65d0ea349f6b996716280e6c4dd-00000000002314c5-0005c5cb84a3a438.journal >>>>>>>>>> >>>>>>>>>> This is interesting, it means the inode 286 is not accessible. >>>>>>>>>> >>>>>>>>>> It can be some orphan inode, but would you dump subvolume 363 then >>>>>>>>>> >>>>>>>>>> try >>>>>>>>>> >>>>>>>>>> to locate the inode 286? >>>>>>>>>> >>>>>>>>>> One example command would be: >>>>>>>>>> >>>>>>>>>> btrfs ins dump-tree -t 363 <dev> | grep -A 5 "(286 " >>>>>>>>>> ==================================================== >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Qu >>>>>>>>>> >>>>>>>>>>> Directories with pattern /root/@<dir>/live/snapshot/ are >>>>>>>>>>> >>>>>>>>>>> subvolumes and directories with pattern >>>>>>>>>>> >>>>>>>>>>> /root/@<dir>/<num>/snapshot/ are snapshots of live. >>>>>>>>>>> >>>>>>>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>>>>>>>> >>>>>>>>>>> On Saturday, July 17th, 2021 at 4:14 PM, Qu Wenruo >>>>>>>>>>> >>>>>>>>>>> quwenruo.btrfs@gmx.com wrote: >>>>>>>>>>> >>>>>>>>>>>> On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Qu, >>>>>>>>>>>>> >>>>>>>>>>>>> Please see below for the dump. >>>>>>>>>>>>> >>>>>>>>>>>>> btrfs-progs v5.12.1 >>>>>>>>>>>>> >>>>>>>>>>>>> leaf 174113599488 items 18 free space 2008 generation 1330906 >>>>>>>>>>>>> >>>>>>>>>>>>> owner 363 >>>>>>>>>>>>> >>>>>>>>>>>>> leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 >>>>>>>>>>>>> >>>>>>>>>>>>> fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 >>>>>>>>>>>>> >>>>>>>>>>>>> chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 >>>>>>>>>>>>> >>>>>>>>>>>>> item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 >>>>>>>>>>>>> >>>>>>>>>>>>> generation 2063 transid 27726 size 40 nbytes 40 >>>>>>>>>>>>> >>>>>>>>>>>>> block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 >>>>>>>>>>>>> >>>>>>>>>>>>> sequence 1501 flags 0x0(none) >>>>>>>>>>>>> >>>>>>>>>>>>> atime 1386484844.468769570 (2013-12-08 14:40:44) >>>>>>>>>>>>> >>>>>>>>>>>>> ctime 1386484844.468769570 (2013-12-08 14:40:44) >>>>>>>>>>>>> >>>>>>>>>>>>> mtime 1386484844.468769570 (2013-12-08 14:40:44) >>>>>>>>>>>>> >>>>>>>>>>>>> otime 0.0 (1970-01-01 08:00:00) >>>>>>>>>>>>> >>>>>>>>>>>>> item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 >>>>>>>>>>>>> >>>>>>>>>>>>> index 12 namelen 1 name: 8 >>>>>>>>>>>>> >>>>>>>>>>>>> item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 >>>>>>>>>>>>> >>>>>>>>>>>>> generation 27726 type 0 (inline) >>>>>>>>>>>>> >>>>>>>>>>>>> inline extent data size 40 ram_bytes 40 compression 0 (none) >>>>>>>>>>>>> >>>>>>>>>>>>> item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 >>>>>>>>>>>>> >>>>>>>>>>>>> generation 2542 transid 61261 size 40 nbytes 40 >>>>>>>>>>>>> >>>>>>>>>>>>> block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 >>>>>>>>>>>>> >>>>>>>>>>>>> sequence 24769 flags 0x0(none) >>>>>>>>>>>>> >>>>>>>>>>>>> atime 1394335806.351857522 (2014-03-09 11:30:06) >>>>>>>>>>>>> >>>>>>>>>>>>> ctime 1394335827.344389955 (2014-03-09 11:30:27) >>>>>>>>>>>>> >>>>>>>>>>>>> mtime 1394335827.344389955 (2014-03-09 11:30:27) >>>>>>>>>>>>> >>>>>>>>>>>>> otime 0.0 (1970-01-01 08:00:00) >>>>>>>>>>>>> >>>>>>>>>>>>> item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 >>>>>>>>>>>>> >>>>>>>>>>>>> index 13 namelen 1 name: 7 >>>>>>>>>>>>> >>>>>>>>>>>>> item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 >>>>>>>>>>>>> >>>>>>>>>>>>> generation 61261 type 0 (inline) >>>>>>>>>>>>> >>>>>>>>>>>>> inline extent data size 40 ram_bytes 40 compression 0 (none) >>>>>>>>>>>>> >>>>>>>>>>>>> item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 >>>>>>>>>>>>> >>>>>>>>>>>>> generation 5754 transid 5767 size 307 nbytes 307 >>>>>>>>>>>>> >>>>>>>>>>>>> block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 >>>>>>>>>>>>> >>>>>>>>>>>>> sequence 7 flags 0x0(none) >>>>>>>>>>>>> >>>>>>>>>>>>> atime 1379834835.428558020 (2013-09-22 15:27:15) >>>>>>>>>>>>> >>>>>>>>>>>>> ctime 1379834835.428558020 (2013-09-22 15:27:15) >>>>>>>>>>>>> >>>>>>>>>>>>> mtime 1379834835.428558020 (2013-09-22 15:27:15) >>>>>>>>>>>>> >>>>>>>>>>>>> otime 0.0 (1970-01-01 08:00:00) >>>>>>>>>>>>> >>>>>>>>>>>>> item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 >>>>>>>>>>>>> >>>>>>>>>>>>> index 6 namelen 17 name: dhcpcd-eth0.lease >>>>>>>>>>>>> >>>>>>>>>>>>> item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 >>>>>>>>>>>>> >>>>>>>>>>>>> generation 5767 type 0 (inline) >>>>>>>>>>>>> >>>>>>>>>>>>> inline extent data size 307 ram_bytes 307 compression 0 (none) >>>>>>>>>>>>> >>>>>>>>>>>>> item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 >>>>>>>>>>>>> >>>>>>>>>>>>> generation 5904 transid 1330906 size 180 nbytes 0 >>>>>>>>>>>>> >>>>>>>>>>>>> block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 >>>>>>>>>>>>> >>>>>>>>>>>>> sequence 177 flags 0x0(none) >>>>>>>>>>>>> >>>>>>>>>>>>> atime 1483277713.141980592 (2017-01-01 21:35:13) >>>>>>>>>>>>> >>>>>>>>>>>>> ctime 1563162901.234656246 (2019-07-15 11:55:01) >>>>>>>>>>>>> >>>>>>>>>>>>> mtime 1406534032.158605559 (2014-07-28 15:53:52) >>>>>>>>>>>>> >>>>>>>>>>>>> otime 0.0 (1970-01-01 08:00:00) >>>>>>>>>>>> >>>>>>>>>>>> This inode is indeed a directory. >>>>>>>>>>>> >>>>>>>>>>>> But it has two hard links, which is definitely something >>>>>>>>>>>> >>>>>>>>>>>> unexpected. >>>>>>>>>>>> >>>>>>>>>>>> Under Linux we shouldn't have any hardlink for directory, as it >>>>>>>>>>>> >>>>>>>>>>>> would >>>>>>>>>>>> >>>>>>>>>>>> easily lead to loops. >>>>>>>>>>>> >>>>>>>>>>>>> item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 >>>>>>>>>>>>> >>>>>>>>>>>>> index 28 namelen 9 name: backlight >>>>>>>>>>>> >>>>>>>>>>>> Its parent inode is 260 in the same root, with the name backlight. >>>>>>>>>>>> >>>>>>>>>>>>> item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 >>>>>>>>>>>>> >>>>>>>>>>>>> index 3 namelen 9 name: backlight >>>>>>>>>>>> >>>>>>>>>>>> Another hardlink in inode 286, which is definitely a regular thing. >>>>>>>>>>>> >>>>>>>>>>>> Btrfs-progs lacks the ability to detect such problem, we need to >>>>>>>>>>>> >>>>>>>>>>>> enhance >>>>>>>>>>>> >>>>>>>>>>>> it first. >>>>>>>>>>>> >>>>>>>>>>>> But do you have any idea how this directory get created? >>>>>>>>>>>> >>>>>>>>>>>> It looks like the content of sysfs. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Qu >>>>>>>>>>>> >>>>>>>>>>>>> item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 >>>>>>>>>>>>> >>>>>>>>>>>>> location key (120417 INODE_ITEM 0) type FILE >>>>>>>>>>>>> >>>>>>>>>>>>> transid 117279 data_len 0 name_len 42 >>>>>>>>>>>>> >>>>>>>>>>>>> name: pci-0000:00:02.0:backlight:intel_backlight >>>>>>>>>>>>> >>>>>>>>>>>>> item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 >>>>>>>>>>>>> >>>>>>>>>>>>> location key (7487 INODE_ITEM 0) type FILE >>>>>>>>>>>>> >>>>>>>>>>>>> transid 5992 data_len 0 name_len 11 >>>>>>>>>>>>> >>>>>>>>>>>>> name: acpi_video0 >>>>>>>>>>>>> >>>>>>>>>>>>> item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 >>>>>>>>>>>>> >>>>>>>>>>>>> location key (55325 INODE_ITEM 0) type FILE >>>>>>>>>>>>> >>>>>>>>>>>>> transid 63351 data_len 0 name_len 37 >>>>>>>>>>>>> >>>>>>>>>>>>> name: platform-VPC2004:00:backlight:ideapad >>>>>>>>>>>>> >>>>>>>>>>>>> item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 >>>>>>>>>>>>> >>>>>>>>>>>>> location key (7487 INODE_ITEM 0) type FILE >>>>>>>>>>>>> >>>>>>>>>>>>> transid 5992 data_len 0 name_len 11 >>>>>>>>>>>>> >>>>>>>>>>>>> name: acpi_video0 >>>>>>>>>>>>> >>>>>>>>>>>>> item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 >>>>>>>>>>>>> >>>>>>>>>>>>> location key (55325 INODE_ITEM 0) type FILE >>>>>>>>>>>>> >>>>>>>>>>>>> transid 63351 data_len 0 name_len 37 >>>>>>>>>>>>> >>>>>>>>>>>>> name: platform-VPC2004:00:backlight:ideapad >>>>>>>>>>>>> >>>>>>>>>>>>> item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 >>>>>>>>>>>>> >>>>>>>>>>>>> location key (120417 INODE_ITEM 0) type FILE >>>>>>>>>>>>> >>>>>>>>>>>>> transid 117279 data_len 0 name_len 42 >>>>>>>>>>>>> >>>>>>>>>>>>> name: pci-0000:00:02.0:backlight:intel_backlight >>>>>>>>>>>>> >>>>>>>>>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>>>>>>>>>> >>>>>>>>>>>>> On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo >>>>>>>>>>>>> >>>>>>>>>>>>> quwenruo.btrfs@gmx.com wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I see this message on dmesg: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> root=363 block=174113599488 slot=9 ino=7415, invalid nlink: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> has 2 expect no more than 1 for dir >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [ 2452.256776] BTRFS error (device dm-0): block=174113599488 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> read time tree block corruption detected >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please provide the following dump: >>>>>>>>>>>>>> >>>>>>>>>>>>>> btrfs ins dump-tree -b 174113599488 /dev/dm-0 >>>>>>>>>>>>>> ============================================= >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Qu >>>>>>>>>>>>>> >>>>>>>>>>>>>>> When I run btrfs scrub and btrfs check, no error was detected. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <162660684635.8.12423097770824212671.10223516@simplelogin.co>]
* Re: Read time tree block corruption detected [not found] ` <162660684635.8.12423097770824212671.10223516@simplelogin.co> @ 2021-07-18 12:16 ` pepperpoint 0 siblings, 0 replies; 75+ messages in thread From: pepperpoint @ 2021-07-18 12:16 UTC (permalink / raw) To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs Hi Qu, I run the command and returns no output. Thank you for helping me solve the problem! Regards, Lester ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Sunday, July 18th, 2021 at 7:14 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > On 2021/7/18 下午6:34, pepperpoint@mb.ardentcoding.com wrote: > > > Hi Qu, > > > > I boot an ISO with Linux 5.1.15, mount the filesystem, wait for a while and restart. > > > > This command did not output anything when I boot the system and run it: > > > > btrfs ins dump-tree -t extent /dev/dm-0 | grep 174113599488 -A 3 > > > > Checking the logs, I do not see the error anymore from the time I boot the system. I also ran btrfs scrub just in case and detected no errors. > > > > I think the filesystem is now fixed? > > Yep, as expected. > > If you want to be extra extra sure, you can try another command (no need > > to unmount the fs): > > btrfs ins dump-tree -t root /dev/dm-0 | grep "(363 ROOT" > ======================================================== > > It should has no output. > > I'll later submit a patch to enhance btrfs-progs to detect such problem. > > Thanks, > > Qu > > > Regards, > > > > Lester > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > On Sunday, July 18th, 2021 at 5:32 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: > > > > > On 2021/7/18 下午4:46, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > Hi Qu, > > > > > > > > When I find which directory some of the filenames are located, they are under /var/lib. > > > > > > > > I had subvolume for /var which was created probably around 2018. > > > > > > Then it's possible by somehow we allowed that hardlink to directory. > > > > > > Not sure if it's a bug in VFS layer or in btrfs itself. > > > > > > But around 2019 (aka, v5.2 kernel), that check for refs of directory is > > > > > > introduced and at the same time, write-time tree checker is introduced. > > > > > > This means if the bug happens after v5.2 kernel, it will be rejected > > > > > > before submitting to disk. > > > > > > So the problem definitely happens before the install of v5.2 kernel. > > > > > > > I don't remember how I created this but I probably use rsync to copy the files from existing /var > > > > > > > > or created a snapshot of root and delete other files that is not under /var. > > > > > > But that's still pretty weird. > > > > > > > Around June, I tried to move the filesystem to another partition through btrfs device add and btrfs device remove > > > > > > > > but failed due to that error and was advised to use btrfs replace instead. > > > > > > > > Then at the beginning of this month, I reorganized it merging most of the /var content back to root > > > > > > > > and created subvolume for /var/lib/mysql and /var/lib/mongodb. I encountered an error when > > > > > > > > I copy some of the files through cp --reflink but I failed for /var/lib/mysql so I created a snapshot > > > > > > > > from /var and remove the extra files. > > > > > > > > This is also the time I saw the errors in the log. Before that, the errors was not in the log. > > > > > > At least, we should prevent such problem from reaching disk. > > > > > > If you reverted to older LTS kernel, using Arch Linux Archive, it would > > > > > > be possible to continue deleting the subvolume and solve the problem. > > > > > > After the root 363 get fully deleted, you can verify that tree block get > > > > > > deleted by the following command: > > > > > > btrfs ins dump-tree -t extent <device> | grep 174113599488 -A 3 > > > =============================================================== > > > > > > Which should show no output. > > > > > > Thanks, > > > > > > Qu > > > > > > > Regards, > > > > > > > > Lester > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > On Sunday, July 18th, 2021 at 3:27 PM, Qu Wenruo quwenruo.btrfs@gmx.com wrote: > > > > > > > > > Hi, > > > > > > > > > > BTW, it's really important for us to know how the directory is hardlinked. > > > > > > > > > > Thus I salvaged the filenames found in the half-dropped root 363. > > > > > > > > > > Since it may contain confidential info, I send the filename list to you > > > > > > > > > > off-list. > > > > > > > > > > If you can remind what the root 363 is used for, and any possible > > > > > > > > > > operations which may be involved in that subvolume, it's better to reply > > > > > > > > > > it to the mail list so that we can get some clue on the root cause. > > > > > > > > > > Thanks, > > > > > > > > > > Qu > > > > > > > > > > On 2021/7/18 下午3:15, Qu Wenruo wrote: > > > > > > > > > > > On 2021/7/18 下午1:26, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > > > May I know if there are any leads on this? What should I do for now? > > > > > > > > > > > > Sorry for the late reply. > > > > > > > > > > > > With the image dump, it's much easier to find what's going wrong. > > > > > > > > > > > > - About root 363 > > > > > > > > > > > > It's an orphan root, thus user can't access it directly. > > > > > > > > > > > > > > > > > > Furthermore, it's being dropped, thus "btrfs ins dump-tree -t 363" > > > > > > > > > > > > reports transid error, as part of the tree has already been dropped, > > > > > > > > > > > > and this is expected. > > > > > > > > > > > > So far your fs is still OK, except that reported error. > > > > > > > > > > > > - About the offending tree block > > > > > > > > > > > > The offending tree block only belongs to the delete subvolume 363, > > > > > > > > > > > > thus it should be delete soon. > > > > > > > > > > > > > > > > > > But unfortunately due to the corrupted content, it's unable to be > > > > > > > > > > > > deleted. > > > > > > > > > > > > For now, if you can re-compile btrfs module, we can workaround the > > > > > > > > > > > > problem by temporarily disable read-time tree-checker so that the > > > > > > > > > > > > deletion can continue, and after the root 363 get fully deleted, the > > > > > > > > > > > > problem should be gone. > > > > > > > > > > > > Or you can use older kernel, any kernel <= v5.1 should not have the > > > > > > > > > > > > enhanced check, thus can continue with the subvolume deletion. > > > > > > > > > > > > If you want to go through the re-compile path, the needed diff is > > > > > > > > > > > > attached > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Qu > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > Lester > > > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > > > On Saturday, July 17th, 2021 at 8:51 PM, > > > > > > > > > > > > > > pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > > > > > I run btrfs ins dump-tree -t 363 unmounted but the same error > > > > > > > > > > > > > > > > appears. Rerunning btrfs check does not show any error. > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > > > Lester > > > > > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > > > > > On Saturday, July 17th, 2021 at 6:48 PM, Qu Wenruo - > > > > > > > > > > > > > > > > quwenruo.btrfs@gmx.com wrote: > > > > > > > > > > > > > > > > > On 2021/7/17 下午6:34, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > > > > > > > > > Unfortunately I cannot find subvolume 363 > > > > > > > > > > > > > > > > > > > > btrfs subvolume list /run/media/root > > > > > > > > > > ==================================== > > > > > > > > > > > > > > > > > > > > ID 361 gen 1814826 top level 584 path @/live/snapshot > > > > > > > > > > > > > > > > > > > > ID 364 gen 1814414 top level 5 path @vtmp/live/snapshot > > > > > > > > > > > > > > > > > > > > ID 369 gen 1814414 top level 5 path @vlmachines/live/snapshot > > > > > > > > > > > > > > > > > > > > ID 493 gen 1814414 top level 5 path @vlportables/live/snapshot > > > > > > > > > > > > > > > > > > > > ID 579 gen 1814828 top level 5 path @vlog/live/snapshot > > > > > > > > > > > > > > > > > > > > ID 580 gen 1814414 top level 5 path @vcache/live/snapshot > > > > > > > > > > > > > > > > > > > > ID 581 gen 1814414 top level 5 path @vlmongodb/live/snapshot > > > > > > > > > > > > > > > > > > > > ID 582 gen 1814414 top level 5 path @vlmysql/live/snapshot > > > > > > > > > > > > > > > > > > > > ID 583 gen 1814414 top level 5 path @vspool/live/snapshot > > > > > > > > > > > > > > > > > > > > ID 584 gen 1814414 top level 5 path @ > > > > > > > > > > > > > > > > > > > > ID 598 gen 1813420 top level 584 path @/4/snapshot > > > > > > > > > > > > > > > > > > Maybe 363 is some subvolume get deleted and later snapshot of it still > > > > > > > > > > > > > > > > > > exists. > > > > > > > > > > > > > > > > > > This will be harder to debug. > > > > > > > > > > > > > > > > > > Can you take a btrfs-image dump of your filesystem? (needs to be taken > > > > > > > > > > > > > > > > > > with the fs unmounted). > > > > > > > > > > > > > > > > > > The dumped image will contain your metadata, including file names and > > > > > > > > > > > > > > > > > > directory structures, but no data inside those files. > > > > > > > > > > > > > > > > > > Although btrfs-image has "-s" option to mask the filenames, but > > > > > > > > > > > > > > > > > > considering the filename in this case is useful to locate the inode, I > > > > > > > > > > > > > > > > > > guess it's better to take the image without any "-s" option. > > > > > > > > > > > > > > > > > > > btrfs ins dump-tree -t 363 /dev/dm-0 | grep -A 5 "(286 " > > > > > > > > > > ======================================================== > > > > > > > > > > > > > > > > > > > > parent transid verify failed on 174170742784 wanted 1789655 found > > > > > > > > > > > > > > > > > > > > 1812621 > > > > > > > > > > > > > > > > > > > > parent transid verify failed on 174170742784 wanted 1789655 found > > > > > > > > > > > > > > > > > > > > 1812621 > > > > > > > > > > > > > > > > > > > > parent transid verify failed on 174170742784 wanted 1789655 found > > > > > > > > > > > > > > > > > > > > 1812621 > > > > > > > > > > > > > > > > > > > > Ignoring transid failure > > > > > > > > > > > > > > > > > > > > ERROR: child eb corrupted: parent bytenr=174170738688 item=0 parent > > > > > > > > > > > > > > > > > > > > level=2 child bytenr=174170742784 child level=0 > > > > > > > > > > > > > > > > > > This transid mismatch may be a problem when running dump-tree on > > > > > > > > > > > > > > > > > > mounted > > > > > > > > > > > > > > > > > > fs, since you mentioned btrfs check reported no error, there shouldn't > > > > > > > > > > > > > > > > > > be a transid mismatch error. > > > > > > > > > > > > > > > > > > Anyway, if you can upload the btrfs-image dump, it would be much easier > > > > > > > > > > > > > > > > > > for us to debug and find out what's really going. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > > > > > > > > > On Saturday, July 17th, 2021 at 6:12 PM, Qu Wenruo wqu@suse.com wrote: > > > > > > > > > > > > > > > > > > > > > On 2021/7/17 下午4:57, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > > > > > > > > > > > > > I don't know how the directory was created but last month, I used > > > > > > > > > > > > > > > > > > > > > > > > btrfs device add and btrfs device remove to move the filesystem > > > > > > > > > > > > > > > > > > > > > > > > from one partition to another. It failed because of the same > > > > > > > > > > > > > > > > > > > > > > > > error and was advised to use btrfs replace instead. I don't know > > > > > > > > > > > > > > > > > > > > > > > > if the error also happened before I move the file system as I > > > > > > > > > > > > > > > > > > > > > > > > don't have any previous logs. > > > > > > > > > > > > > > > > > > > > > > It definitely happens before you moving the fs. > > > > > > > > > > > > > > > > > > > > > > As regular dev replacing/add/move only relocates the metadata, but > > > > > > > > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > > > > > > touching the fs trees. > > > > > > > > > > > > > > > > > > > > > > > Here is the result when I search for the inodes you mentioned if > > > > > > > > > > > > > > > > > > > > > > > > it helps: > > > > > > > > > > > > > > > > > > > > > > > > find /run/media/root -inum 260 -exec ls -ldi {} \; > > > > > > > > > > > > ================================================== > > > > > > > > > > > > > > > > > > > > > > > > 260 -rw-r--r-- 1 root root 36864 Jun 25 06:22 > > > > > > > > > > > > > > > > > > > > > > > > /run/media/root/@vcache/live/snapshot/app-info/cache/en_US.cache > > > > > > > > > > > > > > > > > > > > > > > > 260 drwx------ 1 mongodb mongodb 136 Sep 12 2020 > > > > > > > > > > > > > > > > > > > > > > > > /run/media/root/@vlmongodb/live/snapshot/diagnostic.data > > > > > > > > > > > > > > > > > > > > > > > > 260 -rw-rw---- 1 mysql mysql 50331648 Sep 13 2015 > > > > > > > > > > > > > > > > > > > > > > > > /run/media/root/@vlmysql/live/snapshot/ib_logfile0 > > > > > > > > > > > > > > > > > > > > > > > > 260 -rw-r----- 1 root lp 8641 Mar 5 2014 > > > > > > > > > > > > > > > > > > > > > > > > /run/media/root/@vspool/live/snapshot/cups/d00001-001 > > > > > > > > > > > > > > > > > > > > > > > > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 > > > > > > > > > > > > > > > > > > > > > > > > /run/media/root/@/live/snapshot/sys > > > > > > > > > > > > > > > > > > > > > > > > 260 dr-xr-xr-x 1 root root 0 Sep 13 2013 > > > > > > > > > > > > > > > > > > > > > > > > /run/media/root/@/4/snapshot/sys > > > > > > > > > > > > > > > > > > > > > > Since btrfs can have the same inode number inside different > > > > > > > > > > > > > > > > > > > > > > subvolumes, > > > > > > > > > > > > > > > > > > > > > > you may want to limit the search inside subvolume 363. > > > > > > > > > > > > > > > > > > > > > > "-mount" option of find can do that, you only need to locate > > > > > > > > > > > > > > > > > > > > > > subvolume > > > > > > > > > > > > > > > > > > > > > > 363 by "btrfs subv list". > > > > > > > > > > > > > > > > > > > > > > But from these output I guess above two "sys" directory are more > > > > > > > > > > > > > > > > > > > > > > possible. > > > > > > > > > > > > > > > > > > > > > > Is there any directory named "blaklight" inside those directory? > > > > > > > > > > > > > > > > > > > > > > > find /run/media/root -inum 286 -exec ls -ldi {} \; > > > > > > > > > > > > ================================================== > > > > > > > > > > > > > > > > > > > > > > > > 286 -rw-r--r-- 1 root root 96 Aug 16 2015 > > > > > > > > > > > > > > > > > > > > > > > > /run/media/root/@vcache/live/snapshot/fontconfig/4b172ca7f111e3cffadc3636415fead9-le64.cache-4 > > > > > > > > > > > > > > > > > > > > > > > > 286 -rw-rw---- 1 mysql mysql 4096 Sep 15 2013 > > > > > > > > > > > > > > > > > > > > > > > > /run/media/root/@vlmysql/live/snapshot/mysql/columns_priv.MYI > > > > > > > > > > > > > > > > > > > > > > > > 286 -rw-r-----+ 1 root systemd-journal 16777216 Jul 4 01:14 > > > > > > > > > > > > > > > > > > > > > > > > /run/media/root/@vlog/live/snapshot/journal/5098dd7845ae46d3ba1826c68a809a7c/user-1000@fbd9f65d0ea349f6b996716280e6c4dd-00000000002314c5-0005c5cb84a3a438.journal > > > > > > > > > > > > > > > > > > > > > > This is interesting, it means the inode 286 is not accessible. > > > > > > > > > > > > > > > > > > > > > > It can be some orphan inode, but would you dump subvolume 363 then > > > > > > > > > > > > > > > > > > > > > > try > > > > > > > > > > > > > > > > > > > > > > to locate the inode 286? > > > > > > > > > > > > > > > > > > > > > > One example command would be: > > > > > > > > > > > > > > > > > > > > > > btrfs ins dump-tree -t 363 <dev> | grep -A 5 "(286 " > > > > > > > > > > > ==================================================== > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > > > > > > > > > Directories with pattern /root/@<dir>/live/snapshot/ are > > > > > > > > > > > > > > > > > > > > > > > > subvolumes and directories with pattern > > > > > > > > > > > > > > > > > > > > > > > > /root/@<dir>/<num>/snapshot/ are snapshots of live. > > > > > > > > > > > > > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > > > > > > > > > > > > > On Saturday, July 17th, 2021 at 4:14 PM, Qu Wenruo > > > > > > > > > > > > > > > > > > > > > > > > quwenruo.btrfs@gmx.com wrote: > > > > > > > > > > > > > > > > > > > > > > > > > On 2021/7/17 下午3:51, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Qu, > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please see below for the dump. > > > > > > > > > > > > > > > > > > > > > > > > > > > > btrfs-progs v5.12.1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > leaf 174113599488 items 18 free space 2008 generation 1330906 > > > > > > > > > > > > > > > > > > > > > > > > > > > > owner 363 > > > > > > > > > > > > > > > > > > > > > > > > > > > > leaf 174113599488 flags 0x1(WRITTEN) backref revision 1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > fs uuid a7d327c4-8594-4116-a6f8-8aa2a4162063 > > > > > > > > > > > > > > > > > > > > > > > > > > > > chunk uuid f885f49e-14a0-4c80-9c12-c2302b9a0229 > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 0 key (5471 INODE_ITEM 0) itemoff 3835 itemsize 160 > > > > > > > > > > > > > > > > > > > > > > > > > > > > generation 2063 transid 27726 size 40 nbytes 40 > > > > > > > > > > > > > > > > > > > > > > > > > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > sequence 1501 flags 0x0(none) > > > > > > > > > > > > > > > > > > > > > > > > > > > > atime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > > > > > > > > > > > > > > > > > ctime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > > > > > > > > > > > > > > > > > mtime 1386484844.468769570 (2013-12-08 14:40:44) > > > > > > > > > > > > > > > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 1 key (5471 INODE_REF 4399) itemoff 3824 itemsize 11 > > > > > > > > > > > > > > > > > > > > > > > > > > > > index 12 namelen 1 name: 8 > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 2 key (5471 EXTENT_DATA 0) itemoff 3763 itemsize 61 > > > > > > > > > > > > > > > > > > > > > > > > > > > > generation 27726 type 0 (inline) > > > > > > > > > > > > > > > > > > > > > > > > > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 3 key (5645 INODE_ITEM 0) itemoff 3603 itemsize 160 > > > > > > > > > > > > > > > > > > > > > > > > > > > > generation 2542 transid 61261 size 40 nbytes 40 > > > > > > > > > > > > > > > > > > > > > > > > > > > > block group 0 mode 100600 links 1 uid 0 gid 100 rdev 0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > sequence 24769 flags 0x0(none) > > > > > > > > > > > > > > > > > > > > > > > > > > > > atime 1394335806.351857522 (2014-03-09 11:30:06) > > > > > > > > > > > > > > > > > > > > > > > > > > > > ctime 1394335827.344389955 (2014-03-09 11:30:27) > > > > > > > > > > > > > > > > > > > > > > > > > > > > mtime 1394335827.344389955 (2014-03-09 11:30:27) > > > > > > > > > > > > > > > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 4 key (5645 INODE_REF 4399) itemoff 3592 itemsize 11 > > > > > > > > > > > > > > > > > > > > > > > > > > > > index 13 namelen 1 name: 7 > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 5 key (5645 EXTENT_DATA 0) itemoff 3531 itemsize 61 > > > > > > > > > > > > > > > > > > > > > > > > > > > > generation 61261 type 0 (inline) > > > > > > > > > > > > > > > > > > > > > > > > > > > > inline extent data size 40 ram_bytes 40 compression 0 (none) > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 6 key (7222 INODE_ITEM 0) itemoff 3371 itemsize 160 > > > > > > > > > > > > > > > > > > > > > > > > > > > > generation 5754 transid 5767 size 307 nbytes 307 > > > > > > > > > > > > > > > > > > > > > > > > > > > > block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > sequence 7 flags 0x0(none) > > > > > > > > > > > > > > > > > > > > > > > > > > > > atime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > > > > > > > > > > > > > > > > > ctime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > > > > > > > > > > > > > > > > > mtime 1379834835.428558020 (2013-09-22 15:27:15) > > > > > > > > > > > > > > > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 7 key (7222 INODE_REF 287) itemoff 3344 itemsize 27 > > > > > > > > > > > > > > > > > > > > > > > > > > > > index 6 namelen 17 name: dhcpcd-eth0.lease > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 8 key (7222 EXTENT_DATA 0) itemoff 3016 itemsize 328 > > > > > > > > > > > > > > > > > > > > > > > > > > > > generation 5767 type 0 (inline) > > > > > > > > > > > > > > > > > > > > > > > > > > > > inline extent data size 307 ram_bytes 307 compression 0 (none) > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 9 key (7415 INODE_ITEM 0) itemoff 2856 itemsize 160 > > > > > > > > > > > > > > > > > > > > > > > > > > > > generation 5904 transid 1330906 size 180 nbytes 0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > block group 0 mode 40755 links 2 uid 0 gid 0 rdev 0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > sequence 177 flags 0x0(none) > > > > > > > > > > > > > > > > > > > > > > > > > > > > atime 1483277713.141980592 (2017-01-01 21:35:13) > > > > > > > > > > > > > > > > > > > > > > > > > > > > ctime 1563162901.234656246 (2019-07-15 11:55:01) > > > > > > > > > > > > > > > > > > > > > > > > > > > > mtime 1406534032.158605559 (2014-07-28 15:53:52) > > > > > > > > > > > > > > > > > > > > > > > > > > > > otime 0.0 (1970-01-01 08:00:00) > > > > > > > > > > > > > > > > > > > > > > > > > > This inode is indeed a directory. > > > > > > > > > > > > > > > > > > > > > > > > > > But it has two hard links, which is definitely something > > > > > > > > > > > > > > > > > > > > > > > > > > unexpected. > > > > > > > > > > > > > > > > > > > > > > > > > > Under Linux we shouldn't have any hardlink for directory, as it > > > > > > > > > > > > > > > > > > > > > > > > > > would > > > > > > > > > > > > > > > > > > > > > > > > > > easily lead to loops. > > > > > > > > > > > > > > > > > > > > > > > > > > > item 10 key (7415 INODE_REF 260) itemoff 2837 itemsize 19 > > > > > > > > > > > > > > > > > > > > > > > > > > > > index 28 namelen 9 name: backlight > > > > > > > > > > > > > > > > > > > > > > > > > > Its parent inode is 260 in the same root, with the name backlight. > > > > > > > > > > > > > > > > > > > > > > > > > > > item 11 key (7415 INODE_REF 286) itemoff 2818 itemsize 19 > > > > > > > > > > > > > > > > > > > > > > > > > > > > index 3 namelen 9 name: backlight > > > > > > > > > > > > > > > > > > > > > > > > > > Another hardlink in inode 286, which is definitely a regular thing. > > > > > > > > > > > > > > > > > > > > > > > > > > Btrfs-progs lacks the ability to detect such problem, we need to > > > > > > > > > > > > > > > > > > > > > > > > > > enhance > > > > > > > > > > > > > > > > > > > > > > > > > > it first. > > > > > > > > > > > > > > > > > > > > > > > > > > But do you have any idea how this directory get created? > > > > > > > > > > > > > > > > > > > > > > > > > > It looks like the content of sysfs. > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > > > > > > > > > > > > > item 12 key (7415 DIR_ITEM 3128336373) itemoff 2746 itemsize 72 > > > > > > > > > > > > > > > > > > > > > > > > > > > > location key (120417 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > > > > > > > > > transid 117279 data_len 0 name_len 42 > > > > > > > > > > > > > > > > > > > > > > > > > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 13 key (7415 DIR_ITEM 3218198317) itemoff 2705 itemsize 41 > > > > > > > > > > > > > > > > > > > > > > > > > > > > location key (7487 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > > > > > > > > > transid 5992 data_len 0 name_len 11 > > > > > > > > > > > > > > > > > > > > > > > > > > > > name: acpi_video0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 14 key (7415 DIR_ITEM 3582254411) itemoff 2638 itemsize 67 > > > > > > > > > > > > > > > > > > > > > > > > > > > > location key (55325 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > > > > > > > > > transid 63351 data_len 0 name_len 37 > > > > > > > > > > > > > > > > > > > > > > > > > > > > name: platform-VPC2004:00:backlight:ideapad > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 15 key (7415 DIR_INDEX 2) itemoff 2597 itemsize 41 > > > > > > > > > > > > > > > > > > > > > > > > > > > > location key (7487 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > > > > > > > > > transid 5992 data_len 0 name_len 11 > > > > > > > > > > > > > > > > > > > > > > > > > > > > name: acpi_video0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 16 key (7415 DIR_INDEX 4) itemoff 2530 itemsize 67 > > > > > > > > > > > > > > > > > > > > > > > > > > > > location key (55325 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > > > > > > > > > transid 63351 data_len 0 name_len 37 > > > > > > > > > > > > > > > > > > > > > > > > > > > > name: platform-VPC2004:00:backlight:ideapad > > > > > > > > > > > > > > > > > > > > > > > > > > > > item 17 key (7415 DIR_INDEX 5) itemoff 2458 itemsize 72 > > > > > > > > > > > > > > > > > > > > > > > > > > > > location key (120417 INODE_ITEM 0) type FILE > > > > > > > > > > > > > > > > > > > > > > > > > > > > transid 117279 data_len 0 name_len 42 > > > > > > > > > > > > > > > > > > > > > > > > > > > > name: pci-0000:00:02.0:backlight:intel_backlight > > > > > > > > > > > > > > > > > > > > > > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Saturday, July 17th, 2021 at 3:05 PM, Qu Wenruo > > > > > > > > > > > > > > > > > > > > > > > > > > > > quwenruo.btrfs@gmx.com wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 2021/7/17 上午9:45, pepperpoint@mb.ardentcoding.com wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I see this message on dmesg: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [ 2452.256756] BTRFS critical (device dm-0): corrupt leaf: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > root=363 block=174113599488 slot=9 ino=7415, invalid nlink: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > has 2 expect no more than 1 for dir > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [ 2452.256776] BTRFS error (device dm-0): block=174113599488 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > read time tree block corruption detected > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please provide the following dump: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > btrfs ins dump-tree -b 174113599488 /dev/dm-0 > > > > > > > > > > > > > > > ============================================= > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Qu > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > When I run btrfs scrub and btrfs check, no error was detected. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am running Linux 5.12.15-arch1-1 and btrfs-progs v5.12.1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > How should I fix this error? ^ permalink raw reply [flat|nested] 75+ messages in thread
* read time tree block corruption detected @ 2021-04-16 19:35 Gervais, Francois 2021-04-17 1:01 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Gervais, Francois @ 2021-04-16 19:35 UTC (permalink / raw) To: linux-btrfs We are using btrfs on one of our embedded devices and we got filesystem corruption on one of them. This product undergo a lot of tests on our side and apparently it's the first it happened so it seems to be a pretty rare occurrence. However we still want to get to the bottom of this to ensure it doesn't happen in the future. Some background: - The corruption happened on kernel v5.4.72. - On the debug device I'm on master (v5.12.0-rc7) hoping it might help to have all the latest patches. Here what kernel v5.12.0-rc7 tells me when trying to mount the partition: Apr 16 19:31:45 buildroot kernel: BTRFS info (device loop0p3): disk space caching is enabled Apr 16 19:31:45 buildroot kernel: BTRFS info (device loop0p3): has skinny extents Apr 16 19:31:45 buildroot kernel: BTRFS info (device loop0p3): start tree-log replay Apr 16 19:31:45 buildroot kernel: BTRFS critical (device loop0p3): corrupt leaf: root=18446744073709551610 block=790151168 slot=5 ino=5007, inode ref overflow, ptr 15853 end 15861 namelen 294 Apr 16 19:31:45 buildroot kernel: BTRFS error (device loop0p3): block=790151168 read time tree block corruption detected Apr 16 19:31:45 buildroot kernel: BTRFS critical (device loop0p3): corrupt leaf: root=18446744073709551610 block=790151168 slot=5 ino=5007, inode ref overflow, ptr 15853 end 15861 namelen 294 Apr 16 19:31:45 buildroot kernel: BTRFS error (device loop0p3): block=790151168 read time tree block corruption detected Apr 16 19:31:45 buildroot kernel: BTRFS: error (device loop0p3) in btrfs_recover_log_trees:6246: errno=-5 IO failure (Couldn't read tree log root.) Apr 16 19:31:45 buildroot kernel: BTRFS: error (device loop0p3) in btrfs_replay_log:2341: errno=-5 IO failure (Failed to recover log tree) Apr 16 19:31:45 buildroot e512c123daaa[468]: mount: /root/mnt: can't read superblock on /dev/loop0p3. Apr 16 19:31:45 buildroot kernel: BTRFS error (device loop0p3): open_ctree failed: -5 Any suggestions? ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-04-16 19:35 read " Gervais, Francois @ 2021-04-17 1:01 ` Qu Wenruo 2021-04-19 13:20 ` Gervais, Francois 0 siblings, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2021-04-17 1:01 UTC (permalink / raw) To: Gervais, Francois, linux-btrfs On 2021/4/17 上午3:35, Gervais, Francois wrote: > We are using btrfs on one of our embedded devices and we got filesystem corruption on one of them. > > This product undergo a lot of tests on our side and apparently it's the first it happened so it seems to be a pretty rare occurrence. However we still want to get to the bottom of this to ensure it doesn't happen in the future. > > Some background: > - The corruption happened on kernel v5.4.72. > - On the debug device I'm on master (v5.12.0-rc7) hoping it might help to have all the latest patches. > > Here what kernel v5.12.0-rc7 tells me when trying to mount the partition: > > Apr 16 19:31:45 buildroot kernel: BTRFS info (device loop0p3): disk space caching is enabled > Apr 16 19:31:45 buildroot kernel: BTRFS info (device loop0p3): has skinny extents > Apr 16 19:31:45 buildroot kernel: BTRFS info (device loop0p3): start tree-log replay > Apr 16 19:31:45 buildroot kernel: BTRFS critical (device loop0p3): corrupt leaf: root=18446744073709551610 block=790151168 slot=5 ino=5007, inode ref overflow, ptr 15853 end 15861 namelen 294 Please provide the following dump: #btrfs ins dump-tree -b 18446744073709551610 /dev/loop0p3 I'm wondering why write-time tree-check didn't catch it. Thanks, Qu > Apr 16 19:31:45 buildroot kernel: BTRFS error (device loop0p3): block=790151168 read time tree block corruption detected > Apr 16 19:31:45 buildroot kernel: BTRFS critical (device loop0p3): corrupt leaf: root=18446744073709551610 block=790151168 slot=5 ino=5007, inode ref overflow, ptr 15853 end 15861 namelen 294 > Apr 16 19:31:45 buildroot kernel: BTRFS error (device loop0p3): block=790151168 read time tree block corruption detected > Apr 16 19:31:45 buildroot kernel: BTRFS: error (device loop0p3) in btrfs_recover_log_trees:6246: errno=-5 IO failure (Couldn't read tree log root.) > Apr 16 19:31:45 buildroot kernel: BTRFS: error (device loop0p3) in btrfs_replay_log:2341: errno=-5 IO failure (Failed to recover log tree) > Apr 16 19:31:45 buildroot e512c123daaa[468]: mount: /root/mnt: can't read superblock on /dev/loop0p3. > Apr 16 19:31:45 buildroot kernel: BTRFS error (device loop0p3): open_ctree failed: -5 > > Any suggestions? > ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-04-17 1:01 ` Qu Wenruo @ 2021-04-19 13:20 ` Gervais, Francois 2021-04-19 13:33 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Gervais, Francois @ 2021-04-19 13:20 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs > Please provide the following dump: > #btrfs ins dump-tree -b 18446744073709551610 /dev/loop0p3 > > I'm wondering why write-time tree-check didn't catch it. > > Thanks, > Qu I get: root@debug:~# btrfs ins dump-tree -b 18446744073709551610 /dev/loop0p3 btrfs-progs v5.7 ERROR: tree block bytenr 18446744073709551610 is not aligned to sectorsize 4096 We have an unusual partition table due to an hardware (cpu) requirement. This might be the source of this error? Disk /dev/loop0: 40763392 sectors, 19.4 GiB Sector size (logical/physical): 512/512 bytes Disk identifier (GUID): A18E4543-634A-4E8C-B55D-DA1E217C4D98 Partition table holds up to 24 entries Main partition table begins at sector 2 and ends at sector 7 First usable sector is 8, last usable sector is 40763384 Partitions will be aligned on 8-sector boundaries Total free space is 0 sectors (0 bytes) Number Start (sector) End (sector) Size Code Name 1 8 32775 16.0 MiB 8300 2 32776 237575 100.0 MiB 8300 3 237576 40763384 19.3 GiB 8300 ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-04-19 13:20 ` Gervais, Francois @ 2021-04-19 13:33 ` Qu Wenruo 2021-04-19 14:56 ` Gervais, Francois 0 siblings, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2021-04-19 13:33 UTC (permalink / raw) To: Gervais, Francois, linux-btrfs On 2021/4/19 下午9:20, Gervais, Francois wrote: >> Please provide the following dump: >> #btrfs ins dump-tree -b 18446744073709551610 /dev/loop0p3 >> >> I'm wondering why write-time tree-check didn't catch it. >> >> Thanks, >> Qu > > I get: > > root@debug:~# btrfs ins dump-tree -b 18446744073709551610 /dev/loop0p3 > btrfs-progs v5.7 > ERROR: tree block bytenr 18446744073709551610 is not aligned to sectorsize 4096 My bad, wrong number. The correct number command is: # btrfs ins dump-tree -b 790151168 /dev/loop0p3 Thanks, Qu > > We have an unusual partition table due to an hardware (cpu) requirement. > This might be the source of this error? > > Disk /dev/loop0: 40763392 sectors, 19.4 GiB > Sector size (logical/physical): 512/512 bytes > Disk identifier (GUID): A18E4543-634A-4E8C-B55D-DA1E217C4D98 > Partition table holds up to 24 entries > Main partition table begins at sector 2 and ends at sector 7 > First usable sector is 8, last usable sector is 40763384 > Partitions will be aligned on 8-sector boundaries > Total free space is 0 sectors (0 bytes) > > Number Start (sector) End (sector) Size Code Name > 1 8 32775 16.0 MiB 8300 > 2 32776 237575 100.0 MiB 8300 > 3 237576 40763384 19.3 GiB 8300 > > > > ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-04-19 13:33 ` Qu Wenruo @ 2021-04-19 14:56 ` Gervais, Francois 2021-04-20 1:34 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Gervais, Francois @ 2021-04-19 14:56 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs > My bad, wrong number. > > The correct number command is: > # btrfs ins dump-tree -b 790151168 /dev/loop0p3 root@debug:~# btrfs ins dump-tree -b 790151168 /dev/loop0p3 btrfs-progs v5.7 leaf 790151168 items 10 free space 15237 generation 219603 owner TREE_LOG leaf 790151168 flags 0x1(WRITTEN) backref revision 1 fs uuid 29d53427-f943-43ad-a99e-ac695d225d0b chunk uuid 04c4bf25-55ac-487e-97a3-fbdc84961b4a item 0 key (4614 INODE_ITEM 0) itemoff 16123 itemsize 160 generation 282 transid 219603 size 0 nbytes 0 block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0 sequence 1345948 flags 0x0(none) atime 1610373764.218465480 (2021-01-11 14:02:44) ctime 1617477830.389953334 (2021-04-03 19:23:50) mtime 1617477830.389953334 (2021-04-03 19:23:50) otime 606208.1 (1970-01-08 00:23:28) item 1 key (4614 INODE_REF 1020) itemoff 16101 itemsize 22 index 1217 namelen 12 name: brokerStatus item 2 key (4996 INODE_ITEM 0) itemoff 15941 itemsize 160 generation 290 transid 219603 size 0 nbytes 0 block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0 sequence 4801736 flags 0x0(none) atime 1617304887.496533028 (2021-04-01 19:21:27) ctime 1617477830.681955095 (2021-04-03 19:23:50) mtime 1617477830.681955095 (2021-04-03 19:23:50) otime 0.0 (1970-01-01 00:00:00) item 3 key (4996 INODE_REF 4715) itemoff 15920 itemsize 21 index 9 namelen 11 name: scodes.conf item 4 key (5007 INODE_ITEM 0) itemoff 15760 itemsize 160 generation 294 transid 219603 size 0 nbytes 18446462598731726987 block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0 sequence 476091 flags 0x0(none) atime 1610373772.750632843 (2021-01-11 14:02:52) ctime 1617477826.205928110 (2021-04-03 19:23:46) mtime 1617477826.205928110 (2021-04-03 19:23:46) otime 0.0 (1970-01-01 00:00:00) item 5 key (5007 INODE_REF 4727) itemoff 15732 itemsize 28 index 0 namelen 0 name: index 0 namelen 0 name: index 0 namelen 294 name: item 6 key (5041 INODE_ITEM 0) itemoff 15572 itemsize 160 generation 295 transid 219603 size 4096 nbytes 4096 block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0 sequence 321954 flags 0x0(none) atime 1610373832.763235044 (2021-01-11 14:03:52) ctime 1617477815.541863825 (2021-04-03 19:23:35) mtime 1617477815.541863825 (2021-04-03 19:23:35) otime 0.0 (1970-01-01 00:00:00) item 7 key (5041 INODE_REF 4727) itemoff 15544 itemsize 28 index 12 namelen 18 name: health_metrics.txt item 8 key (5041 EXTENT_DATA 0) itemoff 15491 itemsize 53 generation 219603 type 1 (regular) extent data disk byte 12746752 nr 4096 extent data offset 0 nr 4096 ram 4096 extent compression 0 (none) item 9 key (EXTENT_CSUM EXTENT_CSUM 12746752) itemoff 15487 itemsize 4 range start 12746752 end 12750848 length 4096 ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-04-19 14:56 ` Gervais, Francois @ 2021-04-20 1:34 ` Qu Wenruo 2021-04-20 14:19 ` Gervais, Francois 0 siblings, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2021-04-20 1:34 UTC (permalink / raw) To: Gervais, Francois, linux-btrfs, Filipe Manana On 2021/4/19 下午10:56, Gervais, Francois wrote: >> My bad, wrong number. >> >> The correct number command is: >> # btrfs ins dump-tree -b 790151168 /dev/loop0p3 > > > root@debug:~# btrfs ins dump-tree -b 790151168 /dev/loop0p3 > btrfs-progs v5.7 [...] > item 4 key (5007 INODE_ITEM 0) itemoff 15760 itemsize 160 > generation 294 transid 219603 size 0 nbytes 18446462598731726987 The nbytes looks very strange. It's 0x0xfffeffffffef008b, which definitely looks aweful for an empty inode. > block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0 > sequence 476091 flags 0x0(none) > atime 1610373772.750632843 (2021-01-11 14:02:52) > ctime 1617477826.205928110 (2021-04-03 19:23:46) > mtime 1617477826.205928110 (2021-04-03 19:23:46) > otime 0.0 (1970-01-01 00:00:00) > item 5 key (5007 INODE_REF 4727) itemoff 15732 itemsize 28 > index 0 namelen 0 name: > index 0 namelen 0 name: > index 0 namelen 294 name: Definitely corrupted. I'm afraid tree-checker is correct. The log tree is corrupted. And the check to detect such corrupted inode ref is only introduced in v5.5 kernel, no wonder v5.4 kernel didn't catch it at runtime. I don't have any idea why this could happen, as it doesn't look like an obvious memory flip. Maybe Filipe could have some clue on this? Thanks, Qu > item 6 key (5041 INODE_ITEM 0) itemoff 15572 itemsize 160 > generation 295 transid 219603 size 4096 nbytes 4096 > block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0 > sequence 321954 flags 0x0(none) > atime 1610373832.763235044 (2021-01-11 14:03:52) > ctime 1617477815.541863825 (2021-04-03 19:23:35) > mtime 1617477815.541863825 (2021-04-03 19:23:35) > otime 0.0 (1970-01-01 00:00:00) > item 7 key (5041 INODE_REF 4727) itemoff 15544 itemsize 28 > index 12 namelen 18 name: health_metrics.txt > item 8 key (5041 EXTENT_DATA 0) itemoff 15491 itemsize 53 > generation 219603 type 1 (regular) > extent data disk byte 12746752 nr 4096 > extent data offset 0 nr 4096 ram 4096 > extent compression 0 (none) > item 9 key (EXTENT_CSUM EXTENT_CSUM 12746752) itemoff 15487 itemsize 4 > range start 12746752 end 12750848 length 4096 > ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-04-20 1:34 ` Qu Wenruo @ 2021-04-20 14:19 ` Gervais, Francois 2021-04-20 23:47 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Gervais, Francois @ 2021-04-20 14:19 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs, Filipe Manana > On 2021/4/19 下午10:56, Gervais, Francois wrote: >>> My bad, wrong number. >>> >>> The correct number command is: >>> # btrfs ins dump-tree -b 790151168 /dev/loop0p3 >> >> >> root@debug:~# btrfs ins dump-tree -b 790151168 /dev/loop0p3 >> btrfs-progs v5.7 > [...] >> item 4 key (5007 INODE_ITEM 0) itemoff 15760 itemsize 160 >> generation 294 transid 219603 size 0 nbytes 18446462598731726987 > > The nbytes looks very strange. > > It's 0x0xfffeffffffef008b, which definitely looks aweful for an empty inode. > >> block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0 >> sequence 476091 flags 0x0(none) >> atime 1610373772.750632843 (2021-01-11 14:02:52) >> ctime 1617477826.205928110 (2021-04-03 19:23:46) >> mtime 1617477826.205928110 (2021-04-03 19:23:46) >> otime 0.0 (1970-01-01 00:00:00) >> item 5 key (5007 INODE_REF 4727) itemoff 15732 itemsize 28 >> index 0 namelen 0 name: >> index 0 namelen 0 name: >> index 0 namelen 294 name: > > Definitely corrupted. I'm afraid tree-checker is correct. > > The log tree is corrupted. > And the check to detect such corrupted inode ref is only introduced in > v5.5 kernel, no wonder v5.4 kernel didn't catch it at runtime. Would detecting it at runtime with a newer kernel have helped in any way with the corruption? > > I don't have any idea why this could happen, as it doesn't look like an > obvious memory flip. The test engineer says that the last thing he did was remove power from the device. Could power loss be the cause of this issue? > > Maybe Filipe could have some clue on this? > > Thanks, > Qu > >> item 6 key (5041 INODE_ITEM 0) itemoff 15572 itemsize 160 >> generation 295 transid 219603 size 4096 nbytes 4096 >> block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0 >> sequence 321954 flags 0x0(none) >> atime 1610373832.763235044 (2021-01-11 14:03:52) >> ctime 1617477815.541863825 (2021-04-03 19:23:35) >> mtime 1617477815.541863825 (2021-04-03 19:23:35) >> otime 0.0 (1970-01-01 00:00:00) >> item 7 key (5041 INODE_REF 4727) itemoff 15544 itemsize 28 >> index 12 namelen 18 name: health_metrics.txt >> item 8 key (5041 EXTENT_DATA 0) itemoff 15491 itemsize 53 >> generation 219603 type 1 (regular) >> extent data disk byte 12746752 nr 4096 >> extent data offset 0 nr 4096 ram 4096 >> extent compression 0 (none) >> item 9 key (EXTENT_CSUM EXTENT_CSUM 12746752) itemoff 15487 itemsize 4 >> range start 12746752 end 12750848 length 4096 >> ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-04-20 14:19 ` Gervais, Francois @ 2021-04-20 23:47 ` Qu Wenruo 2021-04-21 14:17 ` Gervais, Francois 0 siblings, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2021-04-20 23:47 UTC (permalink / raw) To: Gervais, Francois, linux-btrfs, Filipe Manana On 2021/4/20 下午10:19, Gervais, Francois wrote: >> On 2021/4/19 下午10:56, Gervais, Francois wrote: >>>> My bad, wrong number. >>>> >>>> The correct number command is: >>>> # btrfs ins dump-tree -b 790151168 /dev/loop0p3 >>> >>> >>> root@debug:~# btrfs ins dump-tree -b 790151168 /dev/loop0p3 >>> btrfs-progs v5.7 >> [...] >>> item 4 key (5007 INODE_ITEM 0) itemoff 15760 itemsize 160 >>> generation 294 transid 219603 size 0 nbytes 18446462598731726987 >> >> The nbytes looks very strange. >> >> It's 0x0xfffeffffffef008b, which definitely looks aweful for an empty inode. >> >>> block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0 >>> sequence 476091 flags 0x0(none) >>> atime 1610373772.750632843 (2021-01-11 14:02:52) >>> ctime 1617477826.205928110 (2021-04-03 19:23:46) >>> mtime 1617477826.205928110 (2021-04-03 19:23:46) >>> otime 0.0 (1970-01-01 00:00:00) >>> item 5 key (5007 INODE_REF 4727) itemoff 15732 itemsize 28 >>> index 0 namelen 0 name: >>> index 0 namelen 0 name: >>> index 0 namelen 294 name: >> >> Definitely corrupted. I'm afraid tree-checker is correct. >> >> The log tree is corrupted. >> And the check to detect such corrupted inode ref is only introduced in >> v5.5 kernel, no wonder v5.4 kernel didn't catch it at runtime. > > Would detecting it at runtime with a newer kernel have helped in any way with > the corruption? Yes, newer kernel will reject the write, so such damaged metadata won't reach disk. But that's just more graceful than corrupted fs. It will still cause error like transaction aborted. > >> >> I don't have any idea why this could happen, as it doesn't look like an >> obvious memory flip. > > The test engineer says that the last thing he did was remove power from the > device. > > Could power loss be the cause of this issue? It shouldn't. The log tree can only be exposed by power loss, but it's not designed to have such corrupted data on-disk. This normally means some code is wrong when generating log tree. Thanks, Qu > >> >> Maybe Filipe could have some clue on this? >> >> Thanks, >> Qu >> >>> item 6 key (5041 INODE_ITEM 0) itemoff 15572 itemsize 160 >>> generation 295 transid 219603 size 4096 nbytes 4096 >>> block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0 >>> sequence 321954 flags 0x0(none) >>> atime 1610373832.763235044 (2021-01-11 14:03:52) >>> ctime 1617477815.541863825 (2021-04-03 19:23:35) >>> mtime 1617477815.541863825 (2021-04-03 19:23:35) >>> otime 0.0 (1970-01-01 00:00:00) >>> item 7 key (5041 INODE_REF 4727) itemoff 15544 itemsize 28 >>> index 12 namelen 18 name: health_metrics.txt >>> item 8 key (5041 EXTENT_DATA 0) itemoff 15491 itemsize 53 >>> generation 219603 type 1 (regular) >>> extent data disk byte 12746752 nr 4096 >>> extent data offset 0 nr 4096 ram 4096 >>> extent compression 0 (none) >>> item 9 key (EXTENT_CSUM EXTENT_CSUM 12746752) itemoff 15487 itemsize 4 >>> range start 12746752 end 12750848 length 4096 >> > ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-04-20 23:47 ` Qu Wenruo @ 2021-04-21 14:17 ` Gervais, Francois 2021-04-21 23:01 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Gervais, Francois @ 2021-04-21 14:17 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs, Filipe Manana >> Would detecting it at runtime with a newer kernel have helped in any way with >> the corruption? > > Yes, newer kernel will reject the write, so such damaged metadata won't > reach disk. > > But that's just more graceful than corrupted fs. > It will still cause error like transaction aborted. > > [...] > >> Could power loss be the cause of this issue? > > It shouldn't. > The log tree can only be exposed by power loss, but it's not designed to > have such corrupted data on-disk. > > This normally means some code is wrong when generating log tree. Alright, for the next step, I feel the best is that we try to reproduce and get more information as of the events that caused state. A few questions ,if you want, before we start. - Anything you would recommend as of configuration of the device? - Should we run a newer kernel than our current v5.4? - Any debug you think would be useful to enable or add? ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-04-21 14:17 ` Gervais, Francois @ 2021-04-21 23:01 ` Qu Wenruo 2021-04-22 14:26 ` Gervais, Francois 0 siblings, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2021-04-21 23:01 UTC (permalink / raw) To: Gervais, Francois, linux-btrfs, Filipe Manana On 2021/4/21 下午10:17, Gervais, Francois wrote: >>> Would detecting it at runtime with a newer kernel have helped in any way with >>> the corruption? >> >> Yes, newer kernel will reject the write, so such damaged metadata won't >> reach disk. >> >> But that's just more graceful than corrupted fs. >> It will still cause error like transaction aborted. >> >> [...] >> >>> Could power loss be the cause of this issue? >> >> It shouldn't. >> The log tree can only be exposed by power loss, but it's not designed to >> have such corrupted data on-disk. >> >> This normally means some code is wrong when generating log tree. > > Alright, for the next step, I feel the best is that we try to reproduce and > get more information as of the events that caused state. > > A few questions ,if you want, before we start. > > - Anything you would recommend as of configuration of the device? You can test using dm-logwrites, which really logs every writes and replay it. By using dm-logwrites, you can emulate powerloss for each write operation. > - Should we run a newer kernel than our current v5.4? Definitely. In fact my fuzzy memory points me to some fix, but I can't remember exactly which fix. > - Any debug you think would be useful to enable or add? > Tree-checker, which is already enabled by default (in fact no way to disable) in newer kernels. Thanks, Qu ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-04-21 23:01 ` Qu Wenruo @ 2021-04-22 14:26 ` Gervais, Francois 2021-05-26 23:03 ` Gervais, Francois 0 siblings, 1 reply; 75+ messages in thread From: Gervais, Francois @ 2021-04-22 14:26 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs, Filipe Manana >> Alright, for the next step, I feel the best is that we try to reproduce and >> get more information as of the events that caused state. >> >> A few questions ,if you want, before we start. >> >> - Anything you would recommend as of configuration of the device? > > You can test using dm-logwrites, which really logs every writes and > replay it. > By using dm-logwrites, you can emulate powerloss for each write operation. > >> - Should we run a newer kernel than our current v5.4? > > Definitely. In fact my fuzzy memory points me to some fix, but I can't > remember exactly which fix. > >> - Any debug you think would be useful to enable or add? >> > > Tree-checker, which is already enabled by default (in fact no way to > disable) in newer kernels. > > Thanks, > Qu Thank you very much, I will report here if/when we get more information on this issue. ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-04-22 14:26 ` Gervais, Francois @ 2021-05-26 23:03 ` Gervais, Francois 2021-05-26 23:25 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Gervais, Francois @ 2021-05-26 23:03 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs, Filipe Manana I had to re-send this message as I forgot to send it as plain text earlier today. I apologize if some people receive it twice. >>> - Anything you would recommend as of configuration of the device? >> >> You can test using dm-logwrites, which really logs every writes and >> replay it. >> By using dm-logwrites, you can emulate powerloss for each write operation. I'm in the process of putting together a test scenario using dm-log-writes on our device but I have a few questions. 1. What I'm thinking of doing is do the firmware operation that the device was doing when we got the corruption but this time using dm-log-writes and then replay the log entry per entry. Will tree-checker catch to issue just by replaying or do I need to do an additional step for every log entry replay? (btrfs check? btrfs scrub?) 2. From what I understand, our corruption of the log-tree can either be software which would have requested a corrupted entry to be written to disk or hardware which would have corrupted the entry when trying to write it to disk. Debugging with dm-log-writes would not catch a corruption by the hardware if due to a one-off glitch or something right? >>> - Should we run a newer kernel than our current v5.4? >> >> Definitely. In fact my fuzzy memory points me to some fix, but I can't >> remember exactly which fix. >> >>> - Any debug you think would be useful to enable or add? >>> >> >> Tree-checker, which is already enabled by default (in fact no way to >> disable) in newer kernels. >> >> Thanks, >> Qu > > Thank you very much, > > I will report here if/when we get more information on this issue. ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2021-05-26 23:03 ` Gervais, Francois @ 2021-05-26 23:25 ` Qu Wenruo 0 siblings, 0 replies; 75+ messages in thread From: Qu Wenruo @ 2021-05-26 23:25 UTC (permalink / raw) To: Gervais, Francois, linux-btrfs, Filipe Manana On 2021/5/27 上午7:03, Gervais, Francois wrote: > I had to re-send this message as I forgot to send it as plain text earlier today. I apologize if some people receive it twice. > >>>> - Anything you would recommend as of configuration of the device? >>> >>> You can test using dm-logwrites, which really logs every writes and >>> replay it. >>> By using dm-logwrites, you can emulate powerloss for each write operation. > > I'm in the process of putting together a test scenario using dm-log-writes on our > device but I have a few questions. > > 1. What I'm thinking of doing is do the firmware operation that the device was > doing when we got the corruption but this time using dm-log-writes and then > replay the log entry per entry. > > Will tree-checker catch to issue just by replaying or do I need to do an additional > step for every log entry replay? (btrfs check? btrfs scrub?) Normally we only care about the filesystem consistency at FUA/FLUSH. Doing extra check at each replay won't hurt, but it would be very slow though. And for the tool to catch the problem, I'm afraid you have to use both kernel tree-checker (aka mount it), and btrfs-check to catch all problems. Tree-check can check all involved metadata, while btrfs-check doesn't check log tree. > > 2. From what I understand, our corruption of the log-tree can either be software > which would have requested a corrupted entry to be written to disk or hardware > which would have corrupted the entry when trying to write it to disk. I don't believe it's hardware, one point to notice is, all tree block written back to disk has checksum. If it's hardware problem leading to some data corruption on-disk, its csum should not match at all. Thus I believe this bug is mostly from btrfs. Filipe is doing a lot of fixes related to log tree, and some of them are related to item size overflow. But if you can reproduce the problem reliably, then it would be a great help to pin down the cause. > > Debugging with dm-log-writes would not catch a corruption by the hardware if due > to a one-off glitch or something right? Although I don't believe it's hardware problem, but just to be clear, no, it won't detect any hardware problem. Dm-log-writes is only to verify the filesystem layer is doing correct work. Thanks, Qu > >>>> - Should we run a newer kernel than our current v5.4? >>> >>> Definitely. In fact my fuzzy memory points me to some fix, but I can't >>> remember exactly which fix. >>> >>>> - Any debug you think would be useful to enable or add? >>>> >>> >>> Tree-checker, which is already enabled by default (in fact no way to >>> disable) in newer kernels. >>> >>> Thanks, >>> Qu >> >> Thank you very much, >> >> I will report here if/when we get more information on this issue. ^ permalink raw reply [flat|nested] 75+ messages in thread
[parent not found: <CAJheHN0FUe-ijMco1ZOc6iKF2zbPocOw+iiVNeTT1r-JuXOJww@mail.gmail.com>]
* Fwd: Read time tree block corruption detected [not found] <CAJheHN0FUe-ijMco1ZOc6iKF2zbPocOw+iiVNeTT1r-JuXOJww@mail.gmail.com> @ 2020-05-06 21:54 ` Tyler Richmond 2020-05-06 23:55 ` Chris Murphy 0 siblings, 1 reply; 75+ messages in thread From: Tyler Richmond @ 2020-05-06 21:54 UTC (permalink / raw) To: linux-btrfs Hello, I looked up this error and it basically says ask a developer to determine if it's a false error or not. I just started getting some slow response times, and looked at the dmesg log to find a ton of these errors. [192088.446299] BTRFS critical (device sdh): corrupt leaf: root=5 block=203510940835840 slot=4 ino=1311670, invalid inode generation: has 18446744073709551492 expect [0, 6875827] [192088.449823] BTRFS error (device sdh): block=203510940835840 read time tree block corruption detected [192088.459238] BTRFS critical (device sdh): corrupt leaf: root=5 block=203510940835840 slot=4 ino=1311670, invalid inode generation: has 18446744073709551492 expect [0, 6875827] [192088.462773] BTRFS error (device sdh): block=203510940835840 read time tree block corruption detected [192088.464711] BTRFS critical (device sdh): corrupt leaf: root=5 block=203510940835840 slot=4 ino=1311670, invalid inode generation: has 18446744073709551492 expect [0, 6875827] [192088.468457] BTRFS error (device sdh): block=203510940835840 read time tree block corruption detected btrfs device stats, however, doesn't show any errors. Is there anything I should do about this, or should I just continue using my array as normal? Thank you! ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: Read time tree block corruption detected 2020-05-06 21:54 ` Fwd: Read " Tyler Richmond @ 2020-05-06 23:55 ` Chris Murphy 2020-05-07 0:51 ` Tyler Richmond 0 siblings, 1 reply; 75+ messages in thread From: Chris Murphy @ 2020-05-06 23:55 UTC (permalink / raw) To: Tyler Richmond; +Cc: Btrfs BTRFS On Wed, May 6, 2020 at 3:54 PM Tyler Richmond <t.d.richmond@gmail.com> wrote: > > Hello, > > I looked up this error and it basically says ask a developer to > determine if it's a false error or not. I just started getting some > slow response times, and looked at the dmesg log to find a ton of > these errors. > > [192088.446299] BTRFS critical (device sdh): corrupt leaf: root=5 > block=203510940835840 slot=4 ino=1311670, invalid inode generation: > has 18446744073709551492 expect [0, 6875827] > [192088.449823] BTRFS error (device sdh): block=203510940835840 read > time tree block corruption detected > [192088.459238] BTRFS critical (device sdh): corrupt leaf: root=5 > block=203510940835840 slot=4 ino=1311670, invalid inode generation: > has 18446744073709551492 expect [0, 6875827] > [192088.462773] BTRFS error (device sdh): block=203510940835840 read > time tree block corruption detected > [192088.464711] BTRFS critical (device sdh): corrupt leaf: root=5 > block=203510940835840 slot=4 ino=1311670, invalid inode generation: > has 18446744073709551492 expect [0, 6875827] > [192088.468457] BTRFS error (device sdh): block=203510940835840 read > time tree block corruption detected > > btrfs device stats, however, doesn't show any errors. > > Is there anything I should do about this, or should I just continue > using my array as normal? What kernel version? This looks like relatively recent kernel reporting format. Can you search for inode 1311670? e.g. $ sudo btrfs insp ino -v 1311670 /mountpoint Note that each subvolume has its own set of inodes. You need to point the command to the correct subvolume. In this case it's root=5 which is the default/top-level. As long as you haven't changed the default subvolume, and you've mounted the file system without subvol or subvolid option, it should point to the correct file that's affected by this. And also maybe useful: $ sudo btrfs insp dump-t -b 203510940835840 /mountpoint -- Chris Murphy ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: Read time tree block corruption detected 2020-05-06 23:55 ` Chris Murphy @ 2020-05-07 0:51 ` Tyler Richmond 2020-05-07 1:06 ` Chris Murphy 0 siblings, 1 reply; 75+ messages in thread From: Tyler Richmond @ 2020-05-07 0:51 UTC (permalink / raw) To: Chris Murphy; +Cc: Btrfs BTRFS Kernel version is 5.4.0-28. Just upgraded to Ubuntu 20.04. btrfs-progs is version 5.6 That inode search returned an error: ERROR: ino paths ioctl: Input/output error I don't use any subvolumes. The next command also resulted in an error. Replaced the actual mountpoint in the error with /mountpoint. ERROR: not a block device or regular file: /mountpoint/ ERROR: device scan /mountpoint/: Input/output error Something definitely doesn't seem right. On Wed, May 6, 2020 at 7:55 PM Chris Murphy <lists@colorremedies.com> wrote: > > On Wed, May 6, 2020 at 3:54 PM Tyler Richmond <t.d.richmond@gmail.com> wrote: > > > > Hello, > > > > I looked up this error and it basically says ask a developer to > > determine if it's a false error or not. I just started getting some > > slow response times, and looked at the dmesg log to find a ton of > > these errors. > > > > [192088.446299] BTRFS critical (device sdh): corrupt leaf: root=5 > > block=203510940835840 slot=4 ino=1311670, invalid inode generation: > > has 18446744073709551492 expect [0, 6875827] > > [192088.449823] BTRFS error (device sdh): block=203510940835840 read > > time tree block corruption detected > > [192088.459238] BTRFS critical (device sdh): corrupt leaf: root=5 > > block=203510940835840 slot=4 ino=1311670, invalid inode generation: > > has 18446744073709551492 expect [0, 6875827] > > [192088.462773] BTRFS error (device sdh): block=203510940835840 read > > time tree block corruption detected > > [192088.464711] BTRFS critical (device sdh): corrupt leaf: root=5 > > block=203510940835840 slot=4 ino=1311670, invalid inode generation: > > has 18446744073709551492 expect [0, 6875827] > > [192088.468457] BTRFS error (device sdh): block=203510940835840 read > > time tree block corruption detected > > > > btrfs device stats, however, doesn't show any errors. > > > > Is there anything I should do about this, or should I just continue > > using my array as normal? > > What kernel version? This looks like relatively recent kernel > reporting format. Can you search for inode 1311670? e.g. > > $ sudo btrfs insp ino -v 1311670 /mountpoint > > Note that each subvolume has its own set of inodes. You need to point > the command to the correct subvolume. In this case it's root=5 which > is the default/top-level. As long as you haven't changed the default > subvolume, and you've mounted the file system without subvol or > subvolid option, it should point to the correct file that's affected > by this. And also maybe useful: > > $ sudo btrfs insp dump-t -b 203510940835840 /mountpoint > > > > -- > Chris Murphy ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: Read time tree block corruption detected 2020-05-07 0:51 ` Tyler Richmond @ 2020-05-07 1:06 ` Chris Murphy 0 siblings, 0 replies; 75+ messages in thread From: Chris Murphy @ 2020-05-07 1:06 UTC (permalink / raw) To: Tyler Richmond; +Cc: Chris Murphy, Btrfs BTRFS On Wed, May 6, 2020 at 6:51 PM Tyler Richmond <t.d.richmond@gmail.com> wrote: > > Kernel version is 5.4.0-28. Just upgraded to Ubuntu 20.04. > btrfs-progs is version 5.6 > > That inode search returned an error: > ERROR: ino paths ioctl: Input/output error > > I don't use any subvolumes. > > The next command also resulted in an error. Replaced the actual > mountpoint in the error with /mountpoint. > > ERROR: not a block device or regular file: /mountpoint/ > ERROR: device scan /mountpoint/: Input/output error > > Something definitely doesn't seem right. You need to substitute the /mountpoint with the actual mountpoint of the file system, be it / or /home or /mnt -- Chris Murphy ^ permalink raw reply [flat|nested] 75+ messages in thread
* read time tree block corruption detected @ 2020-02-12 21:58 Samir Benmendil 2020-02-13 0:26 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Samir Benmendil @ 2020-02-12 21:58 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1272 bytes --] Hello, I've been getting the following "BTRFS errors" for a while now, the wiki [0] advises to report such occurrences to this list. BTRFS critical (device sda2): corrupt leaf: root=466 block=194756837376 slot=72 ino=1359622 file_offset=475136, extent end overflow, have file offset 475136 extent num bytes 18446744073709486080 BTRFS error (device sda2): block=194756837376 read time tree block corruption detected BTRFS critical (device sda2): corrupt leaf: root=466 block=194756837376 slot=72 ino=1359622 file_offset=475136, extent end overflow, have file offset 475136 extent num bytes 18446744073709486080 BTRFS error (device sda2): block=194756837376 read time tree block corruption detected BTRFS critical (device sda2): corrupt leaf: root=466 block=194347958272 slot=131 ino=1357455 file_offset=1044480, extent end overflow, have file offset 1044480 extent num bytes 18446744073708908544 BTRFS error (device sda2): block=194347958272 read time tree block corruption detected I can reproduce these errors consistently by running `updatedb`, I suppose some tree block in one of the file it reads is corrupted. Thanks in advance for your help, Regards, Samir [0] https://btrfs.wiki.kernel.org/index.php/Tree-checker#How_to_handle_such_error [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2020-02-12 21:58 read " Samir Benmendil @ 2020-02-13 0:26 ` Qu Wenruo 2020-02-13 13:04 ` Samir Benmendil 2020-02-13 14:01 ` Qu Wenruo 0 siblings, 2 replies; 75+ messages in thread From: Qu Wenruo @ 2020-02-13 0:26 UTC (permalink / raw) To: Samir Benmendil, linux-btrfs [-- Attachment #1.1: Type: text/plain, Size: 1552 bytes --] On 2020/2/13 上午5:58, Samir Benmendil wrote: > Hello, > > I've been getting the following "BTRFS errors" for a while now, the wiki > [0] advises to report such occurrences to this list. Please provide the following dump: # btrfs ins dump-tree -b 194756837376 /dev/sda2 # btrfs ins dump-tree -b 194347958272 /dev/sda2 Thanks, Qu > > BTRFS critical (device sda2): corrupt leaf: root=466 block=194756837376 > slot=72 ino=1359622 file_offset=475136, extent end overflow, have file > offset 475136 extent num bytes 18446744073709486080 > BTRFS error (device sda2): block=194756837376 read time tree block > corruption detected > BTRFS critical (device sda2): corrupt leaf: root=466 block=194756837376 > slot=72 ino=1359622 file_offset=475136, extent end overflow, have file > offset 475136 extent num bytes 18446744073709486080 > BTRFS error (device sda2): block=194756837376 read time tree block > corruption detected > BTRFS critical (device sda2): corrupt leaf: root=466 block=194347958272 > slot=131 ino=1357455 file_offset=1044480, extent end overflow, have file > offset 1044480 extent num bytes 18446744073708908544 > BTRFS error (device sda2): block=194347958272 read time tree block > corruption detected > > I can reproduce these errors consistently by running `updatedb`, I > suppose some tree block in one of the file it reads is corrupted. > > Thanks in advance for your help, > Regards, > Samir > > [0] > https://btrfs.wiki.kernel.org/index.php/Tree-checker#How_to_handle_such_error > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2020-02-13 0:26 ` Qu Wenruo @ 2020-02-13 13:04 ` Samir Benmendil 2020-02-13 14:01 ` Qu Wenruo 1 sibling, 0 replies; 75+ messages in thread From: Samir Benmendil @ 2020-02-13 13:04 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1.1: Type: text/plain, Size: 0 bytes --] [-- Attachment #1.2: btrfs-dump-tree-194347958272.txt --] [-- Type: text/plain, Size: 40268 bytes --] btrfs-progs v5.2.2 leaf 194347958272 items 171 free space 653 generation 954719 owner 466 leaf 194347958272 flags 0x1(WRITTEN) backref revision 1 fs uuid b4947c1b-447f-4521-841e-5882003e3721 chunk uuid 7e21c972-dd41-447e-97e3-867eb86fa440 item 0 key (1357418 EXTENT_DATA 335872) itemoff 16230 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66214678528 nr 45056 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 1 key (1357418 EXTENT_DATA 466944) itemoff 16177 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66214723584 nr 45056 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 2 key (1357418 EXTENT_DATA 598016) itemoff 16124 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66214768640 nr 40960 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 3 key (1357418 EXTENT_DATA 729088) itemoff 16071 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66214809600 nr 36864 extent data offset 0 nr 106496 ram 106496 extent compression 2 (lzo) item 4 key (1357419 INODE_ITEM 0) itemoff 15911 itemsize 160 generation 952602 transid 952602 size 12489 nbytes 16384 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 2 flags 0x0(none) atime 1548109731.824211431 (2019-01-21 22:28:51) ctime 1548109731.824211431 (2019-01-21 22:28:51) mtime 1548109731.824211431 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 5 key (1357419 INODE_REF 1355322) itemoff 15891 itemsize 20 index 962 namelen 10 name: cdc_eem.ko item 6 key (1357419 EXTENT_DATA 0) itemoff 15838 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66214846464 nr 8192 extent data offset 0 nr 16384 ram 16384 extent compression 2 (lzo) item 7 key (1357420 INODE_ITEM 0) itemoff 15678 itemsize 160 generation 952602 transid 952602 size 51569 nbytes 53248 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.834211502 (2019-01-21 22:28:51) ctime 1548109731.834211502 (2019-01-21 22:28:51) mtime 1548109731.834211502 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 8 key (1357420 INODE_REF 1355322) itemoff 15659 itemsize 19 index 963 namelen 9 name: sunhme.ko item 9 key (1357420 EXTENT_DATA 0) itemoff 15606 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66214854656 nr 24576 extent data offset 0 nr 53248 ram 53248 extent compression 2 (lzo) item 10 key (1357421 INODE_ITEM 0) itemoff 15446 itemsize 160 generation 952602 transid 952602 size 29033 nbytes 32768 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.844211572 (2019-01-21 22:28:51) ctime 1548109731.844211572 (2019-01-21 22:28:51) mtime 1548109731.844211572 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 11 key (1357421 INODE_REF 1355322) itemoff 15427 itemsize 19 index 964 namelen 9 name: vringh.ko item 12 key (1357421 EXTENT_DATA 0) itemoff 15374 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66214879232 nr 12288 extent data offset 0 nr 32768 ram 32768 extent compression 2 (lzo) item 13 key (1357422 INODE_ITEM 0) itemoff 15214 itemsize 160 generation 952602 transid 952602 size 25825 nbytes 28672 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.854211646 (2019-01-21 22:28:51) ctime 1548109731.854211646 (2019-01-21 22:28:51) mtime 1548109731.854211646 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 14 key (1357422 INODE_REF 1355322) itemoff 15190 itemsize 24 index 965 namelen 14 name: wlcore_sdio.ko item 15 key (1357422 EXTENT_DATA 0) itemoff 15137 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66214891520 nr 12288 extent data offset 0 nr 28672 ram 28672 extent compression 2 (lzo) item 16 key (1357423 INODE_ITEM 0) itemoff 14977 itemsize 160 generation 952602 transid 952602 size 19273 nbytes 20480 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.864211716 (2019-01-21 22:28:51) ctime 1548109731.864211716 (2019-01-21 22:28:51) mtime 1548109731.864211716 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 17 key (1357423 INODE_REF 1355322) itemoff 14953 itemsize 24 index 966 namelen 14 name: wl1251_sdio.ko item 18 key (1357423 EXTENT_DATA 0) itemoff 14900 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66214903808 nr 8192 extent data offset 0 nr 20480 ram 20480 extent compression 2 (lzo) item 19 key (1357424 INODE_ITEM 0) itemoff 14740 itemsize 160 generation 952602 transid 952602 size 94969 nbytes 98304 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.874211787 (2019-01-21 22:28:51) ctime 1548109731.884211857 (2019-01-21 22:28:51) mtime 1548109731.884211857 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 20 key (1357424 INODE_REF 1355322) itemoff 14715 itemsize 25 index 967 namelen 15 name: at76c50x-usb.ko item 21 key (1357424 EXTENT_DATA 0) itemoff 14662 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66214912000 nr 40960 extent data offset 0 nr 98304 ram 98304 extent compression 2 (lzo) item 22 key (1357425 INODE_ITEM 0) itemoff 14502 itemsize 160 generation 952602 transid 952602 size 192233 nbytes 192512 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.894211931 (2019-01-21 22:28:51) ctime 1548109731.904212001 (2019-01-21 22:28:51) mtime 1548109731.904212001 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 23 key (1357425 INODE_REF 1355322) itemoff 14480 itemsize 22 index 968 namelen 12 name: ath9k_htc.ko item 24 key (1357425 EXTENT_DATA 0) itemoff 14427 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66214952960 nr 57344 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 25 key (1357425 EXTENT_DATA 131072) itemoff 14374 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215010304 nr 20480 extent data offset 0 nr 61440 ram 61440 extent compression 2 (lzo) item 26 key (1357426 INODE_ITEM 0) itemoff 14214 itemsize 160 generation 952602 transid 952602 size 7633 nbytes 8192 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.914212071 (2019-01-21 22:28:51) ctime 1548109731.914212071 (2019-01-21 22:28:51) mtime 1548109731.914212071 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 27 key (1357426 INODE_REF 1355322) itemoff 14181 itemsize 33 index 969 namelen 23 name: team_mode_roundrobin.ko item 28 key (1357426 EXTENT_DATA 0) itemoff 14128 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215030784 nr 4096 extent data offset 0 nr 8192 ram 8192 extent compression 2 (lzo) item 29 key (1357427 INODE_ITEM 0) itemoff 13968 itemsize 160 generation 952602 transid 952602 size 9633 nbytes 12288 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.924212145 (2019-01-21 22:28:51) ctime 1548109731.924212145 (2019-01-21 22:28:51) mtime 1548109731.924212145 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 30 key (1357427 INODE_REF 1355322) itemoff 13948 itemsize 20 index 970 namelen 10 name: dp83867.ko item 31 key (1357427 EXTENT_DATA 0) itemoff 13895 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215034880 nr 8192 extent data offset 0 nr 12288 ram 12288 extent compression 2 (lzo) item 32 key (1357428 INODE_ITEM 0) itemoff 13735 itemsize 160 generation 952602 transid 952602 size 5377 nbytes 8192 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.934212215 (2019-01-21 22:28:51) ctime 1548109731.934212215 (2019-01-21 22:28:51) mtime 1548109731.934212215 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 33 key (1357428 INODE_REF 1355322) itemoff 13718 itemsize 17 index 971 namelen 7 name: crc8.ko item 34 key (1357428 EXTENT_DATA 0) itemoff 13665 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215043072 nr 4096 extent data offset 0 nr 8192 ram 8192 extent compression 2 (lzo) item 35 key (1357429 INODE_ITEM 0) itemoff 13505 itemsize 160 generation 952602 transid 952602 size 299289 nbytes 278528 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.944212286 (2019-01-21 22:28:51) ctime 1548109731.964212430 (2019-01-21 22:28:51) mtime 1548109731.964212430 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 36 key (1357429 INODE_REF 1355322) itemoff 13483 itemsize 22 index 972 namelen 12 name: mv88e6xxx.ko item 37 key (1357429 EXTENT_DATA 0) itemoff 13430 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215047168 nr 32768 extent data offset 0 nr 73728 ram 73728 extent compression 2 (lzo) item 38 key (1357429 EXTENT_DATA 73728) itemoff 13377 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 16384 ram 16384 extent compression 0 (none) item 39 key (1357429 EXTENT_DATA 90112) itemoff 13324 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215079936 nr 4096 extent data offset 0 nr 8192 ram 8192 extent compression 2 (lzo) item 40 key (1357429 EXTENT_DATA 98304) itemoff 13271 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 8192 ram 8192 extent compression 0 (none) item 41 key (1357429 EXTENT_DATA 106496) itemoff 13218 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215084032 nr 45056 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 42 key (1357429 EXTENT_DATA 237568) itemoff 13165 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215129088 nr 20480 extent data offset 0 nr 65536 ram 65536 extent compression 2 (lzo) item 43 key (1357430 INODE_ITEM 0) itemoff 13005 itemsize 160 generation 952602 transid 952602 size 13905 nbytes 16384 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.964212430 (2019-01-21 22:28:51) ctime 1548109731.974212500 (2019-01-21 22:28:51) mtime 1548109731.974212500 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 44 key (1357430 INODE_REF 1355322) itemoff 12981 itemsize 24 index 973 namelen 14 name: orinoco_tmd.ko item 45 key (1357430 EXTENT_DATA 0) itemoff 12928 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215149568 nr 8192 extent data offset 0 nr 16384 ram 16384 extent compression 2 (lzo) item 46 key (1357431 INODE_ITEM 0) itemoff 12768 itemsize 160 generation 952602 transid 952602 size 98073 nbytes 98304 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.974212500 (2019-01-21 22:28:51) ctime 1548109731.984212571 (2019-01-21 22:28:51) mtime 1548109731.984212571 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 47 key (1357431 INODE_REF 1355322) itemoff 12745 itemsize 23 index 974 namelen 13 name: virtio_net.ko item 48 key (1357431 EXTENT_DATA 0) itemoff 12692 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215157760 nr 45056 extent data offset 0 nr 98304 ram 98304 extent compression 2 (lzo) item 49 key (1357432 INODE_ITEM 0) itemoff 12532 itemsize 160 generation 952602 transid 952602 size 12929 nbytes 16384 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109731.994212641 (2019-01-21 22:28:51) ctime 1548109731.994212641 (2019-01-21 22:28:51) mtime 1548109731.994212641 (2019-01-21 22:28:51) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 50 key (1357432 INODE_REF 1355322) itemoff 12512 itemsize 20 index 975 namelen 10 name: int51x1.ko item 51 key (1357432 EXTENT_DATA 0) itemoff 12459 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215202816 nr 8192 extent data offset 0 nr 16384 ram 16384 extent compression 2 (lzo) item 52 key (1357433 INODE_ITEM 0) itemoff 12299 itemsize 160 generation 952602 transid 952602 size 7425 nbytes 8192 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.4212715 (2019-01-21 22:28:52) ctime 1548109732.4212715 (2019-01-21 22:28:52) mtime 1548109732.4212715 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 53 key (1357433 INODE_REF 1355322) itemoff 12281 itemsize 18 index 976 namelen 8 name: qsemi.ko item 54 key (1357433 EXTENT_DATA 0) itemoff 12228 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215211008 nr 4096 extent data offset 0 nr 8192 ram 8192 extent compression 2 (lzo) item 55 key (1357434 INODE_ITEM 0) itemoff 12068 itemsize 160 generation 952602 transid 952602 size 60105 nbytes 61440 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.14212785 (2019-01-21 22:28:52) ctime 1548109732.14212785 (2019-01-21 22:28:52) mtime 1548109732.14212785 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 56 key (1357434 INODE_REF 1355322) itemoff 12044 itemsize 24 index 977 namelen 14 name: orinoco_usb.ko item 57 key (1357434 EXTENT_DATA 0) itemoff 11991 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215215104 nr 24576 extent data offset 0 nr 61440 ram 61440 extent compression 2 (lzo) item 58 key (1357435 INODE_ITEM 0) itemoff 11831 itemsize 160 generation 952602 transid 952602 size 12161 nbytes 12288 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.24212855 (2019-01-21 22:28:52) ctime 1548109732.34212929 (2019-01-21 22:28:52) mtime 1548109732.34212929 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 59 key (1357435 INODE_REF 1355322) itemoff 11810 itemsize 21 index 978 namelen 11 name: aquantia.ko item 60 key (1357435 EXTENT_DATA 0) itemoff 11757 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215239680 nr 4096 extent data offset 0 nr 12288 ram 12288 extent compression 2 (lzo) item 61 key (1357436 INODE_ITEM 0) itemoff 11597 itemsize 160 generation 952602 transid 952602 size 75089 nbytes 77824 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.34212929 (2019-01-21 22:28:52) ctime 1548109732.44212999 (2019-01-21 22:28:52) mtime 1548109732.44212999 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 62 key (1357436 INODE_REF 1355322) itemoff 11578 itemsize 19 index 979 namelen 9 name: macsec.ko item 63 key (1357436 EXTENT_DATA 0) itemoff 11525 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215243776 nr 32768 extent data offset 0 nr 77824 ram 77824 extent compression 2 (lzo) item 64 key (1357437 INODE_ITEM 0) itemoff 11365 itemsize 160 generation 952602 transid 952602 size 77249 nbytes 77824 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.54213070 (2019-01-21 22:28:52) ctime 1548109732.64213143 (2019-01-21 22:28:52) mtime 1548109732.64213143 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 65 key (1357437 INODE_REF 1355322) itemoff 11349 itemsize 16 index 980 namelen 6 name: jme.ko item 66 key (1357437 EXTENT_DATA 0) itemoff 11296 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215276544 nr 36864 extent data offset 0 nr 77824 ram 77824 extent compression 2 (lzo) item 67 key (1357438 INODE_ITEM 0) itemoff 11136 itemsize 160 generation 952602 transid 952602 size 28729 nbytes 32768 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.74213214 (2019-01-21 22:28:52) ctime 1548109732.74213214 (2019-01-21 22:28:52) mtime 1548109732.74213214 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 68 key (1357438 INODE_REF 1355322) itemoff 11114 itemsize 22 index 981 namelen 12 name: xircom_cb.ko item 69 key (1357438 EXTENT_DATA 0) itemoff 11061 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215313408 nr 12288 extent data offset 0 nr 32768 ram 32768 extent compression 2 (lzo) item 70 key (1357439 INODE_ITEM 0) itemoff 10901 itemsize 160 generation 952602 transid 952602 size 7761 nbytes 8192 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.84213284 (2019-01-21 22:28:52) ctime 1548109732.84213284 (2019-01-21 22:28:52) mtime 1548109732.84213284 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 71 key (1357439 INODE_REF 1355322) itemoff 10872 itemsize 29 index 982 namelen 19 name: team_mode_random.ko item 72 key (1357439 EXTENT_DATA 0) itemoff 10819 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215325696 nr 4096 extent data offset 0 nr 8192 ram 8192 extent compression 2 (lzo) item 73 key (1357440 INODE_ITEM 0) itemoff 10659 itemsize 160 generation 952602 transid 952602 size 10513 nbytes 12288 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.94213357 (2019-01-21 22:28:52) ctime 1548109732.94213357 (2019-01-21 22:28:52) mtime 1548109732.94213357 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 74 key (1357440 INODE_REF 1355322) itemoff 10637 itemsize 22 index 983 namelen 12 name: mdio-gpio.ko item 75 key (1357440 EXTENT_DATA 0) itemoff 10584 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215329792 nr 8192 extent data offset 0 nr 12288 ram 12288 extent compression 2 (lzo) item 76 key (1357441 INODE_ITEM 0) itemoff 10424 itemsize 160 generation 952602 transid 952602 size 23697 nbytes 24576 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.94213357 (2019-01-21 22:28:52) ctime 1548109732.104213428 (2019-01-21 22:28:52) mtime 1548109732.104213428 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 77 key (1357441 INODE_REF 1355322) itemoff 10402 itemsize 22 index 984 namelen 12 name: rt2800pci.ko item 78 key (1357441 EXTENT_DATA 0) itemoff 10349 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215337984 nr 12288 extent data offset 0 nr 24576 ram 24576 extent compression 2 (lzo) item 79 key (1357442 INODE_ITEM 0) itemoff 10189 itemsize 160 generation 952602 transid 952602 size 12953 nbytes 16384 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.104213428 (2019-01-21 22:28:52) ctime 1548109732.114213498 (2019-01-21 22:28:52) mtime 1548109732.114213498 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 80 key (1357442 INODE_REF 1355322) itemoff 10166 itemsize 23 index 985 namelen 13 name: rt2x00mmio.ko item 81 key (1357442 EXTENT_DATA 0) itemoff 10113 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215350272 nr 8192 extent data offset 0 nr 16384 ram 16384 extent compression 2 (lzo) item 82 key (1357443 INODE_ITEM 0) itemoff 9953 itemsize 160 generation 952602 transid 952602 size 9593 nbytes 12288 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.114213498 (2019-01-21 22:28:52) ctime 1548109732.124213571 (2019-01-21 22:28:52) mtime 1548109732.124213571 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 83 key (1357443 INODE_REF 1355322) itemoff 9935 itemsize 18 index 986 namelen 8 name: pppox.ko item 84 key (1357443 EXTENT_DATA 0) itemoff 9882 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215358464 nr 8192 extent data offset 0 nr 12288 ram 12288 extent compression 2 (lzo) item 85 key (1357444 INODE_ITEM 0) itemoff 9722 itemsize 160 generation 952602 transid 952602 size 24265 nbytes 24576 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.124213571 (2019-01-21 22:28:52) ctime 1548109732.134213642 (2019-01-21 22:28:52) mtime 1548109732.134213642 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 86 key (1357444 INODE_REF 1355322) itemoff 9700 itemsize 22 index 987 namelen 12 name: ppp_async.ko item 87 key (1357444 EXTENT_DATA 0) itemoff 9647 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215366656 nr 12288 extent data offset 0 nr 24576 ram 24576 extent compression 2 (lzo) item 88 key (1357445 INODE_ITEM 0) itemoff 9487 itemsize 160 generation 952602 transid 952602 size 102257 nbytes 102400 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.134213642 (2019-01-21 22:28:52) ctime 1548109732.144213712 (2019-01-21 22:28:52) mtime 1548109732.144213712 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 89 key (1357445 INODE_REF 1355322) itemoff 9467 itemsize 20 index 988 namelen 10 name: lan743x.ko item 90 key (1357445 EXTENT_DATA 0) itemoff 9414 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215378944 nr 45056 extent data offset 0 nr 102400 ram 102400 extent compression 2 (lzo) item 91 key (1357446 INODE_ITEM 0) itemoff 9254 itemsize 160 generation 952602 transid 952602 size 288177 nbytes 282624 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.154213785 (2019-01-21 22:28:52) ctime 1548109732.184213999 (2019-01-21 22:28:52) mtime 1548109732.184213999 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 92 key (1357446 INODE_REF 1355322) itemoff 9235 itemsize 19 index 989 namelen 9 name: be2net.ko item 93 key (1357446 EXTENT_DATA 0) itemoff 9182 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215424000 nr 73728 extent data offset 0 nr 106496 ram 106496 extent compression 2 (lzo) item 94 key (1357446 EXTENT_DATA 106496) itemoff 9129 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 8192 ram 8192 extent compression 0 (none) item 95 key (1357446 EXTENT_DATA 114688) itemoff 9076 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215497728 nr 45056 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 96 key (1357446 EXTENT_DATA 245760) itemoff 9023 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215542784 nr 16384 extent data offset 0 nr 45056 ram 45056 extent compression 2 (lzo) item 97 key (1357447 INODE_ITEM 0) itemoff 8863 itemsize 160 generation 952602 transid 952602 size 311681 nbytes 315392 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.184213999 (2019-01-21 22:28:52) ctime 1548109732.214214214 (2019-01-21 22:28:52) mtime 1548109732.214214214 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 98 key (1357447 INODE_REF 1355322) itemoff 8847 itemsize 16 index 990 namelen 6 name: bna.ko item 99 key (1357447 EXTENT_DATA 0) itemoff 8794 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215559168 nr 69632 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 100 key (1357447 EXTENT_DATA 131072) itemoff 8741 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215628800 nr 49152 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 101 key (1357447 EXTENT_DATA 262144) itemoff 8688 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215677952 nr 16384 extent data offset 0 nr 53248 ram 53248 extent compression 2 (lzo) item 102 key (1357448 INODE_ITEM 0) itemoff 8528 itemsize 160 generation 952602 transid 952602 size 15081 nbytes 16384 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.224214284 (2019-01-21 22:28:52) ctime 1548109732.234214355 (2019-01-21 22:28:52) mtime 1548109732.234214355 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 103 key (1357448 INODE_REF 1355322) itemoff 8507 itemsize 21 index 991 namelen 11 name: b53_mmap.ko item 104 key (1357448 EXTENT_DATA 0) itemoff 8454 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215694336 nr 8192 extent data offset 0 nr 16384 ram 16384 extent compression 2 (lzo) item 105 key (1357449 INODE_ITEM 0) itemoff 8294 itemsize 160 generation 952602 transid 952602 size 18529 nbytes 20480 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.234214355 (2019-01-21 22:28:52) ctime 1548109732.244214428 (2019-01-21 22:28:52) mtime 1548109732.244214428 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 106 key (1357449 INODE_REF 1355322) itemoff 8269 itemsize 25 index 992 namelen 15 name: i2c-algo-bit.ko item 107 key (1357449 EXTENT_DATA 0) itemoff 8216 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215702528 nr 8192 extent data offset 0 nr 20480 ram 20480 extent compression 2 (lzo) item 108 key (1357450 INODE_ITEM 0) itemoff 8056 itemsize 160 generation 952602 transid 952602 size 63785 nbytes 65536 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.244214428 (2019-01-21 22:28:52) ctime 1548109732.254214499 (2019-01-21 22:28:52) mtime 1548109732.254214499 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 109 key (1357450 INODE_REF 1355322) itemoff 8034 itemsize 22 index 993 namelen 12 name: at86rf230.ko item 110 key (1357450 EXTENT_DATA 0) itemoff 7981 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215710720 nr 24576 extent data offset 0 nr 65536 ram 65536 extent compression 2 (lzo) item 111 key (1357451 INODE_ITEM 0) itemoff 7821 itemsize 160 generation 952602 transid 952602 size 44881 nbytes 45056 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.264214569 (2019-01-21 22:28:52) ctime 1548109732.264214569 (2019-01-21 22:28:52) mtime 1548109732.264214569 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 112 key (1357451 INODE_REF 1355322) itemoff 7802 itemsize 19 index 994 namelen 9 name: tehuti.ko item 113 key (1357451 EXTENT_DATA 0) itemoff 7749 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215735296 nr 24576 extent data offset 0 nr 45056 ram 45056 extent compression 2 (lzo) item 114 key (1357452 INODE_ITEM 0) itemoff 7589 itemsize 160 generation 952602 transid 952602 size 14489 nbytes 16384 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.274214642 (2019-01-21 22:28:52) ctime 1548109732.274214642 (2019-01-21 22:28:52) mtime 1548109732.274214642 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 115 key (1357452 INODE_REF 1355322) itemoff 7571 itemsize 18 index 995 namelen 8 name: vxcan.ko item 116 key (1357452 EXTENT_DATA 0) itemoff 7518 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215759872 nr 8192 extent data offset 0 nr 16384 ram 16384 extent compression 2 (lzo) item 117 key (1357453 INODE_ITEM 0) itemoff 7358 itemsize 160 generation 952602 transid 952602 size 71553 nbytes 73728 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.284214713 (2019-01-21 22:28:52) ctime 1548109732.284214713 (2019-01-21 22:28:52) mtime 1548109732.284214713 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 118 key (1357453 INODE_REF 1355322) itemoff 7336 itemsize 22 index 996 namelen 12 name: rt2800usb.ko item 119 key (1357453 EXTENT_DATA 0) itemoff 7283 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215768064 nr 24576 extent data offset 0 nr 73728 ram 73728 extent compression 2 (lzo) item 120 key (1357454 INODE_ITEM 0) itemoff 7123 itemsize 160 generation 952602 transid 952602 size 73681 nbytes 73728 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.294214783 (2019-01-21 22:28:52) ctime 1548109732.304214856 (2019-01-21 22:28:52) mtime 1548109732.304214856 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 121 key (1357454 INODE_REF 1355322) itemoff 7105 itemsize 18 index 997 namelen 8 name: atmel.ko item 122 key (1357454 EXTENT_DATA 0) itemoff 7052 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215792640 nr 36864 extent data offset 0 nr 73728 ram 73728 extent compression 2 (lzo) item 123 key (1357455 INODE_ITEM 0) itemoff 6892 itemsize 160 generation 952602 transid 952602 size 1040537 nbytes 507904 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548109732.314214927 (2019-01-21 22:28:52) ctime 1548109732.394215499 (2019-01-21 22:28:52) mtime 1548109732.394215499 (2019-01-21 22:28:52) otime 16107652860835337076.2725157735 (-1108403787-01-21 01:43:05) item 124 key (1357455 INODE_REF 1355322) itemoff 6868 itemsize 24 index 998 namelen 14 name: ath10k_core.ko item 125 key (1357455 EXTENT_DATA 0) itemoff 6815 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215829504 nr 86016 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 126 key (1357455 EXTENT_DATA 131072) itemoff 6762 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215915520 nr 77824 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 127 key (1357455 EXTENT_DATA 262144) itemoff 6709 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66215993344 nr 45056 extent data offset 0 nr 114688 ram 114688 extent compression 2 (lzo) item 128 key (1357455 EXTENT_DATA 376832) itemoff 6656 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 24576 ram 667648 extent compression 0 (none) item 129 key (1357455 EXTENT_DATA 401408) itemoff 6603 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 66216038400 nr 36864 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 130 key (1357455 EXTENT_DATA 532480) itemoff 6550 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 155648 nr 512000 ram 667648 extent compression 0 (none) item 131 key (1357455 EXTENT_DATA 1044480) itemoff 6497 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 18446744073708908544 ram 18446744073708908544 extent compression 0 (none) item 132 key (1357456 INODE_ITEM 0) itemoff 6337 itemsize 160 generation 954716 transid 954716 size 8 nbytes 0 block group 0 mode 40700 links 1 uid 0 gid 0 rdev 0 sequence 1 flags 0x0(none) atime 1548184106.645477005 (2019-01-22 19:08:26) ctime 1548184106.665476628 (2019-01-22 19:08:26) mtime 1548184106.665476628 (2019-01-22 19:08:26) otime 1548184106.645477005 (2019-01-22 19:08:26) item 133 key (1357456 INODE_REF 1707) itemoff 6310 itemsize 27 index 6 namelen 17 name: mkinitcpio.c5mBu2 item 134 key (1357456 DIR_ITEM 4214885080) itemoff 6276 itemsize 34 location key (1357457 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 4 name: root item 135 key (1357456 DIR_INDEX 2) itemoff 6242 itemsize 34 location key (1357457 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 4 name: root item 136 key (1357457 INODE_ITEM 0) itemoff 6082 itemsize 160 generation 954716 transid 954719 size 190 nbytes 0 block group 0 mode 40755 links 1 uid 0 gid 0 rdev 0 sequence 19 flags 0x0(none) atime 1548184106.665476628 (2019-01-22 19:08:26) ctime 1548185863.337550430 (2019-01-22 19:37:43) mtime 1548185863.337550430 (2019-01-22 19:37:43) otime 1548184106.665476628 (2019-01-22 19:08:26) item 137 key (1357457 INODE_REF 1357456) itemoff 6068 itemsize 14 index 2 namelen 4 name: root item 138 key (1357457 DIR_ITEM 145260132) itemoff 6035 itemsize 33 location key (1357461 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: dev item 139 key (1357457 DIR_ITEM 217684952) itemoff 6002 itemsize 33 location key (1357462 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: run item 140 key (1357457 DIR_ITEM 533965061) itemoff 5958 itemsize 44 location key (1357649 INODE_ITEM 0) type FILE transid 954716 data_len 0 name_len 14 name: init_functions item 141 key (1357457 DIR_ITEM 883143349) itemoff 5925 itemsize 33 location key (1357470 INODE_ITEM 0) type SYMLINK transid 954716 data_len 0 name_len 3 name: lib item 142 key (1357457 DIR_ITEM 935466660) itemoff 5891 itemsize 34 location key (1357474 INODE_ITEM 0) type SYMLINK transid 954716 data_len 0 name_len 4 name: sbin item 143 key (1357457 DIR_ITEM 1253479030) itemoff 5857 itemsize 34 location key (1357650 INODE_ITEM 0) type FILE transid 954716 data_len 0 name_len 4 name: init item 144 key (1357457 DIR_ITEM 1289463356) itemoff 5824 itemsize 33 location key (1357466 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: usr item 145 key (1357457 DIR_ITEM 1802746451) itemoff 5789 itemsize 35 location key (1357479 INODE_ITEM 0) type SYMLINK transid 954716 data_len 0 name_len 5 name: lib64 item 146 key (1357457 DIR_ITEM 2063292483) itemoff 5748 itemsize 41 location key (1358618 INODE_ITEM 0) type FILE transid 954719 data_len 0 name_len 11 name: buildconfig item 147 key (1357457 DIR_ITEM 2415965623) itemoff 5715 itemsize 33 location key (1357473 INODE_ITEM 0) type SYMLINK transid 954716 data_len 0 name_len 3 name: bin item 148 key (1357457 DIR_ITEM 2753648287) itemoff 5682 itemsize 33 location key (1357465 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: etc item 149 key (1357457 DIR_ITEM 2879401604) itemoff 5645 itemsize 37 location key (1357480 INODE_ITEM 0) type FILE transid 954716 data_len 0 name_len 7 name: VERSION item 150 key (1357457 DIR_ITEM 3145042590) itemoff 5609 itemsize 36 location key (1358617 INODE_ITEM 0) type FILE transid 954719 data_len 0 name_len 6 name: config item 151 key (1357457 DIR_ITEM 3195381536) itemoff 5576 itemsize 33 location key (1357464 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: var item 152 key (1357457 DIR_ITEM 3284084670) itemoff 5543 itemsize 33 location key (1357463 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: tmp item 153 key (1357457 DIR_ITEM 3578798206) itemoff 5510 itemsize 33 location key (1357460 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: sys item 154 key (1357457 DIR_ITEM 3655494385) itemoff 5475 itemsize 35 location key (1357685 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 5 name: hooks item 155 key (1357457 DIR_ITEM 3750368475) itemoff 5437 itemsize 38 location key (1357458 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 8 name: new_root item 156 key (1357457 DIR_ITEM 4087742454) itemoff 5403 itemsize 34 location key (1357459 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 4 name: proc item 157 key (1357457 DIR_INDEX 2) itemoff 5365 itemsize 38 location key (1357458 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 8 name: new_root item 158 key (1357457 DIR_INDEX 3) itemoff 5331 itemsize 34 location key (1357459 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 4 name: proc item 159 key (1357457 DIR_INDEX 4) itemoff 5298 itemsize 33 location key (1357460 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: sys item 160 key (1357457 DIR_INDEX 5) itemoff 5265 itemsize 33 location key (1357461 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: dev item 161 key (1357457 DIR_INDEX 6) itemoff 5232 itemsize 33 location key (1357462 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: run item 162 key (1357457 DIR_INDEX 7) itemoff 5199 itemsize 33 location key (1357463 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: tmp item 163 key (1357457 DIR_INDEX 8) itemoff 5166 itemsize 33 location key (1357464 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: var item 164 key (1357457 DIR_INDEX 9) itemoff 5133 itemsize 33 location key (1357465 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: etc item 165 key (1357457 DIR_INDEX 10) itemoff 5100 itemsize 33 location key (1357466 INODE_ITEM 0) type DIR transid 954716 data_len 0 name_len 3 name: usr item 166 key (1357457 DIR_INDEX 11) itemoff 5067 itemsize 33 location key (1357470 INODE_ITEM 0) type SYMLINK transid 954716 data_len 0 name_len 3 name: lib item 167 key (1357457 DIR_INDEX 12) itemoff 5034 itemsize 33 location key (1357473 INODE_ITEM 0) type SYMLINK transid 954716 data_len 0 name_len 3 name: bin item 168 key (1357457 DIR_INDEX 13) itemoff 5000 itemsize 34 location key (1357474 INODE_ITEM 0) type SYMLINK transid 954716 data_len 0 name_len 4 name: sbin item 169 key (1357457 DIR_INDEX 14) itemoff 4965 itemsize 35 location key (1357479 INODE_ITEM 0) type SYMLINK transid 954716 data_len 0 name_len 5 name: lib64 item 170 key (1357457 DIR_INDEX 15) itemoff 4928 itemsize 37 location key (1357480 INODE_ITEM 0) type FILE transid 954716 data_len 0 name_len 7 name: VERSION [-- Attachment #1.3: btrfs-dump-tree-194756837376.txt --] [-- Type: text/plain, Size: 33616 bytes --] btrfs-progs v5.2.2 leaf 194756837376 items 142 free space 2196 generation 1635349 owner 466 leaf 194756837376 flags 0x1(WRITTEN) backref revision 1 fs uuid b4947c1b-447f-4521-841e-5882003e3721 chunk uuid 7e21c972-dd41-447e-97e3-867eb86fa440 item 0 key (1359605 EXTENT_DATA 0) itemoff 16230 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66847625216 nr 12288 extent data offset 0 nr 24576 ram 24576 extent compression 2 (lzo) item 1 key (1359606 INODE_ITEM 0) itemoff 16070 itemsize 160 generation 954719 transid 954719 size 102257 nbytes 102400 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184228.843170401 (2019-01-22 19:10:28) ctime 1548184228.853170212 (2019-01-22 19:10:28) mtime 1548184228.853170212 (2019-01-22 19:10:28) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 2 key (1359606 INODE_REF 1357483) itemoff 16050 itemsize 20 index 988 namelen 10 name: lan743x.ko item 3 key (1359606 EXTENT_DATA 0) itemoff 15997 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66847637504 nr 45056 extent data offset 0 nr 102400 ram 102400 extent compression 2 (lzo) item 4 key (1359607 INODE_ITEM 0) itemoff 15837 itemsize 160 generation 954719 transid 954719 size 288177 nbytes 282624 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 2 flags 0x0(none) atime 1548184228.863170023 (2019-01-22 19:10:28) ctime 1548184228.883169645 (2019-01-22 19:10:28) mtime 1548184228.883169645 (2019-01-22 19:10:28) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 5 key (1359607 INODE_REF 1357483) itemoff 15818 itemsize 19 index 989 namelen 9 name: be2net.ko item 6 key (1359607 EXTENT_DATA 0) itemoff 15765 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66847682560 nr 73728 extent data offset 0 nr 106496 ram 106496 extent compression 2 (lzo) item 7 key (1359607 EXTENT_DATA 106496) itemoff 15712 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 8192 ram 8192 extent compression 0 (none) item 8 key (1359607 EXTENT_DATA 114688) itemoff 15659 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66847756288 nr 45056 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 9 key (1359607 EXTENT_DATA 245760) itemoff 15606 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66847801344 nr 16384 extent data offset 0 nr 45056 ram 45056 extent compression 2 (lzo) item 10 key (1359608 INODE_ITEM 0) itemoff 15446 itemsize 160 generation 954719 transid 954719 size 311681 nbytes 315392 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 2 flags 0x0(none) atime 1548184228.893169459 (2019-01-22 19:10:28) ctime 1548184228.923168892 (2019-01-22 19:10:28) mtime 1548184228.923168892 (2019-01-22 19:10:28) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 11 key (1359608 INODE_REF 1357483) itemoff 15430 itemsize 16 index 990 namelen 6 name: bna.ko item 12 key (1359608 EXTENT_DATA 0) itemoff 15377 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66847817728 nr 69632 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 13 key (1359608 EXTENT_DATA 131072) itemoff 15324 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66847887360 nr 49152 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 14 key (1359608 EXTENT_DATA 262144) itemoff 15271 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66847936512 nr 16384 extent data offset 0 nr 53248 ram 53248 extent compression 2 (lzo) item 15 key (1359609 INODE_ITEM 0) itemoff 15111 itemsize 160 generation 954719 transid 954719 size 15081 nbytes 16384 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184228.933168703 (2019-01-22 19:10:28) ctime 1548184228.933168703 (2019-01-22 19:10:28) mtime 1548184228.933168703 (2019-01-22 19:10:28) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 16 key (1359609 INODE_REF 1357483) itemoff 15090 itemsize 21 index 991 namelen 11 name: b53_mmap.ko item 17 key (1359609 EXTENT_DATA 0) itemoff 15037 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66847952896 nr 8192 extent data offset 0 nr 16384 ram 16384 extent compression 2 (lzo) item 18 key (1359610 INODE_ITEM 0) itemoff 14877 itemsize 160 generation 954719 transid 954719 size 18529 nbytes 20480 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184228.933168703 (2019-01-22 19:10:28) ctime 1548184228.943168516 (2019-01-22 19:10:28) mtime 1548184228.943168516 (2019-01-22 19:10:28) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 19 key (1359610 INODE_REF 1357483) itemoff 14852 itemsize 25 index 992 namelen 15 name: i2c-algo-bit.ko item 20 key (1359610 EXTENT_DATA 0) itemoff 14799 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66847961088 nr 8192 extent data offset 0 nr 20480 ram 20480 extent compression 2 (lzo) item 21 key (1359611 INODE_ITEM 0) itemoff 14639 itemsize 160 generation 954719 transid 954719 size 63785 nbytes 65536 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184228.943168516 (2019-01-22 19:10:28) ctime 1548184228.953168328 (2019-01-22 19:10:28) mtime 1548184228.953168328 (2019-01-22 19:10:28) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 22 key (1359611 INODE_REF 1357483) itemoff 14617 itemsize 22 index 993 namelen 12 name: at86rf230.ko item 23 key (1359611 EXTENT_DATA 0) itemoff 14564 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66847969280 nr 24576 extent data offset 0 nr 65536 ram 65536 extent compression 2 (lzo) item 24 key (1359612 INODE_ITEM 0) itemoff 14404 itemsize 160 generation 954719 transid 954719 size 44881 nbytes 45056 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184228.963168139 (2019-01-22 19:10:28) ctime 1548184228.963168139 (2019-01-22 19:10:28) mtime 1548184228.963168139 (2019-01-22 19:10:28) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 25 key (1359612 INODE_REF 1357483) itemoff 14385 itemsize 19 index 994 namelen 9 name: tehuti.ko item 26 key (1359612 EXTENT_DATA 0) itemoff 14332 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66847993856 nr 24576 extent data offset 0 nr 45056 ram 45056 extent compression 2 (lzo) item 27 key (1359613 INODE_ITEM 0) itemoff 14172 itemsize 160 generation 954719 transid 954719 size 14489 nbytes 16384 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184228.973167950 (2019-01-22 19:10:28) ctime 1548184228.973167950 (2019-01-22 19:10:28) mtime 1548184228.973167950 (2019-01-22 19:10:28) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 28 key (1359613 INODE_REF 1357483) itemoff 14154 itemsize 18 index 995 namelen 8 name: vxcan.ko item 29 key (1359613 EXTENT_DATA 0) itemoff 14101 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848018432 nr 8192 extent data offset 0 nr 16384 ram 16384 extent compression 2 (lzo) item 30 key (1359614 INODE_ITEM 0) itemoff 13941 itemsize 160 generation 954719 transid 954719 size 71553 nbytes 73728 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184228.983167760 (2019-01-22 19:10:28) ctime 1548184228.983167760 (2019-01-22 19:10:28) mtime 1548184228.983167760 (2019-01-22 19:10:28) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 31 key (1359614 INODE_REF 1357483) itemoff 13919 itemsize 22 index 996 namelen 12 name: rt2800usb.ko item 32 key (1359614 EXTENT_DATA 0) itemoff 13866 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848026624 nr 24576 extent data offset 0 nr 73728 ram 73728 extent compression 2 (lzo) item 33 key (1359615 INODE_ITEM 0) itemoff 13706 itemsize 160 generation 954719 transid 954719 size 73681 nbytes 73728 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184228.993167574 (2019-01-22 19:10:28) ctime 1548184229.3167385 (2019-01-22 19:10:29) mtime 1548184229.3167385 (2019-01-22 19:10:29) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 34 key (1359615 INODE_REF 1357483) itemoff 13688 itemsize 18 index 997 namelen 8 name: atmel.ko item 35 key (1359615 EXTENT_DATA 0) itemoff 13635 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848051200 nr 36864 extent data offset 0 nr 73728 ram 73728 extent compression 2 (lzo) item 36 key (1359616 INODE_ITEM 0) itemoff 13475 itemsize 160 generation 954719 transid 954719 size 1040537 nbytes 1019904 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184229.13167197 (2019-01-22 19:10:29) ctime 1548184229.83165879 (2019-01-22 19:10:29) mtime 1548184229.83165879 (2019-01-22 19:10:29) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 37 key (1359616 INODE_REF 1357483) itemoff 13451 itemsize 24 index 998 namelen 14 name: ath10k_core.ko item 38 key (1359616 EXTENT_DATA 0) itemoff 13398 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848088064 nr 86016 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 39 key (1359616 EXTENT_DATA 131072) itemoff 13345 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848174080 nr 77824 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 40 key (1359616 EXTENT_DATA 262144) itemoff 13292 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848251904 nr 45056 extent data offset 0 nr 114688 ram 114688 extent compression 2 (lzo) item 41 key (1359616 EXTENT_DATA 376832) itemoff 13239 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 24576 ram 24576 extent compression 0 (none) item 42 key (1359616 EXTENT_DATA 401408) itemoff 13186 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848296960 nr 36864 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 43 key (1359616 EXTENT_DATA 532480) itemoff 13133 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848333824 nr 49152 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 44 key (1359616 EXTENT_DATA 663552) itemoff 13080 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848382976 nr 45056 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 45 key (1359616 EXTENT_DATA 794624) itemoff 13027 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848428032 nr 40960 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 46 key (1359616 EXTENT_DATA 925696) itemoff 12974 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848468992 nr 40960 extent data offset 0 nr 118784 ram 118784 extent compression 2 (lzo) item 47 key (1359617 INODE_ITEM 0) itemoff 12814 itemsize 160 generation 954719 transid 954719 size 28537 nbytes 28672 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184229.93165690 (2019-01-22 19:10:29) ctime 1548184229.93165690 (2019-01-22 19:10:29) mtime 1548184229.93165690 (2019-01-22 19:10:29) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 48 key (1359617 INODE_REF 1357483) itemoff 12795 itemsize 19 index 999 namelen 9 name: dm9601.ko item 49 key (1359617 EXTENT_DATA 0) itemoff 12742 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848509952 nr 12288 extent data offset 0 nr 28672 ram 28672 extent compression 2 (lzo) item 50 key (1359618 INODE_ITEM 0) itemoff 12582 itemsize 160 generation 954719 transid 954719 size 36329 nbytes 36864 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184229.103165501 (2019-01-22 19:10:29) ctime 1548184229.113165312 (2019-01-22 19:10:29) mtime 1548184229.113165312 (2019-01-22 19:10:29) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 51 key (1359618 INODE_REF 1357483) itemoff 12559 itemsize 23 index 1000 namelen 13 name: netconsole.ko item 52 key (1359618 EXTENT_DATA 0) itemoff 12506 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848522240 nr 16384 extent data offset 0 nr 36864 ram 36864 extent compression 2 (lzo) item 53 key (1359619 INODE_ITEM 0) itemoff 12346 itemsize 160 generation 954719 transid 954719 size 7377 nbytes 8192 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184229.113165312 (2019-01-22 19:10:29) ctime 1548184229.113165312 (2019-01-22 19:10:29) mtime 1548184229.113165312 (2019-01-22 19:10:29) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 54 key (1359619 INODE_REF 1357483) itemoff 12319 itemsize 27 index 1001 namelen 17 name: ip6_udp_tunnel.ko item 55 key (1359619 EXTENT_DATA 0) itemoff 12266 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848538624 nr 4096 extent data offset 0 nr 8192 ram 8192 extent compression 2 (lzo) item 56 key (1359620 INODE_ITEM 0) itemoff 12106 itemsize 160 generation 954719 transid 954719 size 140689 nbytes 143360 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184229.123165123 (2019-01-22 19:10:29) ctime 1548184229.133164936 (2019-01-22 19:10:29) mtime 1548184229.133164936 (2019-01-22 19:10:29) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 57 key (1359620 INODE_REF 1357483) itemoff 12088 itemsize 18 index 1002 namelen 8 name: tulip.ko item 58 key (1359620 EXTENT_DATA 0) itemoff 12035 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848542720 nr 57344 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 59 key (1359620 EXTENT_DATA 131072) itemoff 11982 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848600064 nr 4096 extent data offset 0 nr 12288 ram 12288 extent compression 2 (lzo) item 60 key (1359621 INODE_ITEM 0) itemoff 11822 itemsize 160 generation 954719 transid 954719 size 11705 nbytes 12288 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 2 flags 0x0(none) atime 1548184229.143164748 (2019-01-22 19:10:29) ctime 1548184229.143164748 (2019-01-22 19:10:29) mtime 1548184229.143164748 (2019-01-22 19:10:29) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 61 key (1359621 INODE_REF 1357483) itemoff 11799 itemsize 23 index 1003 namelen 13 name: kvaser_pci.ko item 62 key (1359621 EXTENT_DATA 0) itemoff 11746 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848604160 nr 8192 extent data offset 0 nr 12288 ram 12288 extent compression 2 (lzo) item 63 key (1359622 INODE_ITEM 0) itemoff 11586 itemsize 160 generation 954719 transid 954719 size 475136 nbytes 360448 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 5 flags 0x0(none) atime 1548184229.153164559 (2019-01-22 19:10:29) ctime 1548184229.233163052 (2019-01-22 19:10:29) mtime 1548184229.233163052 (2019-01-22 19:10:29) otime 16107652860835337076.2725157735 (-1108403787-01-22 01:43:05) item 64 key (1359622 INODE_REF 1357483) itemoff 11564 itemsize 22 index 1004 namelen 12 name: bluetooth.ko item 65 key (1359622 EXTENT_DATA 0) itemoff 11511 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848624640 nr 86016 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 66 key (1359622 EXTENT_DATA 131072) itemoff 11458 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848710656 nr 94208 extent data offset 0 nr 131072 ram 131072 extent compression 2 (lzo) item 67 key (1359622 EXTENT_DATA 262144) itemoff 11405 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 81920 ram 81920 extent compression 0 (none) item 68 key (1359622 EXTENT_DATA 344064) itemoff 11352 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 8192 ram 40960 extent compression 0 (none) item 69 key (1359622 EXTENT_DATA 352256) itemoff 11299 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848612352 nr 12288 extent data offset 0 nr 32768 ram 32768 extent compression 2 (lzo) item 70 key (1359622 EXTENT_DATA 385024) itemoff 11246 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 24576 ram 90112 extent compression 0 (none) item 71 key (1359622 EXTENT_DATA 409600) itemoff 11193 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 66848845824 nr 12288 extent data offset 0 nr 65536 ram 65536 extent compression 2 (lzo) item 72 key (1359622 EXTENT_DATA 475136) itemoff 11140 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 18446744073709486080 ram 18446744073709486080 extent compression 0 (none) item 73 key (1359629 INODE_ITEM 0) itemoff 10980 itemsize 160 generation 954770 transid 1635337 size 1712797 nbytes 1716224 block group 0 mode 100644 links 1 uid 1000 gid 1000 rdev 0 sequence 7 flags 0x0(none) atime 1572204821.932798461 (2019-10-27 19:33:41) ctime 1557856996.368343436 (2019-05-14 19:03:16) mtime 1557856996.368343436 (2019-05-14 19:03:16) otime 1548187263.115364254 (2019-01-22 20:01:03) item 74 key (1359629 INODE_REF 197739) itemoff 10962 itemsize 18 index 19 namelen 8 name: database item 75 key (1359629 EXTENT_DATA 0) itemoff 10909 itemsize 53 generation 1171531 type 1 (regular) extent data disk byte 91207770112 nr 1716224 extent data offset 0 nr 1716224 ram 1716224 extent compression 0 (none) item 76 key (1359651 INODE_ITEM 0) itemoff 10749 itemsize 160 generation 954773 transid 1635349 size 64496 nbytes 65536 block group 0 mode 100644 links 1 uid 1000 gid 1000 rdev 0 sequence 227 flags 0x0(none) atime 1572205149.925702853 (2019-10-27 19:39:09) ctime 1572205076.417291893 (2019-10-27 19:37:56) mtime 1572205076.417291893 (2019-10-27 19:37:56) otime 1548187349.194062509 (2019-01-22 20:02:29) item 77 key (1359651 INODE_REF 197739) itemoff 10734 itemsize 15 index 20 namelen 5 name: state item 78 key (1359651 EXTENT_DATA 0) itemoff 10681 itemsize 53 generation 1635347 type 1 (regular) extent data disk byte 81073704960 nr 24576 extent data offset 0 nr 65536 ram 65536 extent compression 2 (lzo) item 79 key (1359666 INODE_ITEM 0) itemoff 10521 itemsize 160 generation 954795 transid 1635337 size 32 nbytes 0 block group 0 mode 40700 links 1 uid 1000 gid 1000 rdev 0 sequence 1 flags 0x0(none) atime 1572204828.962646209 (2019-10-27 19:33:48) ctime 1548188187.161114833 (2019-01-22 20:16:27) mtime 1548188187.161114833 (2019-01-22 20:16:27) otime 1548188187.161114833 (2019-01-22 20:16:27) item 80 key (1359666 INODE_REF 223844) itemoff 10479 itemsize 42 index 8 namelen 32 name: ca7c949c26f976e0f53c14399c2ef02e item 81 key (1359666 DIR_ITEM 2051725476) itemoff 10433 itemsize 46 location key (1359667 INODE_ITEM 0) type DIR transid 954795 data_len 0 name_len 16 name: 0255e75cd17e3ecd item 82 key (1359666 DIR_INDEX 2) itemoff 10387 itemsize 46 location key (1359667 INODE_ITEM 0) type DIR transid 954795 data_len 0 name_len 16 name: 0255e75cd17e3ecd item 83 key (1359667 INODE_ITEM 0) itemoff 10227 itemsize 160 generation 954795 transid 1635337 size 240 nbytes 0 block group 0 mode 40700 links 1 uid 1000 gid 1000 rdev 0 sequence 6 flags 0x0(none) atime 1572204828.962646209 (2019-10-27 19:33:48) ctime 1554329459.806022860 (2019-04-03 23:10:59) mtime 1554329459.806022860 (2019-04-03 23:10:59) otime 1548188187.161114833 (2019-01-22 20:16:27) item 84 key (1359667 INODE_REF 1359666) itemoff 10201 itemsize 26 index 2 namelen 16 name: 0255e75cd17e3ecd item 85 key (1359667 DIR_ITEM 148659992) itemoff 10151 itemsize 50 location key (1359669 INODE_ITEM 0) type FILE transid 954795 data_len 0 name_len 20 name: 9334bbb63cd97d82.bin item 86 key (1359667 DIR_ITEM 2133379428) itemoff 10101 itemsize 50 location key (1359668 INODE_ITEM 0) type FILE transid 954795 data_len 0 name_len 20 name: 9334bbb63cd97d82.toc item 87 key (1359667 DIR_ITEM 2644085418) itemoff 10051 itemsize 50 location key (1453872 INODE_ITEM 0) type FILE transid 1075528 data_len 0 name_len 20 name: 2d1d750c26fe5f15.toc item 88 key (1359667 DIR_ITEM 2777212166) itemoff 10001 itemsize 50 location key (1448494 INODE_ITEM 0) type FILE transid 1072132 data_len 0 name_len 20 name: 2d1d750c26fe5f14.toc item 89 key (1359667 DIR_ITEM 3531371386) itemoff 9951 itemsize 50 location key (1448495 INODE_ITEM 0) type FILE transid 1072132 data_len 0 name_len 20 name: 2d1d750c26fe5f14.bin item 90 key (1359667 DIR_ITEM 3933019350) itemoff 9901 itemsize 50 location key (1453873 INODE_ITEM 0) type FILE transid 1075528 data_len 0 name_len 20 name: 2d1d750c26fe5f15.bin item 91 key (1359667 DIR_INDEX 2) itemoff 9851 itemsize 50 location key (1359668 INODE_ITEM 0) type FILE transid 954795 data_len 0 name_len 20 name: 9334bbb63cd97d82.toc item 92 key (1359667 DIR_INDEX 3) itemoff 9801 itemsize 50 location key (1359669 INODE_ITEM 0) type FILE transid 954795 data_len 0 name_len 20 name: 9334bbb63cd97d82.bin item 93 key (1359667 DIR_INDEX 4) itemoff 9751 itemsize 50 location key (1448494 INODE_ITEM 0) type FILE transid 1072132 data_len 0 name_len 20 name: 2d1d750c26fe5f14.toc item 94 key (1359667 DIR_INDEX 5) itemoff 9701 itemsize 50 location key (1448495 INODE_ITEM 0) type FILE transid 1072132 data_len 0 name_len 20 name: 2d1d750c26fe5f14.bin item 95 key (1359667 DIR_INDEX 6) itemoff 9651 itemsize 50 location key (1453872 INODE_ITEM 0) type FILE transid 1075528 data_len 0 name_len 20 name: 2d1d750c26fe5f15.toc item 96 key (1359667 DIR_INDEX 7) itemoff 9601 itemsize 50 location key (1453873 INODE_ITEM 0) type FILE transid 1075528 data_len 0 name_len 20 name: 2d1d750c26fe5f15.bin item 97 key (1359668 INODE_ITEM 0) itemoff 9441 itemsize 160 generation 954795 transid 1054781 size 564 nbytes 564 block group 0 mode 100644 links 1 uid 1000 gid 1000 rdev 0 sequence 0 flags 0x0(none) atime 1553282287.623462056 (2019-03-22 19:18:07) ctime 1551909714.45619774 (2019-03-06 22:01:54) mtime 1551909714.45619774 (2019-03-06 22:01:54) otime 1548188187.161114833 (2019-01-22 20:16:27) item 98 key (1359668 INODE_REF 1359667) itemoff 9411 itemsize 30 index 2 namelen 20 name: 9334bbb63cd97d82.toc item 99 key (1359668 EXTENT_DATA 0) itemoff 8909 itemsize 502 generation 1028065 type 0 (inline) inline extent data size 481 ram_bytes 564 compression 2 (lzo) item 100 key (1359669 INODE_ITEM 0) itemoff 8749 itemsize 160 generation 954795 transid 1054781 size 10001 nbytes 12288 block group 0 mode 100644 links 1 uid 1000 gid 1000 rdev 0 sequence 0 flags 0x0(none) atime 1553282287.633461873 (2019-03-22 19:18:07) ctime 1551909714.45619774 (2019-03-06 22:01:54) mtime 1551909714.45619774 (2019-03-06 22:01:54) otime 1548188187.161114833 (2019-01-22 20:16:27) item 101 key (1359669 INODE_REF 1359667) itemoff 8719 itemsize 30 index 3 namelen 20 name: 9334bbb63cd97d82.bin item 102 key (1359669 EXTENT_DATA 0) itemoff 8666 itemsize 53 generation 976418 type 1 (regular) extent data disk byte 1147408384 nr 4096 extent data offset 0 nr 4096 ram 8192 extent compression 2 (lzo) item 103 key (1359669 EXTENT_DATA 4096) itemoff 8613 itemsize 53 generation 1028064 type 1 (regular) extent data disk byte 58874425344 nr 4096 extent data offset 0 nr 4096 ram 8192 extent compression 2 (lzo) item 104 key (1359669 EXTENT_DATA 8192) itemoff 8560 itemsize 53 generation 1028065 type 1 (regular) extent data disk byte 63909822464 nr 4096 extent data offset 0 nr 4096 ram 4096 extent compression 0 (none) item 105 key (1359746 INODE_ITEM 0) itemoff 8400 itemsize 160 generation 954996 transid 954996 size 42396 nbytes 45056 block group 0 mode 100644 links 1 uid 1000 gid 1000 rdev 0 sequence 53 flags 0x0(none) atime 1548197385.203646006 (2019-01-22 22:49:45) ctime 1548197385.203646006 (2019-01-22 22:49:45) mtime 1548197385.203646006 (2019-01-22 22:49:45) otime 1548197385.203646006 (2019-01-22 22:49:45) item 106 key (1359746 INODE_REF 110364) itemoff 8378 itemsize 22 index 11204 namelen 12 name: bd5a793e.jpg item 107 key (1359746 EXTENT_DATA 0) itemoff 8325 itemsize 53 generation 954996 type 1 (regular) extent data disk byte 4541730816 nr 45056 extent data offset 0 nr 45056 ram 45056 extent compression 0 (none) item 108 key (1359802 INODE_ITEM 0) itemoff 8165 itemsize 160 generation 955004 transid 1072255 size 136 nbytes 0 block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0 sequence 12 flags 0x0(none) atime 1554158333.766861801 (2019-04-01 23:38:53) ctime 1548197604.229726284 (2019-01-22 22:53:24) mtime 1548197604.229726284 (2019-01-22 22:53:24) otime 1548197597.639843733 (2019-01-22 22:53:17) item 109 key (1359802 INODE_REF 188692) itemoff 8151 itemsize 14 index 27 namelen 4 name: temp item 110 key (1359802 DIR_ITEM 3060314850) itemoff 8087 itemsize 64 location key (1359810 INODE_ITEM 0) type DIR transid 955004 data_len 0 name_len 34 name: La_casa_de_papel_s1_WEBRip_eng_zip item 111 key (1359802 DIR_ITEM 4274709003) itemoff 8023 itemsize 64 location key (1359809 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 34 name: La_casa_de_papel_s1_WEBRip_eng.zip item 112 key (1359802 DIR_INDEX 2) itemoff 7959 itemsize 64 location key (1359809 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 34 name: La_casa_de_papel_s1_WEBRip_eng.zip item 113 key (1359802 DIR_INDEX 3) itemoff 7895 itemsize 64 location key (1359810 INODE_ITEM 0) type DIR transid 955004 data_len 0 name_len 34 name: La_casa_de_papel_s1_WEBRip_eng_zip item 114 key (1359809 INODE_ITEM 0) itemoff 7735 itemsize 160 generation 955004 transid 955004 size 223267 nbytes 225280 block group 0 mode 100644 links 1 uid 1000 gid 1000 rdev 0 sequence 9 flags 0x0(none) atime 1548197604.59729311 (2019-01-22 22:53:24) ctime 1548197604.199726821 (2019-01-22 22:53:24) mtime 1548197604.199726821 (2019-01-22 22:53:24) otime 1548197604.59729311 (2019-01-22 22:53:24) item 115 key (1359809 INODE_REF 1359802) itemoff 7691 itemsize 44 index 2 namelen 34 name: La_casa_de_papel_s1_WEBRip_eng.zip item 116 key (1359809 EXTENT_DATA 0) itemoff 7638 itemsize 53 generation 955004 type 1 (regular) extent data disk byte 61471092736 nr 225280 extent data offset 0 nr 225280 ram 225280 extent compression 0 (none) item 117 key (1359810 INODE_ITEM 0) itemoff 7478 itemsize 160 generation 955004 transid 1072255 size 60 nbytes 0 block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0 sequence 29 flags 0x0(none) atime 1554158333.766861801 (2019-04-01 23:38:53) ctime 1548197604.229726284 (2019-01-22 22:53:24) mtime 1548197604.229726284 (2019-01-22 22:53:24) otime 1548197604.229726284 (2019-01-22 22:53:24) item 118 key (1359810 INODE_REF 1359802) itemoff 7434 itemsize 44 index 3 namelen 34 name: La_casa_de_papel_s1_WEBRip_eng_zip item 119 key (1359810 DIR_ITEM 3732099734) itemoff 7374 itemsize 60 location key (1359811 INODE_ITEM 0) type DIR transid 955004 data_len 0 name_len 30 name: La casa de papel s1 WEBRip eng item 120 key (1359810 DIR_INDEX 2) itemoff 7314 itemsize 60 location key (1359811 INODE_ITEM 0) type DIR transid 955004 data_len 0 name_len 30 name: La casa de papel s1 WEBRip eng item 121 key (1359811 INODE_ITEM 0) itemoff 7154 itemsize 160 generation 955004 transid 1072255 size 1092 nbytes 0 block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0 sequence 38 flags 0x0(none) atime 1554158333.766861801 (2019-04-01 23:38:53) ctime 1548197604.299725037 (2019-01-22 22:53:24) mtime 1548197604.299725037 (2019-01-22 22:53:24) otime 1548197604.229726284 (2019-01-22 22:53:24) item 122 key (1359811 INODE_REF 1359810) itemoff 7114 itemsize 40 index 2 namelen 30 name: La casa de papel s1 WEBRip eng item 123 key (1359811 DIR_ITEM 329641938) itemoff 7042 itemsize 72 location key (1359818 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E07.WEBRip.Netflix.srt item 124 key (1359811 DIR_ITEM 534648133) itemoff 6970 itemsize 72 location key (1359820 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E09.WEBRip.Netflix.srt item 125 key (1359811 DIR_ITEM 619059311) itemoff 6898 itemsize 72 location key (1359817 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E06.WEBRip.Netflix.srt item 126 key (1359811 DIR_ITEM 681435896) itemoff 6826 itemsize 72 location key (1359819 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E08.WEBRip.Netflix.srt item 127 key (1359811 DIR_ITEM 1247934229) itemoff 6754 itemsize 72 location key (1359815 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E04.WEBRip.Netflix.srt item 128 key (1359811 DIR_ITEM 2099380392) itemoff 6682 itemsize 72 location key (1359816 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E05.WEBRip.Netflix.srt item 129 key (1359811 DIR_ITEM 2590266380) itemoff 6610 itemsize 72 location key (1359821 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E10.WEBRip.Netflix.srt item 130 key (1359811 DIR_ITEM 2687360604) itemoff 6538 itemsize 72 location key (1359812 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E01.WEBRip.Netflix.srt item 131 key (1359811 DIR_ITEM 2905056177) itemoff 6466 itemsize 72 location key (1359822 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E11.WEBRip.Netflix.srt item 132 key (1359811 DIR_ITEM 3282062539) itemoff 6394 itemsize 72 location key (1359824 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E13.WEBRip.Netflix.srt item 133 key (1359811 DIR_ITEM 3467260198) itemoff 6322 itemsize 72 location key (1359814 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E03.WEBRip.Netflix.srt item 134 key (1359811 DIR_ITEM 4108565366) itemoff 6250 itemsize 72 location key (1359823 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E12.WEBRip.Netflix.srt item 135 key (1359811 DIR_ITEM 4192847515) itemoff 6178 itemsize 72 location key (1359813 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E02.WEBRip.Netflix.srt item 136 key (1359811 DIR_INDEX 2) itemoff 6106 itemsize 72 location key (1359812 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E01.WEBRip.Netflix.srt item 137 key (1359811 DIR_INDEX 3) itemoff 6034 itemsize 72 location key (1359813 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E02.WEBRip.Netflix.srt item 138 key (1359811 DIR_INDEX 4) itemoff 5962 itemsize 72 location key (1359814 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E03.WEBRip.Netflix.srt item 139 key (1359811 DIR_INDEX 5) itemoff 5890 itemsize 72 location key (1359815 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E04.WEBRip.Netflix.srt item 140 key (1359811 DIR_INDEX 6) itemoff 5818 itemsize 72 location key (1359816 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E05.WEBRip.Netflix.srt item 141 key (1359811 DIR_INDEX 7) itemoff 5746 itemsize 72 location key (1359817 INODE_ITEM 0) type FILE transid 955004 data_len 0 name_len 42 name: La.casa.de.papel.S01E06.WEBRip.Netflix.srt [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2020-02-13 0:26 ` Qu Wenruo 2020-02-13 13:04 ` Samir Benmendil @ 2020-02-13 14:01 ` Qu Wenruo 2020-02-15 15:34 ` Samir Benmendil 1 sibling, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2020-02-13 14:01 UTC (permalink / raw) To: Samir Benmendil, linux-btrfs [-- Attachment #1.1: Type: text/plain, Size: 2317 bytes --] On 2020/2/13 上午8:26, Qu Wenruo wrote: > > > On 2020/2/13 上午5:58, Samir Benmendil wrote: >> Hello, >> >> I've been getting the following "BTRFS errors" for a while now, the wiki >> [0] advises to report such occurrences to this list. > > Please provide the following dump: > > # btrfs ins dump-tree -b 194756837376 /dev/sda2 > # btrfs ins dump-tree -b 194347958272 /dev/sda2 > > Thanks, > Qu >> >> BTRFS critical (device sda2): corrupt leaf: root=466 block=194756837376 >> slot=72 ino=1359622 file_offset=475136, extent end overflow, have file >> offset 475136 extent num bytes 18446744073709486080 item 72 key (1359622 EXTENT_DATA 475136) itemoff 11140 itemsize 53 generation 954719 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 18446744073709486080 ram 18446744073709486080 extent compression 0 (none) Also obvious underflow. >> BTRFS critical (device sda2): corrupt leaf: root=466 block=194347958272 >> slot=131 ino=1357455 file_offset=1044480, extent end overflow, have file >> offset 1044480 extent num bytes 18446744073708908544 item 131 key (1357455 EXTENT_DATA 1044480) itemoff 6497 itemsize 53 generation 952602 type 1 (regular) extent data disk byte 0 nr 0 extent data offset 0 nr 18446744073708908544 ram 18446744073708908544 extent compression 0 (none) As you can see, 18446744073708908544 = -653072, which means it overflows. Both look like a bug in older kernels. Since currently btrfsprogs can't detect nor fix it yet, the only way is to delete the offending files. You can use the inode number 1357455, and root id 466 to locate the offending files, and delete it using older kernels. (root id is your subvolume id, which is shown in `btrfs subv list`. inode number can be passed to `find` command using `-inum` option) The btrfs-progs fix will come soon. Thanks, Qu >> BTRFS error (device sda2): block=194347958272 read time tree block >> corruption detected >> >> I can reproduce these errors consistently by running `updatedb`, I >> suppose some tree block in one of the file it reads is corrupted. >> >> Thanks in advance for your help, >> Regards, >> Samir >> >> [0] >> https://btrfs.wiki.kernel.org/index.php/Tree-checker#How_to_handle_such_error >> > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2020-02-13 14:01 ` Qu Wenruo @ 2020-02-15 15:34 ` Samir Benmendil 0 siblings, 0 replies; 75+ messages in thread From: Samir Benmendil @ 2020-02-15 15:34 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 2176 bytes --] On Feb 13, 2020 at 22:01, Qu Wenruo wrote: > On 2020/2/13 上午8:26, Qu Wenruo wrote: >> On 2020/2/13 上午5:58, Samir Benmendil wrote: >>> I've been getting the following "BTRFS errors" for a while now, the >>> wiki [0] advises to report such occurrences to this list. >> >> Please provide the following dump: >> >> # btrfs ins dump-tree -b 194756837376 /dev/sda2 >> # btrfs ins dump-tree -b 194347958272 /dev/sda2 >>> >>> BTRFS critical (device sda2): corrupt leaf: root=466 block=194756837376 >>> slot=72 ino=1359622 file_offset=475136, extent end overflow, have file >>> offset 475136 extent num bytes 18446744073709486080 > > item 72 key (1359622 EXTENT_DATA 475136) itemoff 11140 itemsize 53 > generation 954719 type 1 (regular) > extent data disk byte 0 nr 0 > extent data offset 0 nr 18446744073709486080 ram 18446744073709486080 > extent compression 0 (none) > > Also obvious underflow. > >>> BTRFS critical (device sda2): corrupt leaf: root=466 block=194347958272 >>> slot=131 ino=1357455 file_offset=1044480, extent end overflow, have file >>> offset 1044480 extent num bytes 18446744073708908544 > > item 131 key (1357455 EXTENT_DATA 1044480) itemoff 6497 itemsize 53 > generation 952602 type 1 (regular) > extent data disk byte 0 nr 0 > extent data offset 0 nr 18446744073708908544 ram 18446744073708908544 > extent compression 0 (none) > > As you can see, 18446744073708908544 = -653072, which means it overflows. > > Both look like a bug in older kernels. > > Since currently btrfsprogs can't detect nor fix it yet, the only way is > to delete the offending files. > > You can use the inode number 1357455, and root id 466 to locate the > offending files, and delete it using older kernels. > (root id is your subvolume id, which is shown in `btrfs subv list`. > inode number can be passed to `find` command using `-inum` option) I deleted the offending file for inode 1357455 using kernel 4.4.165. However `find` did not return any results for inode 1359622. The newer kernel doesn't seem to complain about that anymore though. Thank you for you help Qu. Regards Samir [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 75+ messages in thread
* read time tree block corruption detected @ 2019-12-29 20:43 Patrick Erley 2019-12-29 22:07 ` Chris Murphy 2019-12-30 0:46 ` Qu Wenruo 0 siblings, 2 replies; 75+ messages in thread From: Patrick Erley @ 2019-12-29 20:43 UTC (permalink / raw) To: linux-btrfs On a system where I've been tinkering with linux-next for years, my / has got some errors. When migrating from 5.1 to 5.2, I saw these errors, but just ignored them and went back to 5.1, and continued my tinkering. Over the holidays, I decided to try to upgrade the kernel, saw the errors again, and decided to look into them, finding the tree-checker page on the kernel docs, and am writing this e-mail. My / does not contain anything sensitive or important, as /home and /usr/src are both subvolumes on a separate larger drive. btrfs fi show: Label: none uuid: 815266d6-a8b9-4f63-a593-02fde178263f Total devices 2 FS bytes used 93.81GiB devid 1 size 115.12GiB used 115.11GiB path /dev/nvme0n1p2 devid 3 size 115.12GiB used 115.11GiB path /dev/sda3 Label: none uuid: 4bd97711-b63c-40cb-bfa5-aa9c2868cf98 Total devices 1 FS bytes used 536.48GiB devid 1 size 834.63GiB used 833.59GiB path /dev/sda5 Booting a more recent kernel, I get spammed with: [ 8.243899] BTRFS critical (device nvme0n1p2): corrupt leaf: root=5 block=303629811712 slot=30 ino=5380870, invalid inode generation: has 13221446351398931016 expect [0, 2997884] [ 8.243902] BTRFS error (device nvme0n1p2): block=303629811712 read time tree block corruption detected There are 6 corrupted inodes: cat dmesg.foo | grep "BTRFS critical" | sed -re 's:.*block=([0-9]*).*ino=([0-9]+).*:\1 \2:' | sort | uniq 303629811712 5380870 303712501760 3277548 303861395456 5909140 304079065088 2228479 304573444096 3539224 305039556608 1442149 and they all have the same value for the inode generation. Before I reboot into a livecd and btrfs check --repair, is there anything interesting that a snapshot of the state would show? I have 800gb unpartitioned on the nvme, so backing up before is already in the plans, and I could just as easily grab an image of the partitions while I'm at it. ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-29 20:43 Patrick Erley @ 2019-12-29 22:07 ` Chris Murphy 2019-12-29 22:27 ` Patrick Erley 2019-12-30 0:46 ` Qu Wenruo 1 sibling, 1 reply; 75+ messages in thread From: Chris Murphy @ 2019-12-29 22:07 UTC (permalink / raw) To: Patrick Erley; +Cc: Btrfs BTRFS On Sun, Dec 29, 2019 at 1:44 PM Patrick Erley <pat-lkml@erley.org> wrote: > > On a system where I've been tinkering with linux-next for years, my / > has got some errors. When migrating from 5.1 to 5.2, I saw these > errors, but just ignored them and went back to 5.1, and continued my > tinkering. Over the holidays, I decided to try to upgrade the kernel, > saw the errors again, and decided to look into them, finding the > tree-checker page on the kernel docs, and am writing this e-mail. > > My / does not contain anything sensitive or important, as /home and > /usr/src are both subvolumes on a separate larger drive. > > btrfs fi show: > Label: none uuid: 815266d6-a8b9-4f63-a593-02fde178263f > Total devices 2 FS bytes used 93.81GiB > devid 1 size 115.12GiB used 115.11GiB path /dev/nvme0n1p2 > devid 3 size 115.12GiB used 115.11GiB path /dev/sda3 > > Label: none uuid: 4bd97711-b63c-40cb-bfa5-aa9c2868cf98 > Total devices 1 FS bytes used 536.48GiB > devid 1 size 834.63GiB used 833.59GiB path /dev/sda5 > > Booting a more recent kernel, I get spammed with: > [ 8.243899] BTRFS critical (device nvme0n1p2): corrupt leaf: root=5 > block=303629811712 slot=30 ino=5380870, invalid inode generation: has > 13221446351398931016 expect [0, 2997884] > [ 8.243902] BTRFS error (device nvme0n1p2): block=303629811712 read > time tree block corruption detected > > There are 6 corrupted inodes: > cat dmesg.foo | grep "BTRFS critical" | sed -re > 's:.*block=([0-9]*).*ino=([0-9]+).*:\1 \2:' | sort | uniq > 303629811712 5380870 > 303712501760 3277548 > 303861395456 5909140 > 304079065088 2228479 > 304573444096 3539224 > 305039556608 1442149 > > and they all have the same value for the inode generation. Before I > reboot into a livecd and btrfs check --repair, is there anything > interesting that a snapshot of the state would show? I have 800gb > unpartitioned on the nvme, so backing up before is already in the > plans, and I could just as easily grab an image of the partitions > while I'm at it. I'm not certain whether btrfs check can fix these kinds of errors yet. Can you use btrfs-progs v5.4 and just do a 'btrfs check' and also again with 'btrfs check --mode=lowmem' ? I'm curious if either mode sees the same problem the kernel tree checker sees. -- Chris Murphy ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-29 22:07 ` Chris Murphy @ 2019-12-29 22:27 ` Patrick Erley 2019-12-29 22:32 ` Chris Murphy 0 siblings, 1 reply; 75+ messages in thread From: Patrick Erley @ 2019-12-29 22:27 UTC (permalink / raw) To: Chris Murphy; +Cc: Btrfs BTRFS Should I --force it while mounted, or run the checks from RO mount status? On Sun, Dec 29, 2019 at 2:07 PM Chris Murphy <lists@colorremedies.com> wrote: > > On Sun, Dec 29, 2019 at 1:44 PM Patrick Erley <pat-lkml@erley.org> wrote: > > > > On a system where I've been tinkering with linux-next for years, my / > > has got some errors. When migrating from 5.1 to 5.2, I saw these > > errors, but just ignored them and went back to 5.1, and continued my > > tinkering. Over the holidays, I decided to try to upgrade the kernel, > > saw the errors again, and decided to look into them, finding the > > tree-checker page on the kernel docs, and am writing this e-mail. > > > > My / does not contain anything sensitive or important, as /home and > > /usr/src are both subvolumes on a separate larger drive. > > > > btrfs fi show: > > Label: none uuid: 815266d6-a8b9-4f63-a593-02fde178263f > > Total devices 2 FS bytes used 93.81GiB > > devid 1 size 115.12GiB used 115.11GiB path /dev/nvme0n1p2 > > devid 3 size 115.12GiB used 115.11GiB path /dev/sda3 > > > > Label: none uuid: 4bd97711-b63c-40cb-bfa5-aa9c2868cf98 > > Total devices 1 FS bytes used 536.48GiB > > devid 1 size 834.63GiB used 833.59GiB path /dev/sda5 > > > > Booting a more recent kernel, I get spammed with: > > [ 8.243899] BTRFS critical (device nvme0n1p2): corrupt leaf: root=5 > > block=303629811712 slot=30 ino=5380870, invalid inode generation: has > > 13221446351398931016 expect [0, 2997884] > > [ 8.243902] BTRFS error (device nvme0n1p2): block=303629811712 read > > time tree block corruption detected > > > > There are 6 corrupted inodes: > > cat dmesg.foo | grep "BTRFS critical" | sed -re > > 's:.*block=([0-9]*).*ino=([0-9]+).*:\1 \2:' | sort | uniq > > 303629811712 5380870 > > 303712501760 3277548 > > 303861395456 5909140 > > 304079065088 2228479 > > 304573444096 3539224 > > 305039556608 1442149 > > > > and they all have the same value for the inode generation. Before I > > reboot into a livecd and btrfs check --repair, is there anything > > interesting that a snapshot of the state would show? I have 800gb > > unpartitioned on the nvme, so backing up before is already in the > > plans, and I could just as easily grab an image of the partitions > > while I'm at it. > > I'm not certain whether btrfs check can fix these kinds of errors yet. > Can you use btrfs-progs v5.4 and just do a 'btrfs check' and also > again with 'btrfs check --mode=lowmem' ? I'm curious if either mode > sees the same problem the kernel tree checker sees. > > -- > Chris Murphy ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-29 22:27 ` Patrick Erley @ 2019-12-29 22:32 ` Chris Murphy 2019-12-29 22:36 ` Patrick Erley 0 siblings, 1 reply; 75+ messages in thread From: Chris Murphy @ 2019-12-29 22:32 UTC (permalink / raw) To: Patrick Erley; +Cc: Chris Murphy, Btrfs BTRFS On Sun, Dec 29, 2019 at 3:28 PM Patrick Erley <pat-lkml@erley.org> wrote: > > Should I --force it while mounted, or run the checks from RO mount status? Check isn't reliable when the volume is mounted rw so I personally wouldn't bother trying. -- Chris Murphy ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-29 22:32 ` Chris Murphy @ 2019-12-29 22:36 ` Patrick Erley 2019-12-29 23:11 ` Chris Murphy 0 siblings, 1 reply; 75+ messages in thread From: Patrick Erley @ 2019-12-29 22:36 UTC (permalink / raw) To: Chris Murphy; +Cc: Btrfs BTRFS K. Gotta roll a new initrd with btrfs 5.4, then will reboot into it and grab logs of btrfs check (both iterations, against each of my volumes). Probably be a few hours, so don't expect another follow up until tomorrow. Thanks for chiming in. Would it be worth creating an image of the drive while I'm in single user mode for faster analysis/iteration? On Sun, Dec 29, 2019 at 2:32 PM Chris Murphy <lists@colorremedies.com> wrote: > > On Sun, Dec 29, 2019 at 3:28 PM Patrick Erley <pat-lkml@erley.org> wrote: > > > > Should I --force it while mounted, or run the checks from RO mount status? > > Check isn't reliable when the volume is mounted rw so I personally > wouldn't bother trying. > > > -- > Chris Murphy ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-29 22:36 ` Patrick Erley @ 2019-12-29 23:11 ` Chris Murphy 2019-12-29 23:19 ` Patrick Erley 0 siblings, 1 reply; 75+ messages in thread From: Chris Murphy @ 2019-12-29 23:11 UTC (permalink / raw) To: Patrick Erley; +Cc: Chris Murphy, Btrfs BTRFS On Sun, Dec 29, 2019 at 3:36 PM Patrick Erley <pat-lkml@erley.org> wrote: > > K. Gotta roll a new initrd with btrfs 5.4, then will reboot into it > and grab logs of btrfs check (both iterations, against each of my > volumes). Probably be a few hours, so don't expect another follow up > until tomorrow. Thanks for chiming in. > > Would it be worth creating an image of the drive while I'm in single > user mode for faster analysis/iteration? Depends on the developer who replies. These days they usually just ask for selected dump-tree output. The image might be useful if you can't wait and want to blow away the file system and start over sooner than later. -- Chris Murphy ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-29 23:11 ` Chris Murphy @ 2019-12-29 23:19 ` Patrick Erley 2019-12-29 23:24 ` Chris Murphy 0 siblings, 1 reply; 75+ messages in thread From: Patrick Erley @ 2019-12-29 23:19 UTC (permalink / raw) To: Chris Murphy; +Cc: Btrfs BTRFS Nah, this 'corruption' hasn't affected anything other than not being able to go to a newer kernel. The system doesn't do anything important. I'll grab the logs, and clone an image of the drive so I don't need to go single user RO mode to get dumps for devs when follow up requests come in. On Sun, Dec 29, 2019 at 3:11 PM Chris Murphy <lists@colorremedies.com> wrote: > > On Sun, Dec 29, 2019 at 3:36 PM Patrick Erley <pat-lkml@erley.org> wrote: > > > > K. Gotta roll a new initrd with btrfs 5.4, then will reboot into it > > and grab logs of btrfs check (both iterations, against each of my > > volumes). Probably be a few hours, so don't expect another follow up > > until tomorrow. Thanks for chiming in. > > > > Would it be worth creating an image of the drive while I'm in single > > user mode for faster analysis/iteration? > > Depends on the developer who replies. These days they usually just ask > for selected dump-tree output. The image might be useful if you can't > wait and want to blow away the file system and start over sooner than > later. > > > > -- > Chris Murphy ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-29 23:19 ` Patrick Erley @ 2019-12-29 23:24 ` Chris Murphy 2019-12-29 23:26 ` Patrick Erley 0 siblings, 1 reply; 75+ messages in thread From: Chris Murphy @ 2019-12-29 23:24 UTC (permalink / raw) To: Patrick Erley; +Cc: Chris Murphy, Btrfs BTRFS On Sun, Dec 29, 2019 at 4:19 PM Patrick Erley <pat-lkml@erley.org> wrote: > > Nah, this 'corruption' hasn't affected anything other than not being > able to go to a newer kernel. The system doesn't do anything > important. I'll grab the logs, and clone an image of the drive so I > don't need to go single user RO mode to get dumps for devs when follow > up requests come in. A clone of any rw file system is likewise unreliable, for the same reasons that a forced btrfs check on a rw file system is unreliable. So unreliable it's a waste of time for you and the developer, so I wouldn't even bother with it. -- Chris Murphy ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-29 23:24 ` Chris Murphy @ 2019-12-29 23:26 ` Patrick Erley 0 siblings, 0 replies; 75+ messages in thread From: Patrick Erley @ 2019-12-29 23:26 UTC (permalink / raw) To: Chris Murphy; +Cc: Btrfs BTRFS No, I mean, I'm going to reboot single user RO, run the btrfs checks, catch the logs, and dump a copy of the partition, so follow ups don't require RO reboots. On Sun, Dec 29, 2019 at 3:25 PM Chris Murphy <lists@colorremedies.com> wrote: > > On Sun, Dec 29, 2019 at 4:19 PM Patrick Erley <pat-lkml@erley.org> wrote: > > > > Nah, this 'corruption' hasn't affected anything other than not being > > able to go to a newer kernel. The system doesn't do anything > > important. I'll grab the logs, and clone an image of the drive so I > > don't need to go single user RO mode to get dumps for devs when follow > > up requests come in. > > A clone of any rw file system is likewise unreliable, for the same > reasons that a forced btrfs check on a rw file system is unreliable. > So unreliable it's a waste of time for you and the developer, so I > wouldn't even bother with it. > > > -- > Chris Murphy ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-29 20:43 Patrick Erley 2019-12-29 22:07 ` Chris Murphy @ 2019-12-30 0:46 ` Qu Wenruo 2019-12-30 5:36 ` Patrick Erley 1 sibling, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2019-12-30 0:46 UTC (permalink / raw) To: Patrick Erley, linux-btrfs [-- Attachment #1.1: Type: text/plain, Size: 2331 bytes --] On 2019/12/30 上午4:43, Patrick Erley wrote: > On a system where I've been tinkering with linux-next for years, my / > has got some errors. When migrating from 5.1 to 5.2, I saw these > errors, but just ignored them and went back to 5.1, and continued my > tinkering. Over the holidays, I decided to try to upgrade the kernel, > saw the errors again, and decided to look into them, finding the > tree-checker page on the kernel docs, and am writing this e-mail. > > My / does not contain anything sensitive or important, as /home and > /usr/src are both subvolumes on a separate larger drive. > > btrfs fi show: > Label: none uuid: 815266d6-a8b9-4f63-a593-02fde178263f > Total devices 2 FS bytes used 93.81GiB > devid 1 size 115.12GiB used 115.11GiB path /dev/nvme0n1p2 > devid 3 size 115.12GiB used 115.11GiB path /dev/sda3 > > Label: none uuid: 4bd97711-b63c-40cb-bfa5-aa9c2868cf98 > Total devices 1 FS bytes used 536.48GiB > devid 1 size 834.63GiB used 833.59GiB path /dev/sda5 > > Booting a more recent kernel, I get spammed with: > [ 8.243899] BTRFS critical (device nvme0n1p2): corrupt leaf: root=5 > block=303629811712 slot=30 ino=5380870, invalid inode generation: has > 13221446351398931016 expect [0, 2997884] Inode generation repair is introduced in v5.4. So feel free to use `btrfs check --repair` to repair such problems. But please run a `btrfs check` without --repair and paste the output, just in case there are extra problems which may screw up repair. Thanks, Qu > [ 8.243902] BTRFS error (device nvme0n1p2): block=303629811712 read > time tree block corruption detected > > There are 6 corrupted inodes: > cat dmesg.foo | grep "BTRFS critical" | sed -re > 's:.*block=([0-9]*).*ino=([0-9]+).*:\1 \2:' | sort | uniq > 303629811712 5380870 > 303712501760 3277548 > 303861395456 5909140 > 304079065088 2228479 > 304573444096 3539224 > 305039556608 1442149 > > and they all have the same value for the inode generation. Before I > reboot into a livecd and btrfs check --repair, is there anything > interesting that a snapshot of the state would show? I have 800gb > unpartitioned on the nvme, so backing up before is already in the > plans, and I could just as easily grab an image of the partitions > while I'm at it. > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 520 bytes --] ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 0:46 ` Qu Wenruo @ 2019-12-30 5:36 ` Patrick Erley 2019-12-30 5:43 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Patrick Erley @ 2019-12-30 5:36 UTC (permalink / raw) To: Qu Wenruo; +Cc: Btrfs BTRFS (ugh, just realized gmail does top replies. Sorry... will try to figure out how to make gsuite behave like a sane mail client before my next reply): here's btrfs check /dev/nvme0n1p2 (sda3, which is a mirror of it, has exactly the same output) [1/7] checking root items [2/7] checking extents [3/7] checking free space cache [4/7] checking fs roots [5/7] checking only csums items (without verifying data) [6/7] checking root refs [7/7] checking quota groups skipped (not enabled on this FS) Opening filesystem to check... Checking filesystem on /dev/nvme0n1p2 UUID: 815266d6-a8b9-4f63-a593-02fde178263f found 89383137280 bytes used, no error found total csum bytes: 85617340 total tree bytes: 1670774784 total fs tree bytes: 1451180032 total extent tree bytes: 107905024 btree space waste bytes: 413362851 file data blocks allocated: 90769887232 referenced 88836960256 And here's the 'lowmen' variant: [1/7] checking root items [2/7] checking extents [3/7] checking free space cache [4/7] checking fs roots ERROR: root 5 EXTENT_DATA[266 1409024] csum missing, have: 0, expected: 65536 ERROR: root 5 INODE[266] nbytes 1437696 not equal to extent_size 1503232 ERROR: root 5 EXTENT_DATA[4731008 4096] csum missing, have: 0, expected: 2093056 ERROR: root 5 EXTENT_DATA[4731008 2101248] csum missing, have: 0, expected: 8192 ERROR: root 5 INODE[4731008] nbytes 73728 not equal to extent_size 2174976 ERROR: root 5 EXTENT_DATA[4731012 4096] csum missing, have: 0, expected: 8192 ERROR: root 5 INODE[4731012] nbytes 8192 not equal to extent_size 16384 ERROR: root 5 EXTENT_DATA[6563647 8192] csum missing, have: 0, expected: 4096 ERROR: root 5 INODE[6563647] nbytes 12288 not equal to extent_size 16384 ERROR: root 5 EXTENT_DATA[7090739 131072] csum missing, have: 0, expected: 24576 ERROR: root 5 INODE[7090739] nbytes 139264 not equal to extent_size 163840 ERROR: errors found in fs roots Opening filesystem to check... Checking filesystem on /dev/nvme0n1p2 UUID: 815266d6-a8b9-4f63-a593-02fde178263f found 89383137280 bytes used, error(s) found total csum bytes: 85617340 total tree bytes: 1670774784 total fs tree bytes: 1451180032 total extent tree bytes: 107905024 btree space waste bytes: 413362851 file data blocks allocated: 90769887232 referenced 88836960256 On Sun, Dec 29, 2019 at 4:46 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > On 2019/12/30 上午4:43, Patrick Erley wrote: > > On a system where I've been tinkering with linux-next for years, my / > > has got some errors. When migrating from 5.1 to 5.2, I saw these > > errors, but just ignored them and went back to 5.1, and continued my > > tinkering. Over the holidays, I decided to try to upgrade the kernel, > > saw the errors again, and decided to look into them, finding the > > tree-checker page on the kernel docs, and am writing this e-mail. > > > > My / does not contain anything sensitive or important, as /home and > > /usr/src are both subvolumes on a separate larger drive. > > > > btrfs fi show: > > Label: none uuid: 815266d6-a8b9-4f63-a593-02fde178263f > > Total devices 2 FS bytes used 93.81GiB > > devid 1 size 115.12GiB used 115.11GiB path /dev/nvme0n1p2 > > devid 3 size 115.12GiB used 115.11GiB path /dev/sda3 > > > > Label: none uuid: 4bd97711-b63c-40cb-bfa5-aa9c2868cf98 > > Total devices 1 FS bytes used 536.48GiB > > devid 1 size 834.63GiB used 833.59GiB path /dev/sda5 > > > > Booting a more recent kernel, I get spammed with: > > [ 8.243899] BTRFS critical (device nvme0n1p2): corrupt leaf: root=5 > > block=303629811712 slot=30 ino=5380870, invalid inode generation: has > > 13221446351398931016 expect [0, 2997884] > > Inode generation repair is introduced in v5.4. So feel free to use > `btrfs check --repair` to repair such problems. > > But please run a `btrfs check` without --repair and paste the output, > just in case there are extra problems which may screw up repair. > > Thanks, > Qu > > > [ 8.243902] BTRFS error (device nvme0n1p2): block=303629811712 read > > time tree block corruption detected > > > > There are 6 corrupted inodes: > > cat dmesg.foo | grep "BTRFS critical" | sed -re > > 's:.*block=([0-9]*).*ino=([0-9]+).*:\1 \2:' | sort | uniq > > 303629811712 5380870 > > 303712501760 3277548 > > 303861395456 5909140 > > 304079065088 2228479 > > 304573444096 3539224 > > 305039556608 1442149 > > > > and they all have the same value for the inode generation. Before I > > reboot into a livecd and btrfs check --repair, is there anything > > interesting that a snapshot of the state would show? I have 800gb > > unpartitioned on the nvme, so backing up before is already in the > > plans, and I could just as easily grab an image of the partitions > > while I'm at it. > > > ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 5:36 ` Patrick Erley @ 2019-12-30 5:43 ` Qu Wenruo 2019-12-30 5:47 ` Patrick Erley 0 siblings, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2019-12-30 5:43 UTC (permalink / raw) To: Patrick Erley; +Cc: Btrfs BTRFS [-- Attachment #1.1: Type: text/plain, Size: 5560 bytes --] On 2019/12/30 下午1:36, Patrick Erley wrote: > (ugh, just realized gmail does top replies. Sorry... will try to > figure out how to make gsuite behave like a sane mail client before my > next reply): > > here's btrfs check /dev/nvme0n1p2 (sda3, which is a mirror of it, has > exactly the same output) > > [1/7] checking root items > [2/7] checking extents > [3/7] checking free space cache > [4/7] checking fs roots > [5/7] checking only csums items (without verifying data) > [6/7] checking root refs > [7/7] checking quota groups skipped (not enabled on this FS) > Opening filesystem to check... > Checking filesystem on /dev/nvme0n1p2 > UUID: 815266d6-a8b9-4f63-a593-02fde178263f > found 89383137280 bytes used, no error found > total csum bytes: 85617340 > total tree bytes: 1670774784 > total fs tree bytes: 1451180032 > total extent tree bytes: 107905024 > btree space waste bytes: 413362851 > file data blocks allocated: 90769887232 > referenced 88836960256 It looks too good to be true, is the btrfs-progs v5.4? IIRC in v5.4 we should report inodes generation problems. But anyway, your fs looks pretty OK to have a --repair run on it. Please note that, only v5.4 has inode generation repair, older ones can't really repair such problem. > > And here's the 'lowmen' variant: > > [1/7] checking root items > [2/7] checking extents > [3/7] checking free space cache > [4/7] checking fs roots > ERROR: root 5 EXTENT_DATA[266 1409024] csum missing, have: 0, expected: 65536 > ERROR: root 5 INODE[266] nbytes 1437696 not equal to extent_size 1503232 > ERROR: root 5 EXTENT_DATA[4731008 4096] csum missing, have: 0, expected: 2093056 > ERROR: root 5 EXTENT_DATA[4731008 2101248] csum missing, have: 0, expected: 8192 > ERROR: root 5 INODE[4731008] nbytes 73728 not equal to extent_size 2174976 > ERROR: root 5 EXTENT_DATA[4731012 4096] csum missing, have: 0, expected: 8192 > ERROR: root 5 INODE[4731012] nbytes 8192 not equal to extent_size 16384 > ERROR: root 5 EXTENT_DATA[6563647 8192] csum missing, have: 0, expected: 4096 > ERROR: root 5 INODE[6563647] nbytes 12288 not equal to extent_size 16384 > ERROR: root 5 EXTENT_DATA[7090739 131072] csum missing, have: 0, expected: 24576 > ERROR: root 5 INODE[7090739] nbytes 139264 not equal to extent_size 163840 Those are all minor problems, and still no generation problem reported. All these minor problem can also be repaired, or they are just false reported in older lowmem mode. It should be pretty OK to have a v5.4 repair run. Thanks, Qu > ERROR: errors found in fs roots > Opening filesystem to check... > Checking filesystem on /dev/nvme0n1p2 > UUID: 815266d6-a8b9-4f63-a593-02fde178263f > found 89383137280 bytes used, error(s) found > total csum bytes: 85617340 > total tree bytes: 1670774784 > total fs tree bytes: 1451180032 > total extent tree bytes: 107905024 > btree space waste bytes: 413362851 > file data blocks allocated: 90769887232 > referenced 88836960256 > > On Sun, Dec 29, 2019 at 4:46 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >> >> >> >> On 2019/12/30 上午4:43, Patrick Erley wrote: >>> On a system where I've been tinkering with linux-next for years, my / >>> has got some errors. When migrating from 5.1 to 5.2, I saw these >>> errors, but just ignored them and went back to 5.1, and continued my >>> tinkering. Over the holidays, I decided to try to upgrade the kernel, >>> saw the errors again, and decided to look into them, finding the >>> tree-checker page on the kernel docs, and am writing this e-mail. >>> >>> My / does not contain anything sensitive or important, as /home and >>> /usr/src are both subvolumes on a separate larger drive. >>> >>> btrfs fi show: >>> Label: none uuid: 815266d6-a8b9-4f63-a593-02fde178263f >>> Total devices 2 FS bytes used 93.81GiB >>> devid 1 size 115.12GiB used 115.11GiB path /dev/nvme0n1p2 >>> devid 3 size 115.12GiB used 115.11GiB path /dev/sda3 >>> >>> Label: none uuid: 4bd97711-b63c-40cb-bfa5-aa9c2868cf98 >>> Total devices 1 FS bytes used 536.48GiB >>> devid 1 size 834.63GiB used 833.59GiB path /dev/sda5 >>> >>> Booting a more recent kernel, I get spammed with: >>> [ 8.243899] BTRFS critical (device nvme0n1p2): corrupt leaf: root=5 >>> block=303629811712 slot=30 ino=5380870, invalid inode generation: has >>> 13221446351398931016 expect [0, 2997884] >> >> Inode generation repair is introduced in v5.4. So feel free to use >> `btrfs check --repair` to repair such problems. >> >> But please run a `btrfs check` without --repair and paste the output, >> just in case there are extra problems which may screw up repair. >> >> Thanks, >> Qu >> >>> [ 8.243902] BTRFS error (device nvme0n1p2): block=303629811712 read >>> time tree block corruption detected >>> >>> There are 6 corrupted inodes: >>> cat dmesg.foo | grep "BTRFS critical" | sed -re >>> 's:.*block=([0-9]*).*ino=([0-9]+).*:\1 \2:' | sort | uniq >>> 303629811712 5380870 >>> 303712501760 3277548 >>> 303861395456 5909140 >>> 304079065088 2228479 >>> 304573444096 3539224 >>> 305039556608 1442149 >>> >>> and they all have the same value for the inode generation. Before I >>> reboot into a livecd and btrfs check --repair, is there anything >>> interesting that a snapshot of the state would show? I have 800gb >>> unpartitioned on the nvme, so backing up before is already in the >>> plans, and I could just as easily grab an image of the partitions >>> while I'm at it. >>> >> [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 520 bytes --] ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 5:43 ` Qu Wenruo @ 2019-12-30 5:47 ` Patrick Erley 2019-12-30 5:50 ` Patrick Erley 0 siblings, 1 reply; 75+ messages in thread From: Patrick Erley @ 2019-12-30 5:47 UTC (permalink / raw) To: Qu Wenruo; +Cc: Btrfs BTRFS On Sun, Dec 29, 2019 at 9:43 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > On 2019/12/30 下午1:36, Patrick Erley wrote: > > (ugh, just realized gmail does top replies. Sorry... will try to > > figure out how to make gsuite behave like a sane mail client before my > > next reply): > > > > here's btrfs check /dev/nvme0n1p2 (sda3, which is a mirror of it, has > > exactly the same output) > > > > [1/7] checking root items > > [2/7] checking extents > > [3/7] checking free space cache > > [4/7] checking fs roots > > [5/7] checking only csums items (without verifying data) > > [6/7] checking root refs > > [7/7] checking quota groups skipped (not enabled on this FS) > > Opening filesystem to check... > > Checking filesystem on /dev/nvme0n1p2 > > UUID: 815266d6-a8b9-4f63-a593-02fde178263f > > found 89383137280 bytes used, no error found > > total csum bytes: 85617340 > > total tree bytes: 1670774784 > > total fs tree bytes: 1451180032 > > total extent tree bytes: 107905024 > > btree space waste bytes: 413362851 > > file data blocks allocated: 90769887232 > > referenced 88836960256 > > It looks too good to be true, is the btrfs-progs v5.4? IIRC in v5.4 we > should report inodes generation problems. Hurray Bottom Reply? /usr/src/initramfs/bin $ ./btrfs.static --version btrfs-progs v5.4 ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 5:47 ` Patrick Erley @ 2019-12-30 5:50 ` Patrick Erley 2019-12-30 5:58 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Patrick Erley @ 2019-12-30 5:50 UTC (permalink / raw) To: Qu Wenruo; +Cc: Btrfs BTRFS On Sun, Dec 29, 2019 at 9:47 PM Patrick Erley <pat-lkml@erley.org> wrote: > > On Sun, Dec 29, 2019 at 9:43 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > > > > > On 2019/12/30 下午1:36, Patrick Erley wrote: > > > (ugh, just realized gmail does top replies. Sorry... will try to > > > figure out how to make gsuite behave like a sane mail client before my > > > next reply): > > > > > > here's btrfs check /dev/nvme0n1p2 (sda3, which is a mirror of it, has > > > exactly the same output) > > > > > > [1/7] checking root items > > > [2/7] checking extents > > > [3/7] checking free space cache > > > [4/7] checking fs roots > > > [5/7] checking only csums items (without verifying data) > > > [6/7] checking root refs > > > [7/7] checking quota groups skipped (not enabled on this FS) > > > Opening filesystem to check... > > > Checking filesystem on /dev/nvme0n1p2 > > > UUID: 815266d6-a8b9-4f63-a593-02fde178263f > > > found 89383137280 bytes used, no error found > > > total csum bytes: 85617340 > > > total tree bytes: 1670774784 > > > total fs tree bytes: 1451180032 > > > total extent tree bytes: 107905024 > > > btree space waste bytes: 413362851 > > > file data blocks allocated: 90769887232 > > > referenced 88836960256 > > > > It looks too good to be true, is the btrfs-progs v5.4? IIRC in v5.4 we > > should report inodes generation problems. > > Hurray Bottom Reply? > > /usr/src/initramfs/bin $ ./btrfs.static --version > btrfs-progs v5.4 Dumb question, did I need to do that while booting a post 5.1 kernel? I ran these while not having the filesystem mounted, but against kernel 5.1. I can easily repeat against 5.4. ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 5:50 ` Patrick Erley @ 2019-12-30 5:58 ` Qu Wenruo 2019-12-30 6:07 ` Patrick Erley 0 siblings, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2019-12-30 5:58 UTC (permalink / raw) To: Patrick Erley; +Cc: Btrfs BTRFS [-- Attachment #1.1: Type: text/plain, Size: 2950 bytes --] On 2019/12/30 下午1:50, Patrick Erley wrote: > On Sun, Dec 29, 2019 at 9:47 PM Patrick Erley <pat-lkml@erley.org> wrote: >> >> On Sun, Dec 29, 2019 at 9:43 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>> >>> >>> >>> On 2019/12/30 下午1:36, Patrick Erley wrote: >>>> (ugh, just realized gmail does top replies. Sorry... will try to >>>> figure out how to make gsuite behave like a sane mail client before my >>>> next reply): >>>> >>>> here's btrfs check /dev/nvme0n1p2 (sda3, which is a mirror of it, has >>>> exactly the same output) >>>> >>>> [1/7] checking root items >>>> [2/7] checking extents >>>> [3/7] checking free space cache >>>> [4/7] checking fs roots >>>> [5/7] checking only csums items (without verifying data) >>>> [6/7] checking root refs >>>> [7/7] checking quota groups skipped (not enabled on this FS) >>>> Opening filesystem to check... >>>> Checking filesystem on /dev/nvme0n1p2 >>>> UUID: 815266d6-a8b9-4f63-a593-02fde178263f >>>> found 89383137280 bytes used, no error found >>>> total csum bytes: 85617340 >>>> total tree bytes: 1670774784 >>>> total fs tree bytes: 1451180032 >>>> total extent tree bytes: 107905024 >>>> btree space waste bytes: 413362851 >>>> file data blocks allocated: 90769887232 >>>> referenced 88836960256 >>> >>> It looks too good to be true, is the btrfs-progs v5.4? IIRC in v5.4 we >>> should report inodes generation problems. >> >> Hurray Bottom Reply? >> >> /usr/src/initramfs/bin $ ./btrfs.static --version >> btrfs-progs v5.4 This is strange. 6084adam|thinkpad|~$ btrfs check --mode=lowmem test.img Opening filesystem to check... Checking filesystem on test.img UUID: c6c6ddd2-01c1-47fc-b699-cacfae9d4bfd [1/7] checking root items [2/7] checking extents [3/7] checking free space cache cache and super generation don't match, space cache will be invalidated [4/7] checking fs roots ERROR: invalid inode generation for ino 257, have 8858344568388091671 expect [0, 9) ERROR: errors found in fs roots found 131072 bytes used, error(s) found total csum bytes: 0 total tree bytes: 131072 total fs tree bytes: 32768 total extent tree bytes: 16384 btree space waste bytes: 123409 file data blocks allocated: 0 referenced 0 6085adam|thinkpad|~$ btrfs --version btrfs-progs v5.4 As expected, v5.4 should detect such problem without problem. Would you please provide extra tree dump to help us to determine what makes btrfs check unable to detect such problems? # btrfs ins dump-tree -b 303629811712 /dev/dm-1 > > Dumb question, did I need to do that while booting a post 5.1 kernel? > I ran these while not having the filesystem mounted, but against > kernel 5.1. I can easily repeat against 5.4. Kernel version doesn't affect at all. All the detection and repair should be able to be done by btrfs-progs. But it would be better to use v5.4 to quick verify the fix is working. Thanks, Qu [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 520 bytes --] ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 5:58 ` Qu Wenruo @ 2019-12-30 6:07 ` Patrick Erley 2019-12-30 6:09 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Patrick Erley @ 2019-12-30 6:07 UTC (permalink / raw) To: Qu Wenruo; +Cc: Btrfs BTRFS On Sun, Dec 29, 2019 at 9:58 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > On 2019/12/30 下午1:50, Patrick Erley wrote: > > On Sun, Dec 29, 2019 at 9:47 PM Patrick Erley <pat-lkml@erley.org> wrote: > >> > >> On Sun, Dec 29, 2019 at 9:43 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >>> > >>> > >>> > >>> On 2019/12/30 下午1:36, Patrick Erley wrote: > >>>> (ugh, just realized gmail does top replies. Sorry... will try to > >>>> figure out how to make gsuite behave like a sane mail client before my > >>>> next reply): > >>>> > >>>> here's btrfs check /dev/nvme0n1p2 (sda3, which is a mirror of it, has > >>>> exactly the same output) > >>>> > >>>> [1/7] checking root items > >>>> [2/7] checking extents > >>>> [3/7] checking free space cache > >>>> [4/7] checking fs roots > >>>> [5/7] checking only csums items (without verifying data) > >>>> [6/7] checking root refs > >>>> [7/7] checking quota groups skipped (not enabled on this FS) > >>>> Opening filesystem to check... > >>>> Checking filesystem on /dev/nvme0n1p2 > >>>> UUID: 815266d6-a8b9-4f63-a593-02fde178263f > >>>> found 89383137280 bytes used, no error found > >>>> total csum bytes: 85617340 > >>>> total tree bytes: 1670774784 > >>>> total fs tree bytes: 1451180032 > >>>> total extent tree bytes: 107905024 > >>>> btree space waste bytes: 413362851 > >>>> file data blocks allocated: 90769887232 > >>>> referenced 88836960256 > >>> > >>> It looks too good to be true, is the btrfs-progs v5.4? IIRC in v5.4 we > >>> should report inodes generation problems. > >> > >> Hurray Bottom Reply? > >> > >> /usr/src/initramfs/bin $ ./btrfs.static --version > >> btrfs-progs v5.4 > > This is strange. > > > 6084adam|thinkpad|~$ btrfs check --mode=lowmem test.img > Opening filesystem to check... > Checking filesystem on test.img > UUID: c6c6ddd2-01c1-47fc-b699-cacfae9d4bfd > [1/7] checking root items > [2/7] checking extents > [3/7] checking free space cache > cache and super generation don't match, space cache will be invalidated > [4/7] checking fs roots > ERROR: invalid inode generation for ino 257, have 8858344568388091671 > expect [0, 9) > ERROR: errors found in fs roots > found 131072 bytes used, error(s) found > total csum bytes: 0 > total tree bytes: 131072 > total fs tree bytes: 32768 > total extent tree bytes: 16384 > btree space waste bytes: 123409 > file data blocks allocated: 0 > referenced 0 > 6085adam|thinkpad|~$ btrfs --version > btrfs-progs v5.4 > > As expected, v5.4 should detect such problem without problem. > > Would you please provide extra tree dump to help us to determine what > makes btrfs check unable to detect such problems? > > # btrfs ins dump-tree -b 303629811712 /dev/dm-1 anvil ~ # btrfs ins dump-tree -b 303629811712 /dev/nvme0n1p2 btrfs-progs v5.4 Invalid mapping for 303629811712-303629815808, got 476092633088-477166374912 Couldn't map the block 303629811712 Couldn't map the block 303629811712 bad tree block 303629811712, bytenr mismatch, want=303629811712, have=0 ERROR: failed to read tree block 303629811712 ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 6:07 ` Patrick Erley @ 2019-12-30 6:09 ` Qu Wenruo 2019-12-30 8:14 ` Patrick Erley 0 siblings, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2019-12-30 6:09 UTC (permalink / raw) To: Patrick Erley; +Cc: Btrfs BTRFS [-- Attachment #1.1: Type: text/plain, Size: 3490 bytes --] On 2019/12/30 下午2:07, Patrick Erley wrote: > On Sun, Dec 29, 2019 at 9:58 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >> >> >> >> On 2019/12/30 下午1:50, Patrick Erley wrote: >>> On Sun, Dec 29, 2019 at 9:47 PM Patrick Erley <pat-lkml@erley.org> wrote: >>>> >>>> On Sun, Dec 29, 2019 at 9:43 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>>>> >>>>> >>>>> >>>>> On 2019/12/30 下午1:36, Patrick Erley wrote: >>>>>> (ugh, just realized gmail does top replies. Sorry... will try to >>>>>> figure out how to make gsuite behave like a sane mail client before my >>>>>> next reply): >>>>>> >>>>>> here's btrfs check /dev/nvme0n1p2 (sda3, which is a mirror of it, has >>>>>> exactly the same output) >>>>>> >>>>>> [1/7] checking root items >>>>>> [2/7] checking extents >>>>>> [3/7] checking free space cache >>>>>> [4/7] checking fs roots >>>>>> [5/7] checking only csums items (without verifying data) >>>>>> [6/7] checking root refs >>>>>> [7/7] checking quota groups skipped (not enabled on this FS) >>>>>> Opening filesystem to check... >>>>>> Checking filesystem on /dev/nvme0n1p2 >>>>>> UUID: 815266d6-a8b9-4f63-a593-02fde178263f >>>>>> found 89383137280 bytes used, no error found >>>>>> total csum bytes: 85617340 >>>>>> total tree bytes: 1670774784 >>>>>> total fs tree bytes: 1451180032 >>>>>> total extent tree bytes: 107905024 >>>>>> btree space waste bytes: 413362851 >>>>>> file data blocks allocated: 90769887232 >>>>>> referenced 88836960256 >>>>> >>>>> It looks too good to be true, is the btrfs-progs v5.4? IIRC in v5.4 we >>>>> should report inodes generation problems. >>>> >>>> Hurray Bottom Reply? >>>> >>>> /usr/src/initramfs/bin $ ./btrfs.static --version >>>> btrfs-progs v5.4 >> >> This is strange. >> >> >> 6084adam|thinkpad|~$ btrfs check --mode=lowmem test.img >> Opening filesystem to check... >> Checking filesystem on test.img >> UUID: c6c6ddd2-01c1-47fc-b699-cacfae9d4bfd >> [1/7] checking root items >> [2/7] checking extents >> [3/7] checking free space cache >> cache and super generation don't match, space cache will be invalidated >> [4/7] checking fs roots >> ERROR: invalid inode generation for ino 257, have 8858344568388091671 >> expect [0, 9) >> ERROR: errors found in fs roots >> found 131072 bytes used, error(s) found >> total csum bytes: 0 >> total tree bytes: 131072 >> total fs tree bytes: 32768 >> total extent tree bytes: 16384 >> btree space waste bytes: 123409 >> file data blocks allocated: 0 >> referenced 0 >> 6085adam|thinkpad|~$ btrfs --version >> btrfs-progs v5.4 >> >> As expected, v5.4 should detect such problem without problem. >> >> Would you please provide extra tree dump to help us to determine what >> makes btrfs check unable to detect such problems? >> >> # btrfs ins dump-tree -b 303629811712 /dev/dm-1 > anvil ~ # btrfs ins dump-tree -b 303629811712 /dev/nvme0n1p2 > btrfs-progs v5.4 > Invalid mapping for 303629811712-303629815808, got 476092633088-477166374912 > Couldn't map the block 303629811712 > Couldn't map the block 303629811712 > bad tree block 303629811712, bytenr mismatch, want=303629811712, have=0 > ERROR: failed to read tree block 303629811712 > The bytenr is different from your initial report. Anyway, you can try mount with v5.4, and use the bytenr from the dmesg. Then please provide both dmeg (including the "corrupted leaf" line), and the `btrfs ins dump-tree -b` output. Thanks, Qu [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 520 bytes --] ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 6:09 ` Qu Wenruo @ 2019-12-30 8:14 ` Patrick Erley 2019-12-30 8:54 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Patrick Erley @ 2019-12-30 8:14 UTC (permalink / raw) To: Qu Wenruo; +Cc: Btrfs BTRFS On Mon, Dec 30, 2019 at 6:09 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > On 2019/12/30 下午2:07, Patrick Erley wrote: > > On Sun, Dec 29, 2019 at 9:58 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >> > >> > >> > >> On 2019/12/30 下午1:50, Patrick Erley wrote: > >>> On Sun, Dec 29, 2019 at 9:47 PM Patrick Erley <pat-lkml@erley.org> wrote: > >>>> > >>>> On Sun, Dec 29, 2019 at 9:43 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >>>>> > >>>>> > >>>>> > >>>>> On 2019/12/30 下午1:36, Patrick Erley wrote: > >>>>>> (ugh, just realized gmail does top replies. Sorry... will try to > >>>>>> figure out how to make gsuite behave like a sane mail client before my > >>>>>> next reply): > >>>>>> > >>>>>> here's btrfs check /dev/nvme0n1p2 (sda3, which is a mirror of it, has > >>>>>> exactly the same output) > >>>>>> > >>>>>> [1/7] checking root items > >>>>>> [2/7] checking extents > >>>>>> [3/7] checking free space cache > >>>>>> [4/7] checking fs roots > >>>>>> [5/7] checking only csums items (without verifying data) > >>>>>> [6/7] checking root refs > >>>>>> [7/7] checking quota groups skipped (not enabled on this FS) > >>>>>> Opening filesystem to check... > >>>>>> Checking filesystem on /dev/nvme0n1p2 > >>>>>> UUID: 815266d6-a8b9-4f63-a593-02fde178263f > >>>>>> found 89383137280 bytes used, no error found > >>>>>> total csum bytes: 85617340 > >>>>>> total tree bytes: 1670774784 > >>>>>> total fs tree bytes: 1451180032 > >>>>>> total extent tree bytes: 107905024 > >>>>>> btree space waste bytes: 413362851 > >>>>>> file data blocks allocated: 90769887232 > >>>>>> referenced 88836960256 > >>>>> > >>>>> It looks too good to be true, is the btrfs-progs v5.4? IIRC in v5.4 we > >>>>> should report inodes generation problems. > >>>> > >>>> Hurray Bottom Reply? > >>>> > >>>> /usr/src/initramfs/bin $ ./btrfs.static --version > >>>> btrfs-progs v5.4 > >> > >> This is strange. > >> > >> > >> 6084adam|thinkpad|~$ btrfs check --mode=lowmem test.img > >> Opening filesystem to check... > >> Checking filesystem on test.img > >> UUID: c6c6ddd2-01c1-47fc-b699-cacfae9d4bfd > >> [1/7] checking root items > >> [2/7] checking extents > >> [3/7] checking free space cache > >> cache and super generation don't match, space cache will be invalidated > >> [4/7] checking fs roots > >> ERROR: invalid inode generation for ino 257, have 8858344568388091671 > >> expect [0, 9) > >> ERROR: errors found in fs roots > >> found 131072 bytes used, error(s) found > >> total csum bytes: 0 > >> total tree bytes: 131072 > >> total fs tree bytes: 32768 > >> total extent tree bytes: 16384 > >> btree space waste bytes: 123409 > >> file data blocks allocated: 0 > >> referenced 0 > >> 6085adam|thinkpad|~$ btrfs --version > >> btrfs-progs v5.4 > >> > >> As expected, v5.4 should detect such problem without problem. > >> > >> Would you please provide extra tree dump to help us to determine what > >> makes btrfs check unable to detect such problems? > >> > >> # btrfs ins dump-tree -b 303629811712 /dev/dm-1 > > anvil ~ # btrfs ins dump-tree -b 303629811712 /dev/nvme0n1p2 > > btrfs-progs v5.4 > > Invalid mapping for 303629811712-303629815808, got 476092633088-477166374912 > > Couldn't map the block 303629811712 > > Couldn't map the block 303629811712 > > bad tree block 303629811712, bytenr mismatch, want=303629811712, have=0 > > ERROR: failed to read tree block 303629811712 > > > The bytenr is different from your initial report. > > Anyway, you can try mount with v5.4, and use the bytenr from the dmesg. > > Then please provide both dmeg (including the "corrupted leaf" line), and > the `btrfs ins dump-tree -b` output. > > Thanks, > Qu > Good news and bad news. Running btrfs check --repair fixed the errors listed in $subject, but seems to have fully hosed te filesystem. I'm now getting: ./btrfs.static check /dev/nvme0n1p2 Opening filesystem to check... Checking filesystem on /dev/nvme0n1p2 UUID: 815266d6-a8b9-4f63-a593-02fde178263f [1/7] checking root items parent transid verify failed on 499774480384 wanted 3323349 found 3323340 parent transid verify failed on 499774386176 wanted 3323349 found 3323340 parent transid verify failed on 499774394368 wanted 3323349 found 3323340 parent transid verify failed on 499774398464 wanted 3323349 found 3323340 parent transid verify failed on 499774406656 wanted 3323349 found 3323340 parent transid verify failed on 499774423040 wanted 3323349 found 3323340 parent transid verify failed on 499774427136 wanted 3323349 found 3323340 parent transid verify failed on 499774439424 wanted 3323349 found 3323340 parent transid verify failed on 499774324736 wanted 3323349 found 3323340 parent transid verify failed on 499774513152 wanted 3323349 found 3323340 parent transid verify failed on 499774341120 wanted 3323349 found 3323340 [2/7] checking extents leaf parent key incorrect 499774554112 bad block 499774554112 ERROR: errors found in extent allocation tree or chunk allocation [3/7] checking free space cache there is no free space entry for 499922907136-499952799744 there is no free space entry for 499922907136-500822249472 cache appears valid but isn't 499748507648 [4/7] checking fs roots Wrong key of child node/leaf, wanted: (1442047, 1, 0), have: (523365974016, 168, 16384) root 5 inode 1442047 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name etc filetype 2 errors 6, no dir index, no inode ref root 5 inode 1835263 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name bin filetype 2 errors 6, no dir index, no inode ref root 5 inode 1966335 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name proc filetype 2 errors 6, no dir index, no inode ref root 5 inode 2097407 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 5 name lib32 filetype 2 errors 6, no dir index, no inode ref root 5 inode 2228479 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name sys filetype 2 errors 6, no dir index, no inode ref root 5 inode 2359551 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 5 name lib64 filetype 2 errors 6, no dir index, no inode ref root 5 inode 3145983 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name dev filetype 2 errors 6, no dir index, no inode ref root 5 inode 3277055 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name opt filetype 2 errors 6, no dir index, no inode ref root 5 inode 3408127 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name mnt filetype 2 errors 6, no dir index, no inode ref root 5 inode 3539199 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name usr filetype 2 errors 6, no dir index, no inode ref root 5 inode 3670271 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 5 name media filetype 2 errors 6, no dir index, no inode ref root 5 inode 4587775 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name sbin filetype 2 errors 6, no dir index, no inode ref root 5 inode 4980991 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name etc2 filetype 2 errors 6, no dir index, no inode ref root 5 inode 4984678 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name run filetype 2 errors 6, no dir index, no inode ref root 5 inode 5505279 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 7 name exports filetype 2 errors 6, no dir index, no inode ref root 5 inode 5767423 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 5 name debug filetype 2 errors 6, no dir index, no inode ref root 5 inode 5898495 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name etc3 filetype 2 errors 6, no dir index, no inode ref root 5 inode 6029567 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name root filetype 2 errors 6, no dir index, no inode ref root 5 inode 6431345 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name boot filetype 2 errors 6, no dir index, no inode ref root 5 inode 6553855 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name var filetype 2 errors 6, no dir index, no inode ref root 5 inode 7602431 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name tmp filetype 2 errors 6, no dir index, no inode ref root 5 inode 9133291 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 13 name .pulse-cookie filetype 1 errors 6, no dir index, no inode ref root 5 inode 9133292 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 6 name .pulse filetype 2 errors 6, no dir index, no inode ref root 5 inode 9980485 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 9 name bootchart filetype 2 errors 6, no dir index, no inode ref root 5 inode 10254180 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name temp filetype 2 errors 6, no dir index, no inode ref root 5 inode 11827458 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 8 name tftpboot filetype 2 errors 6, no dir index, no inode ref root 5 inode 14435353 errors 2001, no inode item, link count wrong unresolved ref dir 131327 index 0 namelen 4 name s0be filetype 2 errors 6, no dir index, no inode ref root 5 inode 14646004 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name s0be filetype 2 errors 6, no dir index, no inode ref root 5 inode 14910590 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 13 name .bash_history filetype 1 errors 6, no dir index, no inode ref root 5 inode 23438027 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 8 name mknod.sh filetype 1 errors 6, no dir index, no inode ref root 5 inode 25548332 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 8 name ctrl_dir filetype 1 errors 6, no dir index, no inode ref root 5 inode 25548333 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 5 name state filetype 1 errors 6, no dir index, no inode ref root 5 inode 26407893 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 6 name e2fsck filetype 1 errors 6, no dir index, no inode ref root 5 inode 36767707 errors 2001, no inode item, link count wrong unresolved ref dir 924630 index 0 namelen 31 name lib_mysqludf_log-warnings.patch filetype 1 errors 6, no dir index, no inode ref root 5 inode 36767712 errors 2001, no inode item, link count wrong unresolved ref dir 924632 index 0 namelen 32 name lib_mysqludf_stem-mysql_m4.patch filetype 1 errors 6, no dir index, no inode ref root 5 inode 36767838 errors 2001, no inode item, link count wrong unresolved ref dir 924633 index 0 namelen 33 name mysql-udf-base64-signedness.patch filetype 1 errors 6, no dir index, no inode ref root 5 inode 36767839 errors 2001, no inode item, link count wrong unresolved ref dir 924633 index 0 namelen 20 name mysql-udf-base64.sql filetype 1 errors 6, no dir index, no inode ref root 5 inode 36767840 errors 2001, no inode item, link count wrong unresolved ref dir 924634 index 0 namelen 27 name mysql-udf-http-stdlib.patch filetype 1 errors 6, no dir index, no inode ref root 5 inode 36767843 errors 2001, no inode item, link count wrong unresolved ref dir 924635 index 0 namelen 29 name mysql-udf-ipv6-warnings.patch filetype 1 errors 6, no dir index, no inode ref root 5 inode 37030378 errors 2001, no inode item, link count wrong unresolved ref dir 924616 index 0 namelen 12 name metadata.xml filetype 1 errors 6, no dir index, no inode ref root 5 inode 37030384 errors 2001, no inode item, link count wrong unresolved ref dir 924619 index 0 namelen 12 name metadata.xml filetype 1 errors 6, no dir index, no inode ref root 5 inode 37030490 errors 2001, no inode item, link count wrong unresolved ref dir 924625 index 0 namelen 12 name metadata.xml filetype 1 errors 6, no dir index, no inode ref root 5 inode 37030492 errors 2001, no inode item, link count wrong unresolved ref dir 924626 index 0 namelen 12 name metadata.xml filetype 1 errors 6, no dir index, no inode ref root 5 inode 37030497 errors 2001, no inode item, link count wrong unresolved ref dir 924628 index 0 namelen 12 name metadata.xml filetype 1 errors 6, no dir index, no inode ref root 5 inode 39198776 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 8 name bootback filetype 2 errors 6, no dir index, no inode ref root 5 inode 51520157 errors 2001, no inode item, link count wrong unresolved ref dir 924616 index 0 namelen 29 name lib_mysqludf_log-0.0.2.ebuild filetype 1 errors 6, no dir index, no inode ref root 5 inode 51520163 errors 2001, no inode item, link count wrong unresolved ref dir 924619 index 0 namelen 30 name lib_mysqludf_stem-0.9.1.ebuild filetype 1 errors 6, no dir index, no inode ref root 5 inode 51520278 errors 2001, no inode item, link count wrong unresolved ref dir 924625 index 0 namelen 32 name mysql-udf-base64-20010618.ebuild filetype 1 errors 6, no dir index, no inode ref root 5 inode 51520280 errors 2001, no inode item, link count wrong unresolved ref dir 924626 index 0 namelen 25 name mysql-udf-http-1.0.ebuild filetype 1 errors 6, no dir index, no inode ref root 5 inode 51520284 errors 2001, no inode item, link count wrong unresolved ref dir 924628 index 0 namelen 25 name mysql-udf-ipv6-0.4.ebuild filetype 1 errors 6, no dir index, no inode ref root 5 inode 58792107 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name foo filetype 1 errors 6, no dir index, no inode ref root 5 inode 63681251 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name xpra filetype 2 errors 6, no dir index, no inode ref root 5 inode 66715793 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 8 name .viminfo filetype 1 errors 6, no dir index, no inode ref root 5 inode 71469207 errors 2001, no inode item, link count wrong unresolved ref dir 924616 index 0 namelen 8 name Manifest filetype 1 errors 6, no dir index, no inode ref root 5 inode 71469208 errors 2001, no inode item, link count wrong unresolved ref dir 924619 index 0 namelen 8 name Manifest filetype 1 errors 6, no dir index, no inode ref root 5 inode 71469238 errors 2001, no inode item, link count wrong unresolved ref dir 924625 index 0 namelen 8 name Manifest filetype 1 errors 6, no dir index, no inode ref root 5 inode 71469239 errors 2001, no inode item, link count wrong unresolved ref dir 924626 index 0 namelen 8 name Manifest filetype 1 errors 6, no dir index, no inode ref root 5 inode 71469240 errors 2001, no inode item, link count wrong unresolved ref dir 924628 index 0 namelen 8 name Manifest filetype 1 errors 6, no dir index, no inode ref root 5 inode 75334150 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 7 name lib.new filetype 2 errors 6, no dir index, no inode ref root 5 inode 77351088 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name lib filetype 7 errors 6, no dir index, no inode ref ERROR: errors found in fs roots found 951992320 bytes used, error(s) found total csum bytes: 0 total tree bytes: 3895296 total fs tree bytes: 0 total extent tree bytes: 3813376 btree space waste bytes: 1560917 file data blocks allocated: 27983872 referenced 27983872 Should I also paste in the repair log? ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 8:14 ` Patrick Erley @ 2019-12-30 8:54 ` Qu Wenruo 2019-12-30 9:01 ` Patrick Erley 0 siblings, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2019-12-30 8:54 UTC (permalink / raw) To: Patrick Erley; +Cc: Btrfs BTRFS [-- Attachment #1.1: Type: text/plain, Size: 17174 bytes --] On 2019/12/30 下午4:14, Patrick Erley wrote: > On Mon, Dec 30, 2019 at 6:09 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >> >> >> >> On 2019/12/30 下午2:07, Patrick Erley wrote: >>> On Sun, Dec 29, 2019 at 9:58 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>>> >>>> >>>> >>>> On 2019/12/30 下午1:50, Patrick Erley wrote: >>>>> On Sun, Dec 29, 2019 at 9:47 PM Patrick Erley <pat-lkml@erley.org> wrote: >>>>>> >>>>>> On Sun, Dec 29, 2019 at 9:43 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 2019/12/30 下午1:36, Patrick Erley wrote: >>>>>>>> (ugh, just realized gmail does top replies. Sorry... will try to >>>>>>>> figure out how to make gsuite behave like a sane mail client before my >>>>>>>> next reply): >>>>>>>> >>>>>>>> here's btrfs check /dev/nvme0n1p2 (sda3, which is a mirror of it, has >>>>>>>> exactly the same output) >>>>>>>> >>>>>>>> [1/7] checking root items >>>>>>>> [2/7] checking extents >>>>>>>> [3/7] checking free space cache >>>>>>>> [4/7] checking fs roots >>>>>>>> [5/7] checking only csums items (without verifying data) >>>>>>>> [6/7] checking root refs >>>>>>>> [7/7] checking quota groups skipped (not enabled on this FS) >>>>>>>> Opening filesystem to check... >>>>>>>> Checking filesystem on /dev/nvme0n1p2 >>>>>>>> UUID: 815266d6-a8b9-4f63-a593-02fde178263f >>>>>>>> found 89383137280 bytes used, no error found >>>>>>>> total csum bytes: 85617340 >>>>>>>> total tree bytes: 1670774784 >>>>>>>> total fs tree bytes: 1451180032 >>>>>>>> total extent tree bytes: 107905024 >>>>>>>> btree space waste bytes: 413362851 >>>>>>>> file data blocks allocated: 90769887232 >>>>>>>> referenced 88836960256 >>>>>>> >>>>>>> It looks too good to be true, is the btrfs-progs v5.4? IIRC in v5.4 we >>>>>>> should report inodes generation problems. >>>>>> >>>>>> Hurray Bottom Reply? >>>>>> >>>>>> /usr/src/initramfs/bin $ ./btrfs.static --version >>>>>> btrfs-progs v5.4 >>>> >>>> This is strange. >>>> >>>> >>>> 6084adam|thinkpad|~$ btrfs check --mode=lowmem test.img >>>> Opening filesystem to check... >>>> Checking filesystem on test.img >>>> UUID: c6c6ddd2-01c1-47fc-b699-cacfae9d4bfd >>>> [1/7] checking root items >>>> [2/7] checking extents >>>> [3/7] checking free space cache >>>> cache and super generation don't match, space cache will be invalidated >>>> [4/7] checking fs roots >>>> ERROR: invalid inode generation for ino 257, have 8858344568388091671 >>>> expect [0, 9) >>>> ERROR: errors found in fs roots >>>> found 131072 bytes used, error(s) found >>>> total csum bytes: 0 >>>> total tree bytes: 131072 >>>> total fs tree bytes: 32768 >>>> total extent tree bytes: 16384 >>>> btree space waste bytes: 123409 >>>> file data blocks allocated: 0 >>>> referenced 0 >>>> 6085adam|thinkpad|~$ btrfs --version >>>> btrfs-progs v5.4 >>>> >>>> As expected, v5.4 should detect such problem without problem. >>>> >>>> Would you please provide extra tree dump to help us to determine what >>>> makes btrfs check unable to detect such problems? >>>> >>>> # btrfs ins dump-tree -b 303629811712 /dev/dm-1 >>> anvil ~ # btrfs ins dump-tree -b 303629811712 /dev/nvme0n1p2 >>> btrfs-progs v5.4 >>> Invalid mapping for 303629811712-303629815808, got 476092633088-477166374912 >>> Couldn't map the block 303629811712 >>> Couldn't map the block 303629811712 >>> bad tree block 303629811712, bytenr mismatch, want=303629811712, have=0 >>> ERROR: failed to read tree block 303629811712 >>> >> The bytenr is different from your initial report. >> >> Anyway, you can try mount with v5.4, and use the bytenr from the dmesg. >> >> Then please provide both dmeg (including the "corrupted leaf" line), and >> the `btrfs ins dump-tree -b` output. >> >> Thanks, >> Qu >> > > Good news and bad news. Running btrfs check --repair fixed the errors > listed in $subject, but seems to have fully hosed te filesystem. I'm > now getting: > > ./btrfs.static check /dev/nvme0n1p2 > Opening filesystem to check... > Checking filesystem on /dev/nvme0n1p2 > UUID: 815266d6-a8b9-4f63-a593-02fde178263f > [1/7] checking root items > parent transid verify failed on 499774480384 wanted 3323349 found 3323340 > parent transid verify failed on 499774386176 wanted 3323349 found 3323340 > parent transid verify failed on 499774394368 wanted 3323349 found 3323340 > parent transid verify failed on 499774398464 wanted 3323349 found 3323340 > parent transid verify failed on 499774406656 wanted 3323349 found 3323340 > parent transid verify failed on 499774423040 wanted 3323349 found 3323340 > parent transid verify failed on 499774427136 wanted 3323349 found 3323340 > parent transid verify failed on 499774439424 wanted 3323349 found 3323340 > parent transid verify failed on 499774324736 wanted 3323349 found 3323340 > parent transid verify failed on 499774513152 wanted 3323349 found 3323340 > parent transid verify failed on 499774341120 wanted 3323349 found 3323340 > [2/7] checking extents > leaf parent key incorrect 499774554112 > bad block 499774554112 > ERROR: errors found in extent allocation tree or chunk allocation > [3/7] checking free space cache > there is no free space entry for 499922907136-499952799744 > there is no free space entry for 499922907136-500822249472 > cache appears valid but isn't 499748507648 > [4/7] checking fs roots > Wrong key of child node/leaf, wanted: (1442047, 1, 0), have: > (523365974016, 168, 16384) > root 5 inode 1442047 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name etc filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 1835263 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name bin filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 1966335 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name proc filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 2097407 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 5 name lib32 filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 2228479 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name sys filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 2359551 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 5 name lib64 filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 3145983 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name dev filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 3277055 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name opt filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 3408127 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name mnt filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 3539199 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name usr filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 3670271 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 5 name media filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 4587775 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name sbin filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 4980991 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name etc2 filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 4984678 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name run filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 5505279 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 7 name exports filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 5767423 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 5 name debug filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 5898495 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name etc3 filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 6029567 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name root filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 6431345 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name boot filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 6553855 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name var filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 7602431 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name tmp filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 9133291 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 13 name .pulse-cookie > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 9133292 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 6 name .pulse filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 9980485 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 9 name bootchart filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 10254180 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name temp filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 11827458 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 8 name tftpboot filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 14435353 errors 2001, no inode item, link count wrong > unresolved ref dir 131327 index 0 namelen 4 name s0be filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 14646004 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name s0be filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 14910590 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 13 name .bash_history > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 23438027 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 8 name mknod.sh filetype 1 > errors 6, no dir index, no inode ref > root 5 inode 25548332 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 8 name ctrl_dir filetype 1 > errors 6, no dir index, no inode ref > root 5 inode 25548333 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 5 name state filetype 1 > errors 6, no dir index, no inode ref > root 5 inode 26407893 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 6 name e2fsck filetype 1 > errors 6, no dir index, no inode ref > root 5 inode 36767707 errors 2001, no inode item, link count wrong > unresolved ref dir 924630 index 0 namelen 31 name > lib_mysqludf_log-warnings.patch filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 36767712 errors 2001, no inode item, link count wrong > unresolved ref dir 924632 index 0 namelen 32 name > lib_mysqludf_stem-mysql_m4.patch filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 36767838 errors 2001, no inode item, link count wrong > unresolved ref dir 924633 index 0 namelen 33 name > mysql-udf-base64-signedness.patch filetype 1 errors 6, no dir index, > no inode ref > root 5 inode 36767839 errors 2001, no inode item, link count wrong > unresolved ref dir 924633 index 0 namelen 20 name > mysql-udf-base64.sql filetype 1 errors 6, no dir index, no inode ref > root 5 inode 36767840 errors 2001, no inode item, link count wrong > unresolved ref dir 924634 index 0 namelen 27 name > mysql-udf-http-stdlib.patch filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 36767843 errors 2001, no inode item, link count wrong > unresolved ref dir 924635 index 0 namelen 29 name > mysql-udf-ipv6-warnings.patch filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 37030378 errors 2001, no inode item, link count wrong > unresolved ref dir 924616 index 0 namelen 12 name metadata.xml > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 37030384 errors 2001, no inode item, link count wrong > unresolved ref dir 924619 index 0 namelen 12 name metadata.xml > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 37030490 errors 2001, no inode item, link count wrong > unresolved ref dir 924625 index 0 namelen 12 name metadata.xml > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 37030492 errors 2001, no inode item, link count wrong > unresolved ref dir 924626 index 0 namelen 12 name metadata.xml > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 37030497 errors 2001, no inode item, link count wrong > unresolved ref dir 924628 index 0 namelen 12 name metadata.xml > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 39198776 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 8 name bootback filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 51520157 errors 2001, no inode item, link count wrong > unresolved ref dir 924616 index 0 namelen 29 name > lib_mysqludf_log-0.0.2.ebuild filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 51520163 errors 2001, no inode item, link count wrong > unresolved ref dir 924619 index 0 namelen 30 name > lib_mysqludf_stem-0.9.1.ebuild filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 51520278 errors 2001, no inode item, link count wrong > unresolved ref dir 924625 index 0 namelen 32 name > mysql-udf-base64-20010618.ebuild filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 51520280 errors 2001, no inode item, link count wrong > unresolved ref dir 924626 index 0 namelen 25 name > mysql-udf-http-1.0.ebuild filetype 1 errors 6, no dir index, no inode > ref > root 5 inode 51520284 errors 2001, no inode item, link count wrong > unresolved ref dir 924628 index 0 namelen 25 name > mysql-udf-ipv6-0.4.ebuild filetype 1 errors 6, no dir index, no inode > ref > root 5 inode 58792107 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name foo filetype 1 > errors 6, no dir index, no inode ref > root 5 inode 63681251 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name xpra filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 66715793 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 8 name .viminfo filetype 1 > errors 6, no dir index, no inode ref > root 5 inode 71469207 errors 2001, no inode item, link count wrong > unresolved ref dir 924616 index 0 namelen 8 name Manifest filetype > 1 errors 6, no dir index, no inode ref > root 5 inode 71469208 errors 2001, no inode item, link count wrong > unresolved ref dir 924619 index 0 namelen 8 name Manifest filetype > 1 errors 6, no dir index, no inode ref > root 5 inode 71469238 errors 2001, no inode item, link count wrong > unresolved ref dir 924625 index 0 namelen 8 name Manifest filetype > 1 errors 6, no dir index, no inode ref > root 5 inode 71469239 errors 2001, no inode item, link count wrong > unresolved ref dir 924626 index 0 namelen 8 name Manifest filetype > 1 errors 6, no dir index, no inode ref > root 5 inode 71469240 errors 2001, no inode item, link count wrong > unresolved ref dir 924628 index 0 namelen 8 name Manifest filetype > 1 errors 6, no dir index, no inode ref > root 5 inode 75334150 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 7 name lib.new filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 77351088 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name lib filetype 7 > errors 6, no dir index, no inode ref > ERROR: errors found in fs roots > found 951992320 bytes used, error(s) found > total csum bytes: 0 > total tree bytes: 3895296 > total fs tree bytes: 0 > total extent tree bytes: 3813376 > btree space waste bytes: 1560917 > file data blocks allocated: 27983872 > referenced 27983872 > > Should I also paste in the repair log? > Yes please. This sounds very strange, especially for the transid mismatch part. Thanks, Qu [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 520 bytes --] ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 8:54 ` Qu Wenruo @ 2019-12-30 9:01 ` Patrick Erley 2019-12-30 9:09 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Patrick Erley @ 2019-12-30 9:01 UTC (permalink / raw) To: Qu Wenruo; +Cc: Btrfs BTRFS On Mon, Dec 30, 2019 at 8:54 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > On 2019/12/30 下午4:14, Patrick Erley wrote: > > On Mon, Dec 30, 2019 at 6:09 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > Should I also paste in the repair log? > > > Yes please. > > This sounds very strange, especially for the transid mismatch part. > > Thanks, > Qu > enabling repair mode WARNING: Do not use --repair unless you are advised to do so by a developer or an experienced user, and then only after having accepted that no fsck can successfully repair all types of filesystem corruption. Eg. some software or hardware bugs can fatally damage a volume. The operation will start in 10 seconds. Use Ctrl-C to stop it. 10 9 8 7 6 5 4 3 2 1parent transid verify failed on 499774464000 wanted 3323349 found 3323340 parent transid verify failed on 499774521344 wanted 3323349 found 3323340 parent transid verify failed on 499774529536 wanted 3323349 found 3323340 [1/7] checking root items parent transid verify failed on 499774373888 wanted 3323349 found 3323340 parent transid verify failed on 499774480384 wanted 3323349 found 3323340 parent transid verify failed on 499774386176 wanted 3323349 found 3323340 parent transid verify failed on 499774390272 wanted 3323349 found 3323340 parent transid verify failed on 499774394368 wanted 3323349 found 3323340 parent transid verify failed on 499774398464 wanted 3323349 found 3323340 parent transid verify failed on 499774402560 wanted 3323349 found 3323340 parent transid verify failed on 499774406656 wanted 3323349 found 3323340 parent transid verify failed on 499774423040 wanted 3323349 found 3323340 parent transid verify failed on 499774427136 wanted 3323349 found 3323340 parent transid verify failed on 499774435328 wanted 3323349 found 3323340 parent transid verify failed on 499774439424 wanted 3323349 found 3323340 parent transid verify failed on 499774320640 wanted 3323349 found 3323340 parent transid verify failed on 499774324736 wanted 3323349 found 3323340 parent transid verify failed on 499774443520 wanted 3323349 found 3323340 parent transid verify failed on 499774447616 wanted 3323349 found 3323340 parent transid verify failed on 499774504960 wanted 3323349 found 3323340 parent transid verify failed on 499774513152 wanted 3323349 found 3323340 parent transid verify failed on 499774468096 wanted 3323349 found 3323340 parent transid verify failed on 499774337024 wanted 3323349 found 3323340 parent transid verify failed on 499774341120 wanted 3323349 found 3323340 parent transid verify failed on 499774517248 wanted 3323349 found 3323340 parent transid verify failed on 499774558208 wanted 3323349 found 3323340 Fixed 0 roots. [2/7] checking extents extent back ref already exists for 499952799744 parent 0 root 2 extent back ref already exists for 499952803840 parent 0 root 2 extent back ref already exists for 499952807936 parent 0 root 2 extent back ref already exists for 499952812032 parent 0 root 2 extent back ref already exists for 499952816128 parent 0 root 2 extent back ref already exists for 499952824320 parent 0 root 5 extent back ref already exists for 499952832512 parent 0 root 2 extent back ref already exists for 499952852992 parent 0 root 2 extent back ref already exists for 499952857088 parent 0 root 2 extent back ref already exists for 499952861184 parent 0 root 2 extent back ref already exists for 499952877568 parent 0 root 2 extent back ref already exists for 499952885760 parent 0 root 2 extent back ref already exists for 499952889856 parent 0 root 2 extent back ref already exists for 499952898048 parent 0 root 2 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499774554112 leaf parent key incorrect 499773358080 bad block 499773358080 ERROR: errors found in extent allocation tree or chunk allocation [3/7] checking free space cache [4/7] checking fs roots parent transid verify failed on 499774283776 wanted 3323349 found 3323340 Wrong key of child node/leaf, wanted: (1442047, 1, 0), have: (523365974016, 168, 16384) ERROR: child eb corrupted: parent bytenr=499750891520 item=2 parent level=2 child level=0 root 5 inode 1442047 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name etc filetype 2 errors 6, no dir index, no inode ref root 5 inode 1835263 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name bin filetype 2 errors 6, no dir index, no inode ref root 5 inode 1966335 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name proc filetype 2 errors 6, no dir index, no inode ref root 5 inode 2097407 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 5 name lib32 filetype 2 errors 6, no dir index, no inode ref root 5 inode 2228479 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name sys filetype 2 errors 6, no dir index, no inode ref root 5 inode 2359551 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 5 name lib64 filetype 2 errors 6, no dir index, no inode ref root 5 inode 3145983 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name dev filetype 2 errors 6, no dir index, no inode ref root 5 inode 3277055 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name opt filetype 2 errors 6, no dir index, no inode ref root 5 inode 3408127 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name mnt filetype 2 errors 6, no dir index, no inode ref root 5 inode 3539199 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name usr filetype 2 errors 6, no dir index, no inode ref root 5 inode 3670271 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 5 name media filetype 2 errors 6, no dir index, no inode ref root 5 inode 4587775 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name sbin filetype 2 errors 6, no dir index, no inode ref root 5 inode 4980991 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name etc2 filetype 2 errors 6, no dir index, no inode ref root 5 inode 4984678 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name run filetype 2 errors 6, no dir index, no inode ref root 5 inode 5505279 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 7 name exports filetype 2 errors 6, no dir index, no inode ref root 5 inode 5767423 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 5 name debug filetype 2 errors 6, no dir index, no inode ref root 5 inode 5898495 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name etc3 filetype 2 errors 6, no dir index, no inode ref root 5 inode 6029567 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name root filetype 2 errors 6, no dir index, no inode ref root 5 inode 6431345 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name boot filetype 2 errors 6, no dir index, no inode ref ERROR: child eb corrupted: parent bytenr=499762622464 item=37 parent level=2 child level=0 root 5 inode 6553855 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name var filetype 2 errors 6, no dir index, no inode ref root 5 inode 7602431 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name tmp filetype 2 errors 6, no dir index, no inode ref ERROR: child eb corrupted: parent bytenr=499752206336 item=14 parent level=3 child level=0 root 5 inode 9133291 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 13 name .pulse-cookie filetype 1 errors 6, no dir index, no inode ref ERROR: child eb corrupted: parent bytenr=499748732928 item=14 parent level=3 child level=0 root 5 inode 9133292 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 6 name .pulse filetype 2 errors 6, no dir index, no inode ref ERROR: child eb corrupted: parent bytenr=499752206336 item=14 parent level=3 child level=0 root 5 inode 9980485 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 9 name bootchart filetype 2 errors 6, no dir index, no inode ref root 5 inode 10254180 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name temp filetype 2 errors 6, no dir index, no inode ref root 5 inode 11827458 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 8 name tftpboot filetype 2 errors 6, no dir index, no inode ref ERROR: child eb corrupted: parent bytenr=499748732928 item=17 parent level=3 child level=0 root 5 inode 14435353 errors 2001, no inode item, link count wrong unresolved ref dir 131327 index 0 namelen 4 name s0be filetype 2 errors 6, no dir index, no inode ref ERROR: child eb corrupted: parent bytenr=499752206336 item=17 parent level=3 child level=0 root 5 inode 14646004 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name s0be filetype 2 errors 6, no dir index, no inode ref ERROR: child eb corrupted: parent bytenr=499748732928 item=17 parent level=3 child level=0 root 5 inode 14910590 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 13 name .bash_history filetype 1 errors 6, no dir index, no inode ref root 5 inode 23438027 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 8 name mknod.sh filetype 1 errors 6, no dir index, no inode ref root 5 inode 25548332 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 8 name ctrl_dir filetype 1 errors 6, no dir index, no inode ref root 5 inode 25548333 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 5 name state filetype 1 errors 6, no dir index, no inode ref root 5 inode 26407893 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 6 name e2fsck filetype 1 errors 6, no dir index, no inode ref root 5 inode 36767707 errors 2001, no inode item, link count wrong unresolved ref dir 924630 index 0 namelen 31 name lib_mysqludf_log-warnings.patch filetype 1 errors 6, no dir index, no inode ref root 5 inode 36767712 errors 2001, no inode item, link count wrong unresolved ref dir 924632 index 0 namelen 32 name lib_mysqludf_stem-mysql_m4.patch filetype 1 errors 6, no dir index, no inode ref root 5 inode 36767838 errors 2001, no inode item, link count wrong unresolved ref dir 924633 index 0 namelen 33 name mysql-udf-base64-signedness.patch filetype 1 errors 6, no dir index, no inode ref root 5 inode 36767839 errors 2001, no inode item, link count wrong unresolved ref dir 924633 index 0 namelen 20 name mysql-udf-base64.sql filetype 1 errors 6, no dir index, no inode ref root 5 inode 36767840 errors 2001, no inode item, link count wrong unresolved ref dir 924634 index 0 namelen 27 name mysql-udf-http-stdlib.patch filetype 1 errors 6, no dir index, no inode ref root 5 inode 36767843 errors 2001, no inode item, link count wrong unresolved ref dir 924635 index 0 namelen 29 name mysql-udf-ipv6-warnings.patch filetype 1 errors 6, no dir index, no inode ref root 5 inode 37030378 errors 2001, no inode item, link count wrong unresolved ref dir 924616 index 0 namelen 12 name metadata.xml filetype 1 errors 6, no dir index, no inode ref root 5 inode 37030384 errors 2001, no inode item, link count wrong unresolved ref dir 924619 index 0 namelen 12 name metadata.xml filetype 1 errors 6, no dir index, no inode ref root 5 inode 37030490 errors 2001, no inode item, link count wrong unresolved ref dir 924625 index 0 namelen 12 name metadata.xml filetype 1 errors 6, no dir index, no inode ref root 5 inode 37030492 errors 2001, no inode item, link count wrong unresolved ref dir 924626 index 0 namelen 12 name metadata.xml filetype 1 errors 6, no dir index, no inode ref root 5 inode 37030497 errors 2001, no inode item, link count wrong unresolved ref dir 924628 index 0 namelen 12 name metadata.xml filetype 1 errors 6, no dir index, no inode ref root 5 inode 39198776 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 8 name bootback filetype 2 errors 6, no dir index, no inode ref root 5 inode 51520157 errors 2001, no inode item, link count wrong unresolved ref dir 924616 index 0 namelen 29 name lib_mysqludf_log-0.0.2.ebuild filetype 1 errors 6, no dir index, no inode ref root 5 inode 51520163 errors 2001, no inode item, link count wrong unresolved ref dir 924619 index 0 namelen 30 name lib_mysqludf_stem-0.9.1.ebuild filetype 1 errors 6, no dir index, no inode ref root 5 inode 51520278 errors 2001, no inode item, link count wrong unresolved ref dir 924625 index 0 namelen 32 name mysql-udf-base64-20010618.ebuild filetype 1 errors 6, no dir index, no inode ref root 5 inode 51520280 errors 2001, no inode item, link count wrong unresolved ref dir 924626 index 0 namelen 25 name mysql-udf-http-1.0.ebuild filetype 1 errors 6, no dir index, no inode ref root 5 inode 51520284 errors 2001, no inode item, link count wrong unresolved ref dir 924628 index 0 namelen 25 name mysql-udf-ipv6-0.4.ebuild filetype 1 errors 6, no dir index, no inode ref root 5 inode 58792107 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name foo filetype 1 errors 6, no dir index, no inode ref root 5 inode 63681251 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 4 name xpra filetype 2 errors 6, no dir index, no inode ref root 5 inode 66715793 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 8 name .viminfo filetype 1 errors 6, no dir index, no inode ref root 5 inode 71469207 errors 2001, no inode item, link count wrong unresolved ref dir 924616 index 0 namelen 8 name Manifest filetype 1 errors 6, no dir index, no inode ref root 5 inode 71469208 errors 2001, no inode item, link count wrong unresolved ref dir 924619 index 0 namelen 8 name Manifest filetype 1 errors 6, no dir index, no inode ref root 5 inode 71469238 errors 2001, no inode item, link count wrong unresolved ref dir 924625 index 0 namelen 8 name Manifest filetype 1 errors 6, no dir index, no inode ref root 5 inode 71469239 errors 2001, no inode item, link count wrong unresolved ref dir 924626 index 0 namelen 8 name Manifest filetype 1 errors 6, no dir index, no inode ref root 5 inode 71469240 errors 2001, no inode item, link count wrong unresolved ref dir 924628 index 0 namelen 8 name Manifest filetype 1 errors 6, no dir index, no inode ref ERROR: child eb corrupted: parent bytenr=499748732928 item=65 parent level=3 child level=1 root 5 inode 75334150 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 7 name lib.new filetype 2 errors 6, no dir index, no inode ref parent transid verify failed on 499774287872 wanted 3323349 found 3323340 parent transid verify failed on 499774291968 wanted 3323349 found 3323340 parent transid verify failed on 499774296064 wanted 3323349 found 3323340 root 5 inode 77351088 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 0 namelen 3 name lib filetype 7 errors 6, no dir index, no inode ref ERROR: errors found in fs roots Starting repair. Opening filesystem to check... Checking filesystem on /dev/sda3 UUID: 815266d6-a8b9-4f63-a593-02fde178263f cache and super generation don't match, space cache will be invalidated reset isize for dir 256 root 5 reset isize for dir 131327 root 5 reset isize for dir 924616 root 5 Moving file 'lib_mysqludf_log' to 'lost+found' dir since it has no valid backref Fixed the nlink of inode 924616 reset isize for dir 924619 root 5 Moving file 'lib_mysqludf_stem' to 'lost+found' dir since it has no valid backref Fixed the nlink of inode 924619 reset isize for dir 924625 root 5 Moving file 'mysql-udf-base64' to 'lost+found' dir since it has no valid backref Fixed the nlink of inode 924625 reset isize for dir 924626 root 5 Moving file 'mysql-udf-http' to 'lost+found' dir since it has no valid backref Fixed the nlink of inode 924626 reset isize for dir 924628 root 5 Moving file 'mysql-udf-ipv6' to 'lost+found' dir since it has no valid backref Fixed the nlink of inode 924628 reset isize for dir 924630 root 5 reset isize for dir 924632 root 5 reset isize for dir 924633 root 5 reset isize for dir 924634 root 5 reset isize for dir 924635 root 5 Trying to rebuild inode:1442047 Trying to rebuild inode:1835263 Trying to rebuild inode:1966335 Trying to rebuild inode:2097407 Trying to rebuild inode:2228479 Trying to rebuild inode:2359551 Trying to rebuild inode:3145983 Trying to rebuild inode:3277055 Trying to rebuild inode:3408127 Trying to rebuild inode:3539199 Trying to rebuild inode:3670271 Trying to rebuild inode:4587775 Trying to rebuild inode:4980991 Trying to rebuild inode:4984678 Trying to rebuild inode:5505279 Trying to rebuild inode:5767423 Trying to rebuild inode:5898495 Trying to rebuild inode:6029567 Trying to rebuild inode:6431345 Trying to rebuild inode:6553855 Trying to rebuild inode:7602431 Trying to rebuild inode:9133291 Trying to rebuild inode:9133292 Trying to rebuild inode:9980485 Trying to rebuild inode:10254180 Trying to rebuild inode:11827458 Trying to rebuild inode:14435353 Trying to rebuild inode:14646004 Trying to rebuild inode:14910590 Trying to rebuild inode:23438027 Trying to rebuild inode:25548332 Trying to rebuild inode:25548333 Trying to rebuild inode:26407893 Trying to rebuild inode:36767707 Trying to rebuild inode:36767712 Trying to rebuild inode:36767838 Trying to rebuild inode:36767839 Trying to rebuild inode:36767840 Trying to rebuild inode:36767843 Trying to rebuild inode:37030378 Trying to rebuild inode:37030384 Trying to rebuild inode:37030490 Trying to rebuild inode:37030492 Trying to rebuild inode:37030497 Trying to rebuild inode:39198776 Trying to rebuild inode:51520157 Trying to rebuild inode:51520163 Trying to rebuild inode:51520278 Trying to rebuild inode:51520280 Trying to rebuild inode:51520284 Trying to rebuild inode:58792107 Trying to rebuild inode:63681251 Trying to rebuild inode:66715793 Trying to rebuild inode:71469207 Trying to rebuild inode:71469208 Trying to rebuild inode:71469238 Trying to rebuild inode:71469239 Trying to rebuild inode:71469240 Trying to rebuild inode:75334150 Trying to rebuild inode:77351088 found 89594781696 bytes used, error(s) found total csum bytes: 0 total tree bytes: 108093440 total fs tree bytes: 53248 total extent tree bytes: 107958272 btree space waste bytes: 36265970 file data blocks allocated: 27983872 referenced 27983872 ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 9:01 ` Patrick Erley @ 2019-12-30 9:09 ` Qu Wenruo 2019-12-30 9:21 ` Patrick Erley 0 siblings, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2019-12-30 9:09 UTC (permalink / raw) To: Patrick Erley; +Cc: Btrfs BTRFS [-- Attachment #1.1: Type: text/plain, Size: 21506 bytes --] On 2019/12/30 下午5:01, Patrick Erley wrote: > On Mon, Dec 30, 2019 at 8:54 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >> On 2019/12/30 下午4:14, Patrick Erley wrote: >>> On Mon, Dec 30, 2019 at 6:09 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>> >>> Should I also paste in the repair log? >>> >> Yes please. >> >> This sounds very strange, especially for the transid mismatch part. >> >> Thanks, >> Qu >> > > > > enabling repair mode > WARNING: > > Do not use --repair unless you are advised to do so by a developer > or an experienced user, and then only after having accepted that no > fsck can successfully repair all types of filesystem corruption. Eg. > some software or hardware bugs can fatally damage a volume. > The operation will start in 10 seconds. > Use Ctrl-C to stop it. > 10 9 8 7 6 5 4 3 2 1parent transid verify failed on 499774464000 > wanted 3323349 found 3323340 > parent transid verify failed on 499774521344 wanted 3323349 found 3323340 > parent transid verify failed on 499774529536 wanted 3323349 found 3323340 This message is from open_ctree(), which means the fs is already corrupted. Would you like to provide the history between last good btrfs check run and --repair run? Thanks, Qu > [1/7] checking root items > parent transid verify failed on 499774373888 wanted 3323349 found 3323340 > parent transid verify failed on 499774480384 wanted 3323349 found 3323340 > parent transid verify failed on 499774386176 wanted 3323349 found 3323340 > parent transid verify failed on 499774390272 wanted 3323349 found 3323340 > parent transid verify failed on 499774394368 wanted 3323349 found 3323340 > parent transid verify failed on 499774398464 wanted 3323349 found 3323340 > parent transid verify failed on 499774402560 wanted 3323349 found 3323340 > parent transid verify failed on 499774406656 wanted 3323349 found 3323340 > parent transid verify failed on 499774423040 wanted 3323349 found 3323340 > parent transid verify failed on 499774427136 wanted 3323349 found 3323340 > parent transid verify failed on 499774435328 wanted 3323349 found 3323340 > parent transid verify failed on 499774439424 wanted 3323349 found 3323340 > parent transid verify failed on 499774320640 wanted 3323349 found 3323340 > parent transid verify failed on 499774324736 wanted 3323349 found 3323340 > parent transid verify failed on 499774443520 wanted 3323349 found 3323340 > parent transid verify failed on 499774447616 wanted 3323349 found 3323340 > parent transid verify failed on 499774504960 wanted 3323349 found 3323340 > parent transid verify failed on 499774513152 wanted 3323349 found 3323340 > parent transid verify failed on 499774468096 wanted 3323349 found 3323340 > parent transid verify failed on 499774337024 wanted 3323349 found 3323340 > parent transid verify failed on 499774341120 wanted 3323349 found 3323340 > parent transid verify failed on 499774517248 wanted 3323349 found 3323340 > parent transid verify failed on 499774558208 wanted 3323349 found 3323340 > Fixed 0 roots. > [2/7] checking extents > extent back ref already exists for 499952799744 parent 0 root 2 > extent back ref already exists for 499952803840 parent 0 root 2 > extent back ref already exists for 499952807936 parent 0 root 2 > extent back ref already exists for 499952812032 parent 0 root 2 > extent back ref already exists for 499952816128 parent 0 root 2 > extent back ref already exists for 499952824320 parent 0 root 5 > extent back ref already exists for 499952832512 parent 0 root 2 > extent back ref already exists for 499952852992 parent 0 root 2 > extent back ref already exists for 499952857088 parent 0 root 2 > extent back ref already exists for 499952861184 parent 0 root 2 > extent back ref already exists for 499952877568 parent 0 root 2 > extent back ref already exists for 499952885760 parent 0 root 2 > extent back ref already exists for 499952889856 parent 0 root 2 > extent back ref already exists for 499952898048 parent 0 root 2 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499774554112 > leaf parent key incorrect 499773358080 > bad block 499773358080 > ERROR: errors found in extent allocation tree or chunk allocation > [3/7] checking free space cache > [4/7] checking fs roots > parent transid verify failed on 499774283776 wanted 3323349 found 3323340 > Wrong key of child node/leaf, wanted: (1442047, 1, 0), have: > (523365974016, 168, 16384) > ERROR: child eb corrupted: parent bytenr=499750891520 item=2 parent > level=2 child level=0 > root 5 inode 1442047 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name etc filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 1835263 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name bin filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 1966335 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name proc filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 2097407 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 5 name lib32 filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 2228479 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name sys filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 2359551 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 5 name lib64 filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 3145983 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name dev filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 3277055 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name opt filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 3408127 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name mnt filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 3539199 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name usr filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 3670271 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 5 name media filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 4587775 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name sbin filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 4980991 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name etc2 filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 4984678 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name run filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 5505279 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 7 name exports filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 5767423 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 5 name debug filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 5898495 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name etc3 filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 6029567 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name root filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 6431345 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name boot filetype 2 > errors 6, no dir index, no inode ref > ERROR: child eb corrupted: parent bytenr=499762622464 item=37 parent > level=2 child level=0 > root 5 inode 6553855 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name var filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 7602431 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name tmp filetype 2 > errors 6, no dir index, no inode ref > ERROR: child eb corrupted: parent bytenr=499752206336 item=14 parent > level=3 child level=0 > root 5 inode 9133291 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 13 name .pulse-cookie > filetype 1 errors 6, no dir index, no inode ref > ERROR: child eb corrupted: parent bytenr=499748732928 item=14 parent > level=3 child level=0 > root 5 inode 9133292 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 6 name .pulse filetype 2 > errors 6, no dir index, no inode ref > ERROR: child eb corrupted: parent bytenr=499752206336 item=14 parent > level=3 child level=0 > root 5 inode 9980485 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 9 name bootchart filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 10254180 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name temp filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 11827458 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 8 name tftpboot filetype 2 > errors 6, no dir index, no inode ref > ERROR: child eb corrupted: parent bytenr=499748732928 item=17 parent > level=3 child level=0 > root 5 inode 14435353 errors 2001, no inode item, link count wrong > unresolved ref dir 131327 index 0 namelen 4 name s0be filetype 2 > errors 6, no dir index, no inode ref > ERROR: child eb corrupted: parent bytenr=499752206336 item=17 parent > level=3 child level=0 > root 5 inode 14646004 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name s0be filetype 2 > errors 6, no dir index, no inode ref > ERROR: child eb corrupted: parent bytenr=499748732928 item=17 parent > level=3 child level=0 > root 5 inode 14910590 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 13 name .bash_history > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 23438027 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 8 name mknod.sh filetype 1 > errors 6, no dir index, no inode ref > root 5 inode 25548332 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 8 name ctrl_dir filetype 1 > errors 6, no dir index, no inode ref > root 5 inode 25548333 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 5 name state filetype 1 > errors 6, no dir index, no inode ref > root 5 inode 26407893 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 6 name e2fsck filetype 1 > errors 6, no dir index, no inode ref > root 5 inode 36767707 errors 2001, no inode item, link count wrong > unresolved ref dir 924630 index 0 namelen 31 name > lib_mysqludf_log-warnings.patch filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 36767712 errors 2001, no inode item, link count wrong > unresolved ref dir 924632 index 0 namelen 32 name > lib_mysqludf_stem-mysql_m4.patch filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 36767838 errors 2001, no inode item, link count wrong > unresolved ref dir 924633 index 0 namelen 33 name > mysql-udf-base64-signedness.patch filetype 1 errors 6, no dir index, > no inode ref > root 5 inode 36767839 errors 2001, no inode item, link count wrong > unresolved ref dir 924633 index 0 namelen 20 name > mysql-udf-base64.sql filetype 1 errors 6, no dir index, no inode ref > root 5 inode 36767840 errors 2001, no inode item, link count wrong > unresolved ref dir 924634 index 0 namelen 27 name > mysql-udf-http-stdlib.patch filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 36767843 errors 2001, no inode item, link count wrong > unresolved ref dir 924635 index 0 namelen 29 name > mysql-udf-ipv6-warnings.patch filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 37030378 errors 2001, no inode item, link count wrong > unresolved ref dir 924616 index 0 namelen 12 name metadata.xml > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 37030384 errors 2001, no inode item, link count wrong > unresolved ref dir 924619 index 0 namelen 12 name metadata.xml > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 37030490 errors 2001, no inode item, link count wrong > unresolved ref dir 924625 index 0 namelen 12 name metadata.xml > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 37030492 errors 2001, no inode item, link count wrong > unresolved ref dir 924626 index 0 namelen 12 name metadata.xml > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 37030497 errors 2001, no inode item, link count wrong > unresolved ref dir 924628 index 0 namelen 12 name metadata.xml > filetype 1 errors 6, no dir index, no inode ref > root 5 inode 39198776 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 8 name bootback filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 51520157 errors 2001, no inode item, link count wrong > unresolved ref dir 924616 index 0 namelen 29 name > lib_mysqludf_log-0.0.2.ebuild filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 51520163 errors 2001, no inode item, link count wrong > unresolved ref dir 924619 index 0 namelen 30 name > lib_mysqludf_stem-0.9.1.ebuild filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 51520278 errors 2001, no inode item, link count wrong > unresolved ref dir 924625 index 0 namelen 32 name > mysql-udf-base64-20010618.ebuild filetype 1 errors 6, no dir index, no > inode ref > root 5 inode 51520280 errors 2001, no inode item, link count wrong > unresolved ref dir 924626 index 0 namelen 25 name > mysql-udf-http-1.0.ebuild filetype 1 errors 6, no dir index, no inode > ref > root 5 inode 51520284 errors 2001, no inode item, link count wrong > unresolved ref dir 924628 index 0 namelen 25 name > mysql-udf-ipv6-0.4.ebuild filetype 1 errors 6, no dir index, no inode > ref > root 5 inode 58792107 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name foo filetype 1 > errors 6, no dir index, no inode ref > root 5 inode 63681251 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 4 name xpra filetype 2 > errors 6, no dir index, no inode ref > root 5 inode 66715793 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 8 name .viminfo filetype 1 > errors 6, no dir index, no inode ref > root 5 inode 71469207 errors 2001, no inode item, link count wrong > unresolved ref dir 924616 index 0 namelen 8 name Manifest filetype > 1 errors 6, no dir index, no inode ref > root 5 inode 71469208 errors 2001, no inode item, link count wrong > unresolved ref dir 924619 index 0 namelen 8 name Manifest filetype > 1 errors 6, no dir index, no inode ref > root 5 inode 71469238 errors 2001, no inode item, link count wrong > unresolved ref dir 924625 index 0 namelen 8 name Manifest filetype > 1 errors 6, no dir index, no inode ref > root 5 inode 71469239 errors 2001, no inode item, link count wrong > unresolved ref dir 924626 index 0 namelen 8 name Manifest filetype > 1 errors 6, no dir index, no inode ref > root 5 inode 71469240 errors 2001, no inode item, link count wrong > unresolved ref dir 924628 index 0 namelen 8 name Manifest filetype > 1 errors 6, no dir index, no inode ref > ERROR: child eb corrupted: parent bytenr=499748732928 item=65 parent > level=3 child level=1 > root 5 inode 75334150 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 7 name lib.new filetype 2 > errors 6, no dir index, no inode ref > parent transid verify failed on 499774287872 wanted 3323349 found 3323340 > parent transid verify failed on 499774291968 wanted 3323349 found 3323340 > parent transid verify failed on 499774296064 wanted 3323349 found 3323340 > root 5 inode 77351088 errors 2001, no inode item, link count wrong > unresolved ref dir 256 index 0 namelen 3 name lib filetype 7 > errors 6, no dir index, no inode ref > ERROR: errors found in fs roots > > Starting repair. > Opening filesystem to check... > Checking filesystem on /dev/sda3 > UUID: 815266d6-a8b9-4f63-a593-02fde178263f > cache and super generation don't match, space cache will be invalidated > reset isize for dir 256 root 5 > reset isize for dir 131327 root 5 > reset isize for dir 924616 root 5 > Moving file 'lib_mysqludf_log' to 'lost+found' dir since it has no valid backref > Fixed the nlink of inode 924616 > reset isize for dir 924619 root 5 > Moving file 'lib_mysqludf_stem' to 'lost+found' dir since it has no > valid backref > Fixed the nlink of inode 924619 > reset isize for dir 924625 root 5 > Moving file 'mysql-udf-base64' to 'lost+found' dir since it has no valid backref > Fixed the nlink of inode 924625 > reset isize for dir 924626 root 5 > Moving file 'mysql-udf-http' to 'lost+found' dir since it has no valid backref > Fixed the nlink of inode 924626 > reset isize for dir 924628 root 5 > Moving file 'mysql-udf-ipv6' to 'lost+found' dir since it has no valid backref > Fixed the nlink of inode 924628 > reset isize for dir 924630 root 5 > reset isize for dir 924632 root 5 > reset isize for dir 924633 root 5 > reset isize for dir 924634 root 5 > reset isize for dir 924635 root 5 > Trying to rebuild inode:1442047 > Trying to rebuild inode:1835263 > Trying to rebuild inode:1966335 > Trying to rebuild inode:2097407 > Trying to rebuild inode:2228479 > Trying to rebuild inode:2359551 > Trying to rebuild inode:3145983 > Trying to rebuild inode:3277055 > Trying to rebuild inode:3408127 > Trying to rebuild inode:3539199 > Trying to rebuild inode:3670271 > Trying to rebuild inode:4587775 > Trying to rebuild inode:4980991 > Trying to rebuild inode:4984678 > Trying to rebuild inode:5505279 > Trying to rebuild inode:5767423 > Trying to rebuild inode:5898495 > Trying to rebuild inode:6029567 > Trying to rebuild inode:6431345 > Trying to rebuild inode:6553855 > Trying to rebuild inode:7602431 > Trying to rebuild inode:9133291 > Trying to rebuild inode:9133292 > Trying to rebuild inode:9980485 > Trying to rebuild inode:10254180 > Trying to rebuild inode:11827458 > Trying to rebuild inode:14435353 > Trying to rebuild inode:14646004 > Trying to rebuild inode:14910590 > Trying to rebuild inode:23438027 > Trying to rebuild inode:25548332 > Trying to rebuild inode:25548333 > Trying to rebuild inode:26407893 > Trying to rebuild inode:36767707 > Trying to rebuild inode:36767712 > Trying to rebuild inode:36767838 > Trying to rebuild inode:36767839 > Trying to rebuild inode:36767840 > Trying to rebuild inode:36767843 > Trying to rebuild inode:37030378 > Trying to rebuild inode:37030384 > Trying to rebuild inode:37030490 > Trying to rebuild inode:37030492 > Trying to rebuild inode:37030497 > Trying to rebuild inode:39198776 > Trying to rebuild inode:51520157 > Trying to rebuild inode:51520163 > Trying to rebuild inode:51520278 > Trying to rebuild inode:51520280 > Trying to rebuild inode:51520284 > Trying to rebuild inode:58792107 > Trying to rebuild inode:63681251 > Trying to rebuild inode:66715793 > Trying to rebuild inode:71469207 > Trying to rebuild inode:71469208 > Trying to rebuild inode:71469238 > Trying to rebuild inode:71469239 > Trying to rebuild inode:71469240 > Trying to rebuild inode:75334150 > Trying to rebuild inode:77351088 > found 89594781696 bytes used, error(s) found > total csum bytes: 0 > total tree bytes: 108093440 > total fs tree bytes: 53248 > total extent tree bytes: 107958272 > btree space waste bytes: 36265970 > file data blocks allocated: 27983872 > referenced 27983872 > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 520 bytes --] ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 9:09 ` Qu Wenruo @ 2019-12-30 9:21 ` Patrick Erley 2019-12-30 9:27 ` Qu Wenruo 0 siblings, 1 reply; 75+ messages in thread From: Patrick Erley @ 2019-12-30 9:21 UTC (permalink / raw) To: Qu Wenruo; +Cc: Btrfs BTRFS On Mon, Dec 30, 2019 at 9:09 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > On 2019/12/30 下午5:01, Patrick Erley wrote: > > On Mon, Dec 30, 2019 at 8:54 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >> On 2019/12/30 下午4:14, Patrick Erley wrote: > >>> On Mon, Dec 30, 2019 at 6:09 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >>> > >>> Should I also paste in the repair log? > >>> > >> Yes please. > >> > >> This sounds very strange, especially for the transid mismatch part. > >> > >> Thanks, > >> Qu > >> > > > > > > > > enabling repair mode > > WARNING: > > > > Do not use --repair unless you are advised to do so by a developer > > or an experienced user, and then only after having accepted that no > > fsck can successfully repair all types of filesystem corruption. Eg. > > some software or hardware bugs can fatally damage a volume. > > The operation will start in 10 seconds. > > Use Ctrl-C to stop it. > > 10 9 8 7 6 5 4 3 2 1parent transid verify failed on 499774464000 > > wanted 3323349 found 3323340 > > parent transid verify failed on 499774521344 wanted 3323349 found 3323340 > > parent transid verify failed on 499774529536 wanted 3323349 found 3323340 > > This message is from open_ctree(), which means the fs is already corrupted. > > Would you like to provide the history between last good btrfs check run > and --repair run? > > Thanks, > Qu In theory, all I did was boot back into 5.1 and continued using the system. After you said I should go ahead and try to --repair, I rebooted into initramfs and ran the repair, then continued booting(which failed spectacularly, due to almost all of / being missing). I then rebooted back into initramfs to assess what was going on, and made a liveusb (from which I'm sending this on that system). Some 'background' on the FS: It was migrated from ext4 ~7? years ago, and has been moved between multiple discs and systems using dd. Interesting point: The only files/folders that still exist in / were created after I migrated the filesystem. If I can get /etc and maybe /var back, I'm golden (there are a few bits in each I don't include in my hot backups, so will have to go to offline storage to fetch them). ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 9:21 ` Patrick Erley @ 2019-12-30 9:27 ` Qu Wenruo 2019-12-30 10:06 ` Patrick Erley 0 siblings, 1 reply; 75+ messages in thread From: Qu Wenruo @ 2019-12-30 9:27 UTC (permalink / raw) To: Patrick Erley; +Cc: Btrfs BTRFS [-- Attachment #1.1: Type: text/plain, Size: 2888 bytes --] On 2019/12/30 下午5:21, Patrick Erley wrote: > On Mon, Dec 30, 2019 at 9:09 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >> >> >> >> On 2019/12/30 下午5:01, Patrick Erley wrote: >>> On Mon, Dec 30, 2019 at 8:54 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>>> On 2019/12/30 下午4:14, Patrick Erley wrote: >>>>> On Mon, Dec 30, 2019 at 6:09 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>>>> >>>>> Should I also paste in the repair log? >>>>> >>>> Yes please. >>>> >>>> This sounds very strange, especially for the transid mismatch part. >>>> >>>> Thanks, >>>> Qu >>>> >>> >>> >>> >>> enabling repair mode >>> WARNING: >>> >>> Do not use --repair unless you are advised to do so by a developer >>> or an experienced user, and then only after having accepted that no >>> fsck can successfully repair all types of filesystem corruption. Eg. >>> some software or hardware bugs can fatally damage a volume. >>> The operation will start in 10 seconds. >>> Use Ctrl-C to stop it. >>> 10 9 8 7 6 5 4 3 2 1parent transid verify failed on 499774464000 >>> wanted 3323349 found 3323340 >>> parent transid verify failed on 499774521344 wanted 3323349 found 3323340 >>> parent transid verify failed on 499774529536 wanted 3323349 found 3323340 >> >> This message is from open_ctree(), which means the fs is already corrupted. >> >> Would you like to provide the history between last good btrfs check run >> and --repair run? >> >> Thanks, >> Qu > > In theory, all I did was boot back into 5.1 and continued using the > system. If that's the only thing before --repair (if there is only one repair run, and output is exactly what you pasted), then I guess something didn't go right in that 5.1 run? Is that pasted output from the first --repair run? If there is another run before the pasted output, then it could be previous --repair. Either way, I'm very sorry for the data loss... > After you said I should go ahead and try to --repair, I > rebooted into initramfs and ran the repair, then continued > booting(which failed spectacularly, due to almost all of / being > missing). I then rebooted back into initramfs to assess what was > going on, and made a liveusb (from which I'm sending this on that > system). Some 'background' on the FS: It was migrated from ext4 ~7? > years ago, and has been moved between multiple discs and systems using > dd. Interesting point: The only files/folders that still exist in / > were created after I migrated the filesystem. If I can get /etc and > maybe /var back, I'm golden (there are a few bits in each I don't > include in my hot backups, so will have to go to offline storage to > fetch them). I'm afraid the only chance we left is btrfs-restore. And normally for transid error, the chance is pretty low then. Thanks, Qu > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 520 bytes --] ^ permalink raw reply [flat|nested] 75+ messages in thread
* Re: read time tree block corruption detected 2019-12-30 9:27 ` Qu Wenruo @ 2019-12-30 10:06 ` Patrick Erley 0 siblings, 0 replies; 75+ messages in thread From: Patrick Erley @ 2019-12-30 10:06 UTC (permalink / raw) To: Qu Wenruo; +Cc: Btrfs BTRFS That sucks :-( The only reason I had hope for /etc and /var was they show up in btrfs check as not having a parent and i figured I could reparent them onto the new / and have a chance of extracting something useful from them. I guess I'll plan on grabbing images of the FS tomorrow and starting a full rebuild. Hopefully someone will chime in with a way to get /etc/ and /var/ before I reach the point I need them. On Mon, Dec 30, 2019 at 9:27 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > On 2019/12/30 下午5:21, Patrick Erley wrote: > > On Mon, Dec 30, 2019 at 9:09 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >> > >> > >> > >> On 2019/12/30 下午5:01, Patrick Erley wrote: > >>> On Mon, Dec 30, 2019 at 8:54 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >>>> On 2019/12/30 下午4:14, Patrick Erley wrote: > >>>>> On Mon, Dec 30, 2019 at 6:09 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >>>>> > >>>>> Should I also paste in the repair log? > >>>>> > >>>> Yes please. > >>>> > >>>> This sounds very strange, especially for the transid mismatch part. > >>>> > >>>> Thanks, > >>>> Qu > >>>> > >>> > >>> > >>> > >>> enabling repair mode > >>> WARNING: > >>> > >>> Do not use --repair unless you are advised to do so by a developer > >>> or an experienced user, and then only after having accepted that no > >>> fsck can successfully repair all types of filesystem corruption. Eg. > >>> some software or hardware bugs can fatally damage a volume. > >>> The operation will start in 10 seconds. > >>> Use Ctrl-C to stop it. > >>> 10 9 8 7 6 5 4 3 2 1parent transid verify failed on 499774464000 > >>> wanted 3323349 found 3323340 > >>> parent transid verify failed on 499774521344 wanted 3323349 found 3323340 > >>> parent transid verify failed on 499774529536 wanted 3323349 found 3323340 > >> > >> This message is from open_ctree(), which means the fs is already corrupted. > >> > >> Would you like to provide the history between last good btrfs check run > >> and --repair run? > >> > >> Thanks, > >> Qu > > > > In theory, all I did was boot back into 5.1 and continued using the > > system. > > If that's the only thing before --repair (if there is only one repair > run, and output is exactly what you pasted), then I guess something > didn't go right in that 5.1 run? > > Is that pasted output from the first --repair run? > > If there is another run before the pasted output, then it could be > previous --repair. > > Either way, I'm very sorry for the data loss... > > > After you said I should go ahead and try to --repair, I > > rebooted into initramfs and ran the repair, then continued > > booting(which failed spectacularly, due to almost all of / being > > missing). I then rebooted back into initramfs to assess what was > > going on, and made a liveusb (from which I'm sending this on that > > system). Some 'background' on the FS: It was migrated from ext4 ~7? > > years ago, and has been moved between multiple discs and systems using > > dd. Interesting point: The only files/folders that still exist in / > > were created after I migrated the filesystem. If I can get /etc and > > maybe /var back, I'm golden (there are a few bits in each I don't > > include in my hot backups, so will have to go to offline storage to > > fetch them). > > I'm afraid the only chance we left is btrfs-restore. > > And normally for transid error, the chance is pretty low then. > > Thanks, > Qu > > > > ^ permalink raw reply [flat|nested] 75+ messages in thread
end of thread, other threads:[~2021-11-23 5:56 UTC | newest] Thread overview: 75+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-01-16 13:40 read time tree block corruption detected Peter Luladjiev 2020-01-16 16:12 ` Nikolay Borisov [not found] ` <CA+ZCqs5=N5Hdf3NxZAmPCnA8wbcJPrcH8zM-fRbt-w8tL+TjUQ@mail.gmail.com> 2020-01-17 7:34 ` Nikolay Borisov 2020-01-17 7:51 ` Peter Luladjiev 2020-01-17 7:54 ` Peter Luladjiev 2020-01-17 7:59 ` Qu Wenruo 2020-01-17 8:14 ` Nikolay Borisov 2020-01-17 8:22 ` Peter Luladjiev 2020-01-17 9:10 ` Nikolay Borisov 2020-01-17 12:04 ` Peter Luladjiev -- strict thread matches above, loose matches on Subject: below -- 2021-11-22 5:26 x8062 2021-11-22 7:24 ` Nikolay Borisov 2021-11-22 10:07 ` x8062 2021-11-22 10:36 ` Nikolay Borisov 2021-11-23 2:42 ` x8062 2021-11-23 5:56 ` Nikolay Borisov 2021-07-17 1:45 Read " pepperpoint 2021-07-17 7:05 ` Qu Wenruo [not found] ` <162650555086.7.16811903270475408953.10183708@simplelogin.co> 2021-07-17 7:51 ` pepperpoint [not found] ` <162650826457.7.1050455337652772013.10184548@mb.ardentcoding.com> 2021-07-17 8:14 ` Qu Wenruo [not found] ` <162650966150.7.11743767259405124657.10185986@simplelogin.co> 2021-07-17 8:57 ` pepperpoint 2021-07-17 10:12 ` Qu Wenruo [not found] ` <162651674065.6.7912816287233908759.10188327@simplelogin.co> 2021-07-17 10:34 ` pepperpoint [not found] ` <162651809235.7.7061042874033963922.10188873@mb.ardentcoding.com> 2021-07-17 10:48 ` Qu Wenruo [not found] ` <162651892663.6.17938009695497100557.10189169@simplelogin.co> 2021-07-17 12:51 ` pepperpoint [not found] ` <CQeY09U34m7SrT6nTAlkSQrbLJtmyKF1tDfuGDtKUkwJqujMI_nZU4MpGiU4F_Q1U3Lk1aWD1mFCT5SlsOsOcILWECflIw7EhVQTQpy_1Ts=@email.ardentcoding.com> 2021-07-18 5:26 ` pepperpoint 2021-07-18 7:15 ` Qu Wenruo [not found] ` <162659327011.8.12718249358254503695.10218325@simplelogin.co> 2021-07-18 8:46 ` pepperpoint [not found] ` <162659797656.6.14385932343235367381.10220447@mb.ardentcoding.com> 2021-07-18 9:32 ` Qu Wenruo [not found] ` <162660074747.7.3300740266059717894.10221270@simplelogin.co> 2021-07-18 10:34 ` pepperpoint [not found] ` <162660447733.8.9782212603273165785.10222524@mb.ardentcoding.com> 2021-07-18 11:13 ` Qu Wenruo [not found] ` <162660684635.8.12423097770824212671.10223516@simplelogin.co> 2021-07-18 12:16 ` pepperpoint 2021-04-16 19:35 read " Gervais, Francois 2021-04-17 1:01 ` Qu Wenruo 2021-04-19 13:20 ` Gervais, Francois 2021-04-19 13:33 ` Qu Wenruo 2021-04-19 14:56 ` Gervais, Francois 2021-04-20 1:34 ` Qu Wenruo 2021-04-20 14:19 ` Gervais, Francois 2021-04-20 23:47 ` Qu Wenruo 2021-04-21 14:17 ` Gervais, Francois 2021-04-21 23:01 ` Qu Wenruo 2021-04-22 14:26 ` Gervais, Francois 2021-05-26 23:03 ` Gervais, Francois 2021-05-26 23:25 ` Qu Wenruo [not found] <CAJheHN0FUe-ijMco1ZOc6iKF2zbPocOw+iiVNeTT1r-JuXOJww@mail.gmail.com> 2020-05-06 21:54 ` Fwd: Read " Tyler Richmond 2020-05-06 23:55 ` Chris Murphy 2020-05-07 0:51 ` Tyler Richmond 2020-05-07 1:06 ` Chris Murphy 2020-02-12 21:58 read " Samir Benmendil 2020-02-13 0:26 ` Qu Wenruo 2020-02-13 13:04 ` Samir Benmendil 2020-02-13 14:01 ` Qu Wenruo 2020-02-15 15:34 ` Samir Benmendil 2019-12-29 20:43 Patrick Erley 2019-12-29 22:07 ` Chris Murphy 2019-12-29 22:27 ` Patrick Erley 2019-12-29 22:32 ` Chris Murphy 2019-12-29 22:36 ` Patrick Erley 2019-12-29 23:11 ` Chris Murphy 2019-12-29 23:19 ` Patrick Erley 2019-12-29 23:24 ` Chris Murphy 2019-12-29 23:26 ` Patrick Erley 2019-12-30 0:46 ` Qu Wenruo 2019-12-30 5:36 ` Patrick Erley 2019-12-30 5:43 ` Qu Wenruo 2019-12-30 5:47 ` Patrick Erley 2019-12-30 5:50 ` Patrick Erley 2019-12-30 5:58 ` Qu Wenruo 2019-12-30 6:07 ` Patrick Erley 2019-12-30 6:09 ` Qu Wenruo 2019-12-30 8:14 ` Patrick Erley 2019-12-30 8:54 ` Qu Wenruo 2019-12-30 9:01 ` Patrick Erley 2019-12-30 9:09 ` Qu Wenruo 2019-12-30 9:21 ` Patrick Erley 2019-12-30 9:27 ` Qu Wenruo 2019-12-30 10:06 ` Patrick Erley
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).