* [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
@ 2017-01-23 9:13 Qu Wenruo
2017-01-23 9:13 ` [PATCH 1/9] btrfs-progs: lowmem check: Fix wrong block group check when search slot points to slot beyong leaf Qu Wenruo
` (10 more replies)
0 siblings, 11 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-23 9:13 UTC (permalink / raw)
To: linux-btrfs; +Cc: calestyo, chris, dsterba
Patches can be fetch from github:
https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes
Although there are near 10 patches, but they are all small.
No need to be scared. :)
Thanks for reports from Chris Murphy and Christoph Anton Mitterer,
several new bugs are exposed for lowmem mode fsck.
And one original mode bug, not fixed in this patchset.
The original mode bug is caused by fsck/006, which repairs doesn't
fix backrefs of a data extent, which lowmem mode detects it correctly.
1) Block group used space false alert
If a BLOCK_GROUP_ITEM or its first EXTENT/METADATA_ITEM is located at
the first slot of a leaf, search_slot() used by lowmem mode can
point to previous leaf, with path->slots[0] beyond valid leaf slots.
This makes us to read out uninitialized data, and can abort block
group used space check loop, causing a false alert.
Fix it with a test case image inside fsck-tests/020/extent-ref-cases
Reported by Christoph.
2) Partly written prealloc extent false alert
If a prealloc extent gets partily written, lowmem mode will report
prealloc extent shouldn't have csum.
Lowmem mode passed wrong variable to csum checking code, causing it
to check the whole range of the prealloc extent, making the bug
happens.
Fix it with a test case inside fsck-tests/020/extent-ref-cases.
Reported by Chirs Murphy And Christoph.
3) Extent item size false alert.
Under certain case, btrfs lowmem mode check reports data backref
lost.
It's because newly introduced extent item size check aborts normal
check routine.
It can happen if a data/metadata extent item has no inline ref.
Fix it, test case already submitted before and merged, but due to
fsck-tests framework bugs, it never get called for lowmem mode.
4) fsck-tests Lowmem mode override fixes
Allow lowmem mode override to get called for all tests, and allow
them all to pass lowmem mode except fsck-tests/006, which is a
original repair mode bug.
Lu Fengqi (1):
btrfs-progs: fsck: Fix lowmem mode override to allow it skip repair
work
Qu Wenruo (8):
btrfs-progs: lowmem check: Fix wrong block group check when search
slot points to slot beyong leaf
btrfs-progs: fsck-test: Add test image for lowmem mode block group
false alert
btrfs-progs: fsck: Output verbose error when fsck found a bug
btrfs-progs: lowmem check: Fix false alert in checking data extent
pointing to prealloc extent
btrfs-progs: lowmem check: Fix extent item size false alert
btrfs-progs: tests: Move fsck-tests/015 to fuzz tests
btrfs-progs: fsck-tests: Make 013 compatible with lowmem mode
btrfs-progs: fsck-tests: Add new test case for partly written prealloc
extent
cmds-check.c | 80 ++++++++++++++++-----
tests/common | 8 +--
tests/common.local | 2 +-
tests/fsck-tests/013-extent-tree-rebuild/test.sh | 2 +-
.../block_group_item_false_alert.raw.xz | Bin 0 -> 47792 bytes
tests/fsck-tests/020-extent-ref-cases/test.sh | 30 ++++++--
.../images}/bko-97171-btrfs-image.raw.txt | 0
.../images}/bko-97171-btrfs-image.raw.xz | Bin
8 files changed, 95 insertions(+), 27 deletions(-)
create mode 100644 tests/fsck-tests/020-extent-ref-cases/block_group_item_false_alert.raw.xz
rename tests/{fsck-tests/015-check-bad-memory-access => fuzz-tests/images}/bko-97171-btrfs-image.raw.txt (100%)
rename tests/{fsck-tests/015-check-bad-memory-access => fuzz-tests/images}/bko-97171-btrfs-image.raw.xz (100%)
--
2.11.0
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 1/9] btrfs-progs: lowmem check: Fix wrong block group check when search slot points to slot beyong leaf
2017-01-23 9:13 [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Qu Wenruo
@ 2017-01-23 9:13 ` Qu Wenruo
2017-01-23 9:13 ` [PATCH 2/9] btrfs-progs: fsck-test: Add test image for lowmem mode block group false alert Qu Wenruo
` (9 subsequent siblings)
10 siblings, 0 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-23 9:13 UTC (permalink / raw)
To: linux-btrfs; +Cc: calestyo, chris, dsterba
Since btrfs_search_slot() can points to the slot which is beyond the
leaves' capacity, in the following case, btrfs lowmem mode check will
skip the block group and report false alert:
leaf 29405184 items 37 free space 1273 generation 11 owner 2
...
item 36 key (77594624 EXTENT_ITEM 2097152)
extent refs 1 gen 8 flags DATA
extent data backref root 5 objectid 265 offset 0 count 1
leaf 29409280 items 43 free space 670 generation 11 owner 2
item 0 key (96468992 EXTENT_ITEM 2097152)
extent refs 1 gen 8 flags DATA
extent data backref root 5 objectid 274 offset 0 count 1
item 1 key (96468992 BLOCK_GROUP_ITEM 33554432)
block group used 2265088 chunk_objectid 256 flags DATA
When checking block group item, we will search key(96468992 0 0) to
start from the first item in the block group.
While search_slot() will point to leaf 29405184, slot 37 which is beyond
leaf capacity.
And when reading key from slot 37, uninitialized data can be read out
and cause us to exit block group item check, leading to false alert.
Fix it by checking path.slot[0] before reading out the key.
Reported-by: Christoph Anton Mitterer <calestyo@scientia.net>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
cmds-check.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/cmds-check.c b/cmds-check.c
index 1dba2985..c39392b7 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -10961,6 +10961,11 @@ static int check_block_group_item(struct btrfs_fs_info *fs_info,
/* Iterate extent tree to account used space */
while (1) {
leaf = path.nodes[0];
+
+ /* Search slot can point to the last item beyond leaf nritems */
+ if (path.slots[0] >= btrfs_header_nritems(leaf))
+ goto next;
+
btrfs_item_key_to_cpu(leaf, &extent_key, path.slots[0]);
if (extent_key.objectid >= bg_key.objectid + bg_key.offset)
break;
--
2.11.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 2/9] btrfs-progs: fsck-test: Add test image for lowmem mode block group false alert
2017-01-23 9:13 [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Qu Wenruo
2017-01-23 9:13 ` [PATCH 1/9] btrfs-progs: lowmem check: Fix wrong block group check when search slot points to slot beyong leaf Qu Wenruo
@ 2017-01-23 9:13 ` Qu Wenruo
2017-01-23 9:13 ` [PATCH 3/9] btrfs-progs: fsck: Output verbose error when fsck found a bug Qu Wenruo
` (8 subsequent siblings)
10 siblings, 0 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-23 9:13 UTC (permalink / raw)
To: linux-btrfs; +Cc: calestyo, chris, dsterba
Add a minimal image which can reproduce the block group used space
false alert for lowmem mode fsck.
Reported-by: Christoph Anton Mitterer <calestyo@scientia.net>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
.../block_group_item_false_alert.raw.xz | Bin 0 -> 47792 bytes
tests/fsck-tests/020-extent-ref-cases/test.sh | 15 +++++++++++----
2 files changed, 11 insertions(+), 4 deletions(-)
create mode 100644 tests/fsck-tests/020-extent-ref-cases/block_group_item_false_alert.raw.xz
diff --git a/tests/fsck-tests/020-extent-ref-cases/block_group_item_false_alert.raw.xz b/tests/fsck-tests/020-extent-ref-cases/block_group_item_false_alert.raw.xz
new file mode 100644
index 0000000000000000000000000000000000000000..559c3fa9e8491f3ce1f424d1baef29853e8fb889
GIT binary patch
literal 47792
zcmeHwWmukDmL={If(LhZcL)S`cXxMpcPD`W!5xAV+%>qnyL)h$?&&{0(>=H9R^2;Q
z-Ti*Qd7ksJzrEL4dz~#W9G>c$ARrKXb9JI%AVi?JARr*`#??pO-s~W@bwEJuUf$mF
ze!PkCi=#Vo87MR+Qtsu2LO-1C;0I1K^l|foN>%#~MMlan<RLEl6>dk9peSu<VD-5~
zZn@nzB>N#)cCI-~7;EQFhOF(7$wFo%;?(r9$c-B&8)gSv5^r*S24&1YlJ6HO@I5-X
zMm=+|4QW$f92}?pM0A{MA>fqJm@%6us;Ff@N+Zs65^bwee?I@KoP9G|&?dIU)1?`f
z4M(i5PN{Y3DY4oU7paaNp*_2dZ+Ar<+4GVa;iO<oDMs~6f>f(R@$4vz3=WTw4%Jkc
z!0_XpyKSs!ByoH#K|!dqiZ7C;7K1l`aP_3bf$)5#L&szRR+LuAGa|IOgTifxCf#-T
z(LGUl?exf#!FLTA!OAMf+%4N%IaCqW+ls=Oz>+P64u$MzZO@a6fKyEAtKyH|ZzTJ1
z*|gv(!ggvf5O+#vp3dCF*euT_%Nqvn5fg>bjl!loHUf={<3BOM^fj`sHr(}n!w6Sx
z<4%=oG1p|C<=pua55w8;Mmk}y5>!*%yJm=d@BFM~bu>Z;Bl^OemlRq0vRWT*&jdYg
z;f-!nqvd8#=lnh`q=6x=?6R&clZV8O^$o3BK>LJg(DBH#fXFhC{@k)T|K4-?>G0QV
zxf(2#%(dgu^<d$Ur|I)mRrvTy?U_vIL);}<^VbNABe#IEtT{j8Q3-a=nHsn>iowEz
z)L6@2g|d8oc7Fl-k)%ZPE<3kpw;LQS*YWK$lN_o*K77Tb$paUxyWq22XaOtZ18eSQ
zMqILf0b$pH{Sp&p!&rzq{@No?e{5*~6gfJsqRO;@NV!iQb&wSUwbZ%Sfj<__U|Z=2
z4*U1Y;MNqn1oJ=IJV$_z3Lo1o9%;X6r?5EGr(Y>aT5{)bHdH?dp_c?*-yAO=J4$sO
z4u&XBRUXnD4W1u32hJ{Xr5V_oaV@J<R730i>g)vR3?ET?4?@fb!e2q&DF_l*kFBY~
zwb_=-uWI5eely{$8k7NHJJPmz5Lxks?v4KVD_-+hH3gQoM(ClbqI&zGYq`eGt`;QB
z!@yF-^OGWt^C-R+;m3znCBAzqV^|*lVQTC3dY+5kHhR5MWPT~rkMXETZ|{=s9o#kU
zyq#mj*g7HQPgqD}tJ@T>`_h6?C(FxC;xFQkx3qTiL-e=V9jyAeiXkK+*BPQ%%|m>5
zAz11uSfU_pm4sK*lMy3E<Ljs|KC4qE*?q<)0I`imtB(e~zp2|E`|txR_8$6u?=9>w
zIw5h-Xv@8+{3Jt^I{2ZLo(QynUBs`Dw@U?rp}-Jq4nqjUeMF9Su_z_K&3J;JIKR3h
zJnHIF?)oPi-@hgtNkxu`nmG|Rz64RdgxRQpIj9MESq-C|p2PY16{)6&!xYXr2DZw7
zfZ?sINZXU|!sX3GlmeYoA8^HwSh!NquhCp;j+(d^-Na5g!7|gGY9{S#coB|-V;9)d
zs&+3Hy=|YI`%Jzow$F^;F=I(EcMG$SaPF!q)$Mij6bC+oKHNRb+G5FE?du@w07C+5
zgQ~_Vho9B{-J9t{){ht_Tl;TCBzh;5!y%I?sgb_a30D*@ci5d!sWPcA3zf6+$h~Ar
z_qPe`OppatXLeooGGYjm2`Hr%`oH;0GN~(^X1%=wr2#o{QYw1+lTZ6yesd2$`j$c|
zid^#Sz4r8#kzTbkF625Wpwr}U`bYn$qZN<^YO>yfz=}ZlFJbpc6RGCYOt%lV%mo*s
zC4_j{paX6t?|9|*zL?oVd&h*UF_c3!Q9drg?vVEF2l@-$W_Y662zU=p$d>UDN~E6I
zYrw-e=Unf94wBWIE(;6O`H~559gnEDaexalAs>$sDKU<_laNa&kf%f6Vu>?w2u<HN
z$S-%)+HV{xq!MQ9g`ph-`ST6Uwaf<TLP=-`G$CDR^C3mGn=353F}}Cb3p({}mY5`0
z3AXL445Es~#F{zE=>*m?CG>`vojw-wps>>5Yiz$-4Tt~dqC+|C*<Lpp`5?cQyDcg$
z{_yq~JJQ+S>u(W|;)pDn5VK@h26sZbjj~ufR-RwPba=ALKKpZxs8GDUAZUV~jLq>q
zL@Ml#ZV*$rg*Bf~)<x>xY_r|tgr+QO)qHz=Fr4~zx0*(A3wm2%I^&r1OW%Dp*iF(q
zlh7tgvq>7fh}FEL(D}As-EPcsI346#EGxseSh@dg9{OdhGKZezepni26VK0#$<Q8!
zi_-n}ls$u)%gZM*#?tWH<-3&|PPSa{S)Jq1kEncR*&_EGbo#5ujx{@?zmzY+U1%AX
zyXBP=h`&IA>kZlSpmj2Hq!yrq798ZkQlFH*X_A*aB48vIwR6RhZvH88_D8wj?+Wqz
z^~{wDAiDqW3+Xq2<=>(kfW!au1_uoGJ1GxfFu-8Hn~MEWR|J4z|5+9SgybI`lE|P<
zARVs(pQwi+&D80L+UeY2$9zDD5SH59g7}Mn3=kmm6EK<mwBc?1gzJbLY&Q>Y6JU+=
z*t#NI5s0YHw2#g@{JxLUEm*iS@x=?<YPvB=OF}$jhrBRn##ZNHgh)^rTj=W%8sd#c
z5cR;&K(nxKk%&l$lu$xEIuBwWe57SUw}o`P5{pAx%x8GJ@OJ{e807qUBRyx9U9Woc
zG-Gcz$$v-I`Wyc8yExe2kTCz*M~C5QRspOKV1<9DuJapi3S2>d(-rg?lm%q|35->Q
zz=<6{S;k{=>VI_l`d2__i3V}%C4glBmH}AyPo|Kr;bDPnmZJJr6D)R(B1=dSuImNJ
z%z-!c;Hc?mmvarg%EnY@3V(P!l<8_(V4=!mYFvKhMa(#vm!QD@CKsOTGJ9r)hzwn#
zF}Y$BGv@zkNV?z#z7&Q`1-1UYadBKTC`>jQPSHglccQQ3(FCM2WNz@7_{tGzvM7%e
zSKouB|A*j@_vejJWi0!AjcgLGt7~If1$2jYhbURc25P!R!ehD>A9HRSr6sOhG!H}<
zhxm*1h!2}^j47LW3hX^T*`P#P2o!$PT>yLD$-daiKgU1Xi;vEz+{(LF0(m6_FI#ii
zT?{v@rqk<O65!p%8G^s1shn3((G;t8e_;_Mfe)b72s@FDpb$_L<UkUW3Q*d!G3JSD
zQ6S?%_?cjN>;|7IuF{g9L#~zj{j))0(DgHVOQF3KRqNSdEu&&XzlOTdF*O$WxNYl1
znZ<!D!FG{gzBP(jmo4m=&FQJY7~U*>LF(lVdBigH_qzsq^7xs)t^}zN8>WmD>;fle
zi#88CNj@i!$W|J>FC@z6%QPzr;CtewO6=c17S4!9^r2tZti%q4O&*LAmH6P)A|bt8
zc1(bHQMKy#s(<UjN?Hby++8j>`^q?Ls;&cSWa>@NmD?34(w`Wd?4O>qp@z$ojE+aI
z_bX=ZH9<7AM9JbiaU`tekAh=-tuV9!7Myhlr#8Js5M^|{q?vGSm!||!oGB4Sg>By0
z65*wr;=WO1g=FtXO+yCV{T5e;O-72AxYcg7_ZeI9Wl-82LkzrHPM)=kn}~{2Cv%Um
zP{x6q*T=05Zxm_U88Eo6=e8oCpr9M6{ZSVF8|wiGU-<IP1A>xxVDv8^jq8{-b{NGR
z-xD$x8c4&_RzLevn!qM^)z1|Lmsv4}i%G6v(Aw&d_%>4+a_4LvVj%Xr{Mdauy;bC}
z%F(yG&&LxBN*oy$ox;Uwr(<J(c>x=;mNAf7B$6_BJD|G*Es`*c%J`H3Z+8dzB+g;X
z=p0rIlUXuR=`5_d(*l8O;w-|}vE#1WmJ#&}jA1Efx}+y;kDK$f2~&*v<Inq|jaU?h
zw|vhUWyn>9ERZ#0UIMK_EbszclI9|+gUBx>e8-vF-ndsS+-<r-*`yH0mb&P~6!zHs
zS&2#w1MXoIu+%&*!JRWQl%Lk6KM^up%t}3X-eg)-kc4k>>)ev7*Hr86N%fkWt0C$V
zo)Z%l{`BB8(dxEMbsv9%r+bq;t8H4ApW5XksH)%ZPLHP4&=@v)s)8sy^~;$dA4T^h
z<+K7lr0VP)##2znTXYP3BdH0A;JgB(Kr$*D%GFin598KYGcGWDNiZyZL%i#^#6ujG
z<aEgzzYH-}o{bHg;H8bPa)GdV9KN$8xndG7WG0%Ypm~7HQ!sH?dr4_2r6*uc&y$`@
zLQ`pHv6yMiuc1IVAIp)wh+L{p4`%{z#p_s<7MxP)8BHZB(j}xm$B6H#B03_s;C=Pi
zUoorf;mJFl6RI4QpN=R#5OCe8hm_az3x-roEaWa`%(Uht3qmvGJ&EA^8Mijl@q=qP
zI`yYVme1FzQT(hkMw;}=i#eGc29sX?6pwSRF$_Ay{df84(AuL?l<zV$_C7gT&Tyue
zdJ*@{sM)7LGo$LJohPGm+jSnEg!l@)2Z!`-P{>BUCN*wN6Bu*ESCipRcBn{{TvJ#v
z#@+A1%lBbUH4vGFa(cBQpy%P&=Y}7<R$-xGgpqHI#76s^i=-EvOMeDo6ra9=X=oqv
znY6IKVlvV@ipqNQg|sI6CA%+4;O7IWTiyNbp1t8b5i9uMR}=+8(U!tuS5qzS#ldgp
zVrh#gpmdiDJldXYE`Hb!<tFBB*%gR4^%jsuS`vQDB0pgWa+)>`s2#XGJlN=tCg>_o
z3!s9virJ>yaP)}+zYjBsIB4BE_0>{c**J+y$3nb=64Jf$`K;4wjrrrEHv`kidB(4(
zRtQav#Al|MuDe|#Ll?;%99pmR_@0<q+;`1;^j79Mtf{tVan+)GpRjdzI+eH+JI-8o
zdui^5EnAbad1FxhdD-F1*(IXhKEnr+&zLPe{P}JnG%s*@V$vp5xuY8b0fv=flc}dL
zeS-n;h3u9lwczyZ%pmOhLleyCvM|IB+(qzD?WAwWuTt+I7d*jB*%Yk6NTV?EwKTNO
zM2x;gfm>uu|FA~;{8~()6NR`V^D@aonwpJTY@q~0g;V3K##O(v%A>uF^~N&`^OmA(
z@&HbcQq*`w_G&d|0ml@H4s|{{yHfIvAJ*_)C59~>;uWNz*T_xuct;t#))E(IV5XTT
zGMo{s(~rplSy-a+s(WKTiiC$5rumy7INLp(vwpKh3NtS)0|iW5JQUN$b2&+*)`-p?
zy)TK%aei$d5l33mpox=B?XvOX<T~s8m}KSID4b5c$NWVi8F0yPk>L##b;BT7!1d}$
znJKCEszmBDs$?q0C%(hxtlt!62X#VpAvI;Fj;5i}3ea*nG&PV!E`jixht$v9eH$X<
zyokJRt+|&_i;dJvnBRRBjUR3y76C(t_mp75{Dmr;YV<C+v~M1jVGm54C%LWs!J`a!
zWBp4Tt_b*?#22=$Km!n5=kE~S3z;8ZzkH#g@F1&hkw~i0D`C&q*;bVwoU{Vvz-XsL
zZ*#sOVOxrVnDZ)04SoB?!QwoTj-P|BbBZOOn4aiqo7zE&>ayvDy+GI?=XgQ;V-mKO
zD{^LqInM+a0WTAoFZtp;BkKuuE0rBIe#5I<CIq6b{i!iX{5X+VF%<i;54N_~IsD5P
zl=NkD@sG%OX>Pt(R02~)AcfkMWQ>?E^S9yTD-W(KI0zJRvT6t;{bajWs{uuXs|A*r
z1Cc}(otGplxn-YUa68-@?E@R3jgqU%T8`3erE`tf+P0eLTeJP8G9&W6@vdx@_v&B=
zT!kI>_`e@lt)tw2u)H*$lI8I-Z`lc~MnuZgO?*z+7xK+Y;NUROZ2LC60@dnK*}=jz
zZ+gmm-MR$}rtRbXef1t^?{Nh@?7&l*-}}L~k+O84RtYIBsJOwqyo#Hv-F*%AVhokx
zKCDMM?l{0;D;_lt$^BUFC$n*QG(GwywZi>-EygtWkB$)*@ff^lrl6&bl<I^c!e@5n
zd#`+zaeT!LQ|byxYEW(gli5^Nx3A3+uA~P$<D|#tMLW)B2iKtLBVFfDWYt9z75>;*
zH_vjRcyQ<gr1eQV{Aw7Q6z|B)^g7{^=wp3z!+IW0IuhmQJ-a09-?TOeG1LV)4hUam
zYcw-G#G9oyW3FeWt_8?8o_0#JjL~-Y%MUG7{Dy;NJ@8H~?x(lKrPBzxD7}<-;OYFl
z4bY+R&SD;h6itU6E0l2Tq4TZPVHv39zn;XhlF41#^L~MEu=(nY<tVPmKT+K;zEEdA
zHPj=*51OPeL3EPJaL|)>aBp>^hr7m4LByJzNoa8#;2#_p+;%c(09Eqq9FuSxYD5us
zw7WwO&FemD?`KvM%8>!=2Fdg<hLYd|CwWVK%8gA&%i#5L%a-_J5_7*PdfPLY6#5e^
z;c$B^eet|JKW8+5Z64(^q8Epu;Vk3U_%hi$7Mak@>IPOWB<6e#p9vaz=wk~T4u}P$
zEn=;s*!ig@l03p}%6?SOCwI)s!=Q~HfzR&>h2FnlyEujv+I!F1k6{j#7``4{cpK)P
z%sHW9hF;4{*fB?$7Pr_vnZASg5}eiQxRt{Ap&8`vM3?il6Bp@SfT@?~Jmu&c;)5TR
zJ1Ry(jbhU`wq%I_y8g4$lZ($_h>UJhl^^w-pMR#zYn{qaydOFvTwZf)rso^I-z{{B
zz{KG4fI`cCfVCZHQ*&0>LOHF1=>M#)%Q(#xR>(kito-ERC|YbVHHA~sj$NnZ__jP&
zyr4XMvS2lwDweU9dhcxXik446W^Fm3e-_7EliGrV+$CvXjvXBE@oEIW%@t&?Yxdo(
zNUa6|2glpAAm{#mVNj{IM(;t6E|fmBbW9U+5PP<3l=TB`d>Xt^Jq%^31*H31J&N%s
z>{%($rf!+c^?{lXC6rAln#EZNEW~8I{v{2Bwu9dODUSY6K~va5oWv{Y6G#}aBDC=d
zpPpF5n}1GXnud;x?0j8C9<=HpQE2)(LV#PwSb31x?~EDu%_LwJ>Ya6J4LP?;y-eNI
zvsFF3dFo3fZLH~?Q4(32HNscXiC+nspaE1T+dRctsf-<yNqn#@pEv3zM7P@lwBCuK
z<~VGGpB0`Nw6gUkDugf^!=UMYa9hp5BcLer?lD3-DApkOL|wa{44oUvPq@)5U7S<-
zHBxM0wtzq1kj3mPp9fV`;{#*F2o;B^3+ns9cqS*gyF}x#c^Wpq8KfTHtE*lSX*YAG
z!wm>Mb<p+iF&M#h92{VnMAu$x4xQb<evv!Fok-qbR1RypB$RYUl`}%dOqp)u*Jjf#
zeNWBALqlBn)@k-7gIc#pZ_pm6#ilxyd61ABM`w(5J~~xghxH*gupc{3l5S-!T#=gN
z*RDw=0$#}geoF|?iTPNAjAO53b&*c)IvAr{5SWESl^@=_qi|X?{ALF<o?Kj~!}cU6
z!}vA*&86u)J31Wjqq$>Q-}IotxF&=d5JRwnOib=j?)Xc3l~aQ7cZKmyhP|o}n=!le
z+#T~n1#lQERIM*~ZKPQr)xLG(sp{3;AAB&n7Az1V<J(jHDH^6mr8R!5E%6PENg(Tm
z#uT9$YA?6JUoyPjy$zBGj5PkCVSLSR(Gs`cR?CC!xy$tmVJ?sTg4@oi(4sL%7)oos
zK2B5$gE<{lH`vjdIyBy!%`2rU6>IWU<}NeNlTXoTad($<?MqAd>579g=67NuJWy<i
z9(t4EU#Qz5irfZ~RJJ0_`|NRM_)-V^sV|7ql9j|BJHjs??mMFn`_><K>|2ncAN+|1
zRfnLS41bsuX`qC6kRD36olYcYA2uXB9>KF{6`x5YJH~Qc`W~{I=7znV9xs(L!QBd*
zQU{mI_;8&ikS!ZBOZ4`<)MHIipz97A=RtNxZ6DOOvkq*aJNso#QRs$hh0zO%QkfA}
zlxa~=5LPmC>bEa;ERIHU`qVZGKYWdlTO~ItNvAT5m>VvvZ%7@u(T|pK;2btcEPIIv
z(nsaat+cl8vH9S71(i9G`7;jEkmHv#2G23o>TZx$Ckp0g?*6WCohFmb1LCSliz}H{
zXq%S7k=gAW>J>{>5NaX;$Gv**1;O9d5>jj&)0LS-Z=}8qmLC<yVS9X_=T{G#ZF<`G
zr@`~qx~1@VvXtm62EU9t*SKTCrAH5i++i7~*JD+6|M3Mco2V$-XGdqIsW?eZ+1y9<
ztvF^X{@%8dY7gcNm(GDdNI`n4fv0+;?rPt6H+Hy@_Kx&*H4s-n`a1Jcv$iIZ(ic6&
zhe7%Sn)O;Cnm4-OLL|G0_{b9+Eg|B3ivO~mG@t7P7C5pe5}UrSe}VT?-*<C)L-<v^
z&XvZuEw@RcvmDMWeNevG=3*Dbo(LZvT9)Mqo`f8;^6#>{Fk0?=^yF3c+Dph6o~|9U
zg_xO~3@3DDx`Cu&^PpeL^XZ+8N~^n9%kha`(UD%5;EY+pv%hOS9FW5bJ?{SUkskl{
zvuxMUH+Lj^;@UzJ4O-e)B$r~FismF!z`zDSj!lEFE5~T57p1m0Kk)WU$|qd$7{@X*
zjT1&GdU?Mg|5$Lv{nZ-^-$NBkDNsO5z8<<=?gx!zP1n)nZ4>|aV+71D<t|&Xz6qS)
zPr9{Vwu$}9+gmen;O7kkCn0bvaQVH~X^e|4MuAK4dNIx}DKKy*=Emi!7C}(Gh1aBF
zZ#53bG&E!s!C1UHpW5$~DMKpc+hma-lH`K(KGZ8u9;%C(mhy}9YERR?ALNoq8<nGz
zXCjq!C?HP%z^p!U!4o5OGvT0_AVaOECwlpbbiZ&p{AO&diQ>i^rXW{m>S7Pig(@gv
zTx*K)T~fS{zARVTCzjjC`XI9I&-?0rD9#N*<OwXEqBF;8MHC2Cx-QuJ+v(85PHLU}
zaLA_7q-QoI<7Qdd!gW;Ta#6%dm`4o<5ci@{F<(<QIgXMB(a4t%_n~#Kti|7s9o!Y_
zgtC0uQzC5%i}kl34cFa#G4RV!zlY%3e;nbD9=WPd7V9I=xMk4B@7bGxgI{gx*ZuHx
z_TBRPng?SuL90(Zd|W+>P1wwg#)8w4t$Gl=LuRM@XOYL*08HIu6^(agWIR>c4j|ry
z&sKx+@Ue#@?Ksl9Zo-&1TDUnZ3kfAVD?}(zf+bt1`Qy7*IVMNCtnJ;2XD!;zu!R#j
zQVr54d;V=9itDQSX-~`uuRM2q!b&G26Hh;~FGtV>%O=hcI)W3?f@T|7-nH-IQtqo)
z(-5R@MA2)gNL$G*sB|qKsQA4T#FJ{)?^sdn3GCVDC03P!=<F(tdMc8{ZapVnC#PMm
z+ziMZ>rgK#{g67H?Ki$I-K*u3H$wxLP(J;riYy-7K5|*mGXN9FdsSUy|0siPAchwU
zvgJ`k#3^FHxaOOHOv6OXwzXvq^Km*QFH6aQxG5$@iHG?;nm4v*?_@36duu}^v#Ws^
zWe9O&TYoP7wf%{@5nHAe5XNB(64`{z@4nuv&5W*0{ApD>ST$Aj!A2e;*NYJSGy1uu
z@xxFt4$lI~$Y?akgnSC4ddJ`w22nIM#aPECMGGnlGhU>@^vYMcn27vXsH_&o$I01X
zV>nQiZ=*RZ1`=|GKTxCMnfB3KIgAkm6^C;vCryZh9_h-(Dzk-pbAn{(GB1aIj%BLc
zZKypC?1G9%Qf{#!$AOP*^5G)cyyy?B7eQ#oB8`qw5I$zzadd;l3@v^Y#~_pI(<-Mg
zX6x;iE+bSkzO&PL=oQs9=43CWLZZ`5Cn*(dXOqF_Yc;VMV>RN&A3vC9L8e*coJNzL
zgK@~(YuQ^rHPeniCJG2Kmm7UPIC1}FX<yE^CKn}1kym~zlzL-_t9FU<{$6-8%b2`7
zbEExDMCIiR2NUjnI_7OKsz7My@wg%8P-e;CH3Ll&BejaM@28>QrqP!dl-)fj53QWG
z$pDdJwtGferB^$$C+6lYEsJ9W6zW3P=Z39ABBxy5)tFxmB~P8=n)<ep{zPf5p_Ukr
zxRTd%laMY!#F(S$<@DoiA{aUsj@|i?yRGGFL$65H6_dgw_%$KkdT^FGh@+_A$s07(
z-wW9Wrs-<nU*WJDyIBP|b|jOjvy0?&hmFMrl$2qms9vLqMrdQDX*okOq@j#?7dH2`
zQm1L+oeVS6pr%UOeWUGBX6UPBmXR2LUmdMX11sQkmJa6K^@=zJvb{|-0Zm&Y8glsh
zoyqE20Ah<bRgwvYFs+7VYSa&XhDWUK#Ka15IYjT{4Y7v#koE*Be9b+l^_Hv(Hph%A
z6}lm~MH(@|8*Q;em6zlv2(cZ6Y>N89t`#g?`DCwmd$glewPaNXBUvZ%u?S7RC~;xE
z+EZrdOHZqj)0~ta44SOhUE2s69D_@V7h)jQO(-r<sWBB#VC%+%;lStgj=#|Jh!-u7
z1(%kVanl8H|18~QMJZKiKhuG6Enb7qd$D^Ha`j+^!;0*efnG<QTdHk;X-CA5LVY_@
zq6lSe<xa`lEFf>j??uoEwqu9q>11;A?Y{sQOP<07+bYjAI+qlVFF~x#H7@nu7>V|c
zOfuG|kaVwoeKn0F37?+gNIKg4<X++&?L=$7!PsMQ7&EldR)}ts#IU!%^g`qKn(r;L
za_GW#M|1}M!S^l5n-FVMIbw~%s6ccBtWE5YocS`MA_6&vsgsw$x5g=xFHi8XW%+x?
znBlkym5vK&DFf?-p|wQz_yNnHw69Z2Zl6D528x9;=^4+SF8Lh-MHEyxYs9pqJ{F~z
za=8u_WlkZr-RBV&&=8}I_vD&7=(Qd-%nqFl<P$#hj6Z5X--8+v4~b8ueKnUzsM($W
z`hx&Bjrrb#>h#Bw#PG}Rq9iXEwS_tNeofcCwiwp8T=(?MYytIG2^f%WQNQLKo8H_5
zyX)gGCsvPC)E}<QM<cD<WD%`~VFcqneu0Fq`0)r)ng6K0=m%$s+*7c_f4T6~_57h^
znOz;js`QoXNBKvn8llX_{VKI3Xz}k2&9zI0x-L(iYH(xRuGxkh_WrD)v_kJlYZ2#y
zVGS1FJ98)Fiu4RKt$i4c=;ux~#YWd!-P(Iw*gwh6)$f1a@1iW8!5~ss3V+{p0|$pz
zMlY>rS{yRI54x|rjUem^7u*KcF#1_0<#0Ixy`y)5a3)6kXTZ}n`WYeQ^4E4Ds6NV(
zociIeRzVuSFr^p{^#k^6lY3OC8{ttA;{@5;wBGB<ZoY$F8n>pXph_pz&OEp`dFxCO
zN*n$6>$h`FJRMFVW$b=e>04;N<=kVU^XXuGM7;BeHdZd5Z00&Uci7rKXSd8va}LB{
z&(vsCt9}AyOQ?`=8yJehv&57CRRj4g6h*}+h-$i<6TO#_fO$L}3T*0JmT4>MOqS84
zK4(^suin!)M3_IG-Q;0&E-HC_e-&qQ<o;^oWWeucQhlj5&vQiO;3-;2yy8XtlE4il
z;r-;;qtz$Qtn&py?2#oB)uA6XVaz5pR;oEnanB0D#V$PDC(77sKaBN_s9)&8z8Nme
zvxp7secSuEA;PH9k-rWrh#11sqe+Pu{p|CB;UIU{FTb_O3suXz*HpjWiWF<h5ouUK
zGy|kD0Ca5!M#YMMcFIPBbe>w4O*L%=Gk!977wH}YJSjdLIvfpigZGj=1`eNG7Ka#-
zt!lbo;H;zuWHT=8X{(wMeLIX@Q*6wAscVLi!t29O0XApGH#^v-dv=HJ5PO<72;5UY
z#I^Gn!3{Zz7Qc^nqZP+V^4Fin1vyuuzBSS3gbJ|bx*NA%+%LG6VMfwX3#Hh5A-{ef
zgT+x1KJYyAmq1%fOkXBh7MPq>=;iceU)s*fbEX_T-DywJ)axjp+hKQ6r#!qmdc?XS
zo8Fl>La-Gu^*rhd-Ww~K9KP)4R-IWb;&6aYv33>-v;Syska4~SExEj_KJb2Jh`0O2
zO=O}OiMh0F=sE8h4_uFA1j6V;%Ns8OsaH5f428YQR<ReMV4_5>dSJ6<9bKX|ZpY{#
zGrlyQc@L<llgxc?`=agi_DnbDrfSZDRg${2PfUFPVZH~Oh-hea$Z|O8-6=)!X)+=)
z!chop)SdjXy%70AFZ2M$dWAlBLXE?+6Lx)+Hc`4)WxQFFUJ6a2BBc>m)g*C)L#v_6
zjIOIe+PA!Q^ppu)YNr5LVkzWj9O_}YD@SZ&q+<nVL*3B3Q_Z>uoyypJ@N?`KMzpz}
zPvcBp@20OE15MTF`3K0a-FGS|c=J@@2s9W2KE||F4lvYV;;P$B_9HEk%LbUTccgG_
z&&afK8uh|fvYg*8h>Il{eW$Z}GszmGjf%VlLtPz=TD(7oh=nL&`C)Gwy8VW`*=nV-
zpcALEZe5AFDYP`W{;B-rU_FvieL}s?{)b7vL%By7$tKHP7emh`lyve(K0kkHl^(aF
zWr)~<$?Bpucnlpkad~SP+aMK<G-w)FXeVg;1&l<DlC>gUnSS^7qQadQU;8jBWiGvZ
z>3I_W%)hsp<&UbM{4)!@>=~L+fr15~U;!vt_$Mn^0Gd94rjNg?&lRYL{G%((fsh13
z5(vqEs>Wv^B!Q3wLJ|nczclpYH^|VRnO*<uypi8H`rrCeWI+W%j^kOhMBfN%hdQV_
z?{e#mlJJK=#2fffbsa84BhW^WQ)r$*=1UWZy}edI8jK34GiL7%Ys=uws5#jbZVtYB
z5Y-p6#2E(rXet;cI!kEx+SDZOXAh1+L!4!;6?_~H7tjWYn~<#}r%ZMr;QMbiRN!G?
zaF=)ukT8C?gpm!(401eRRgqw!uA3r0(DWXdQSlcB3YA6s!vc=>yB!VSXaGk8I2ypw
z{ws4d0K)(b127E0FaX2;@(joS3eVnuRbmc+J^=av=>L;}J{`!3bFi?+SLBeM0;K`^
z{8KYW!9TBf;U6dt{@q6#ncQL<{HtC;3k=i!ck`eBGd=b-F|^LsFho`(P<bZn4Hc9L
z<iwJ{I($b2pXyZdigvxdMbP{I(|GFt?CbsG#|;0rGdk|ojswCNAdLNaI1Ehi`kN+r
z0Sp#kumFSoCu6X{Zi9cnj^j7@4gmT9=>Gu=UjPpS3Soii=l^jL(r?7YZ>M{I{|=_#
zI2r(b0Q3RS|0mor1>};yGnd2$W%z#!uSq^m0o~iH;u|+ppTw{2)o1^fRd&AtqyInz
z{~M3S)rH#yL_|PD1Vluj6ZJ1|@%-n6K7hOe<P{*V0D1lEHW33D_QwfRe<SJqtG7n~
z#%~3HJ^=av=mVe+fc{^*<sGOHW@myds5-L)48~;p-)F!1U$Gz#kaqxi2atCFc?V$<
z=%dj)p&SmGOi7LOrB1k_aJj?&pD=F|XgmUL$zC!*t@xdi0bnq|V1KU00(cnUVStAL
zJS-@n)1;RS(DZ*N8UO|Z4EE>3C*Wa#hXEc2JhK7MY{07#@M`pyLMe(|@*mLvxFvyG
z61XM*V_X38&hLy1z+iyE{#+mdJPhzKz{3E=6;NEMt=H>$E`BE(08Ae+*q;lZfQJDd
z26z~dO9F~3ptu5xE1<a2IFADTlK+6N9e{@cJnYZsVgAnkAaMVCU{QU{a)CF<zkKfN
mXF=`|Mjk;xLvO+S|K$S;0&?rL4uOWr!wCMbe*mGO`M&@R<KJBX
literal 0
HcmV?d00001
diff --git a/tests/fsck-tests/020-extent-ref-cases/test.sh b/tests/fsck-tests/020-extent-ref-cases/test.sh
index c2b6a006..5dc5e55d 100755
--- a/tests/fsck-tests/020-extent-ref-cases/test.sh
+++ b/tests/fsck-tests/020-extent-ref-cases/test.sh
@@ -8,16 +8,23 @@
# * shared_data_ref
# * no_inline_ref (a extent item without inline ref)
# * no_skinny_ref
+#
+# Special check for lowmem regression
+# * block_group_item_false_alert
+# Containing a block group and its first extent at
+# the beginning of leaf.
+# Which caused false alert for lowmem mode.
source $TOP/tests/common
check_prereq btrfs
-for img in *.img
+for img in *.img *.raw.xz
do
image=$(extract_image $img)
- run_check_stdout $TOP/btrfs check "$image" 2>&1 |
- grep -q "Errors found in extent allocation tree or chunk allocation" &&
- _fail "unexpected error occurred when checking $img"
+
+ # Since the return value bug is already fixed, we don't need
+ # the old grep hack to detect bug.
+ run_check $TOP/btrfs check "$image"
rm -f "$image"
done
--
2.11.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 3/9] btrfs-progs: fsck: Output verbose error when fsck found a bug
2017-01-23 9:13 [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Qu Wenruo
2017-01-23 9:13 ` [PATCH 1/9] btrfs-progs: lowmem check: Fix wrong block group check when search slot points to slot beyong leaf Qu Wenruo
2017-01-23 9:13 ` [PATCH 2/9] btrfs-progs: fsck-test: Add test image for lowmem mode block group false alert Qu Wenruo
@ 2017-01-23 9:13 ` Qu Wenruo
2017-01-23 9:13 ` [PATCH 4/9] btrfs-progs: lowmem check: Fix false alert in checking data extent pointing to prealloc extent Qu Wenruo
` (7 subsequent siblings)
10 siblings, 0 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-23 9:13 UTC (permalink / raw)
To: linux-btrfs; +Cc: calestyo, chris, dsterba
Although we output error like "errors found in extent allocation tree or
chunk allocation", but we lacks such output for other trees, but leaving
the final "found error is %d" to catch the last return value(and
sometime it's cleared)
This patch adds extra error message for top level error path, and modify
the last "found error is %d" to "error(s) found" or "no error found".
Cc: Christoph Anton Mitterer <calestyo@scientia.net>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
cmds-check.c | 43 +++++++++++++++++++++++++++++++++----------
1 file changed, 33 insertions(+), 10 deletions(-)
diff --git a/cmds-check.c b/cmds-check.c
index c39392b7..f158daf9 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -12913,8 +12913,10 @@ int cmd_check(int argc, char **argv)
ret = repair_root_items(info);
err |= !!ret;
- if (ret < 0)
+ if (ret < 0) {
+ error("failed to repair root items: %s", strerror(-ret));
goto close_out;
+ }
if (repair) {
fprintf(stderr, "Fixed %d roots.\n", ret);
ret = 0;
@@ -12937,8 +12939,13 @@ int cmd_check(int argc, char **argv)
}
ret = check_space_cache(root);
err |= !!ret;
- if (ret)
+ if (ret) {
+ if (btrfs_fs_compat_ro(info, FREE_SPACE_TREE))
+ error("errors found in free space tree");
+ else
+ error("errors found in free space cache");
goto out;
+ }
/*
* We used to have to have these hole extents in between our real
@@ -12954,22 +12961,28 @@ int cmd_check(int argc, char **argv)
else
ret = check_fs_roots(root, &root_cache);
err |= !!ret;
- if (ret)
+ if (ret) {
+ error("errors found in fs roots");
goto out;
+ }
fprintf(stderr, "checking csums\n");
ret = check_csums(root);
err |= !!ret;
- if (ret)
+ if (ret) {
+ error("errors found in csum tree");
goto out;
+ }
fprintf(stderr, "checking root refs\n");
/* For low memory mode, check_fs_roots_v2 handles root refs */
if (check_mode != CHECK_MODE_LOWMEM) {
ret = check_root_refs(root, &root_cache);
err |= !!ret;
- if (ret)
+ if (ret) {
+ error("errors found in root refs");
goto out;
+ }
}
while (repair && !list_empty(&root->fs_info->recow_ebs)) {
@@ -12980,8 +12993,10 @@ int cmd_check(int argc, char **argv)
list_del_init(&eb->recow);
ret = recow_extent_buffer(root, eb);
err |= !!ret;
- if (ret)
+ if (ret) {
+ error("fails to fix transid errors");
break;
+ }
}
while (!list_empty(&delete_items)) {
@@ -13000,13 +13015,17 @@ int cmd_check(int argc, char **argv)
fprintf(stderr, "checking quota groups\n");
ret = qgroup_verify_all(info);
err |= !!ret;
- if (ret)
+ if (ret) {
+ error("failed to check quota groups");
goto out;
+ }
report_qgroups(0);
ret = repair_qgroups(info, &qgroups_repaired);
err |= !!ret;
- if (err)
+ if (err) {
+ error("failed to repair quota groups");
goto out;
+ }
ret = 0;
}
@@ -13027,8 +13046,12 @@ out:
"backup data and re-format the FS. *\n\n");
err |= 1;
}
- printf("found %llu bytes used err is %d\n",
- (unsigned long long)bytes_used, ret);
+ printf("found %llu bytes used, ",
+ (unsigned long long)bytes_used);
+ if (err)
+ printf("error(s) found\n");
+ else
+ printf("no error found\n");
printf("total csum bytes: %llu\n",(unsigned long long)total_csum_bytes);
printf("total tree bytes: %llu\n",
(unsigned long long)total_btree_bytes);
--
2.11.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 4/9] btrfs-progs: lowmem check: Fix false alert in checking data extent pointing to prealloc extent
2017-01-23 9:13 [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Qu Wenruo
` (2 preceding siblings ...)
2017-01-23 9:13 ` [PATCH 3/9] btrfs-progs: fsck: Output verbose error when fsck found a bug Qu Wenruo
@ 2017-01-23 9:13 ` Qu Wenruo
2017-01-23 9:13 ` [PATCH 5/9] btrfs-progs: lowmem check: Fix extent item size false alert Qu Wenruo
` (6 subsequent siblings)
10 siblings, 0 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-23 9:13 UTC (permalink / raw)
To: linux-btrfs; +Cc: calestyo, chris, dsterba
Btrfs lowmem check can report false csum error like:
ERROR: root 5 EXTENT_DATA[257 0] datasum missing
ERROR: root 5 EXTENT_DATA[257 4096] prealloc shouldn't have datasum
This is because lowmem check code always compare the found csum size
with the whole extent which data extents points to.
Normally it's OK, but when prealloc extent is written, or reflink is
done, data extent can points to part of a larger extent, making the csum
check wrong.
The fix changes the csum check part to the data extent size, other than
the disk_bytenr/disk_num_bytes which points to a larger extent.
Reported-by: Chris Murphy <chris@colorremedies.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
cmds-check.c | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/cmds-check.c b/cmds-check.c
index f158daf9..fd176b76 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -4695,6 +4695,7 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_key *fkey,
u64 disk_bytenr;
u64 disk_num_bytes;
u64 extent_num_bytes;
+ u64 extent_offset;
u64 found;
unsigned int extent_type;
unsigned int is_hole;
@@ -4731,17 +4732,28 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_key *fkey,
disk_bytenr = btrfs_file_extent_disk_bytenr(node, fi);
disk_num_bytes = btrfs_file_extent_disk_num_bytes(node, fi);
extent_num_bytes = btrfs_file_extent_num_bytes(node, fi);
+ extent_offset = btrfs_file_extent_offset(node, fi);
is_hole = (disk_bytenr == 0) && (disk_num_bytes == 0);
- /* Check EXTENT_DATA datasum */
- ret = count_csum_range(root, disk_bytenr, disk_num_bytes, &found);
+ /*
+ * Check EXTENT_DATA datasum
+ *
+ * We should only check the range we're referring to, as it's possible
+ * that part of prealloc extent has been written, and has csum:
+ *
+ * |<------- Original large preallocate extent A -------->|
+ * |<- Prealloc File Extent ->|<- Regular Extent ->|
+ * No csum Has csum
+ */
+ ret = count_csum_range(root, disk_bytenr + extent_offset,
+ extent_num_bytes, &found);
if (found > 0 && nodatasum) {
err |= ODD_CSUM_ITEM;
error("root %llu EXTENT_DATA[%llu %llu] nodatasum shouldn't have datasum",
root->objectid, fkey->objectid, fkey->offset);
} else if (extent_type == BTRFS_FILE_EXTENT_REG && !nodatasum &&
!is_hole &&
- (ret < 0 || found == 0 || found < disk_num_bytes)) {
+ (ret < 0 || found == 0 || found < extent_num_bytes)) {
err |= CSUM_ITEM_MISSING;
error("root %llu EXTENT_DATA[%llu %llu] datasum missing",
root->objectid, fkey->objectid, fkey->offset);
--
2.11.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 5/9] btrfs-progs: lowmem check: Fix extent item size false alert
2017-01-23 9:13 [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Qu Wenruo
` (3 preceding siblings ...)
2017-01-23 9:13 ` [PATCH 4/9] btrfs-progs: lowmem check: Fix false alert in checking data extent pointing to prealloc extent Qu Wenruo
@ 2017-01-23 9:13 ` Qu Wenruo
2017-01-23 9:13 ` [PATCH 6/9] btrfs-progs: tests: Move fsck-tests/015 to fuzz tests Qu Wenruo
` (5 subsequent siblings)
10 siblings, 0 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-23 9:13 UTC (permalink / raw)
To: linux-btrfs; +Cc: calestyo, chris, dsterba
If one extent item has no inline ref, btrfs lowmem mode check can give
false alert without outputting any error message.
The problem is lowmem mode always assume that extent item has inline
refs, and when it encounters such case it flags the extent item has
wrong size, but doesn't output the error message.
Although we already have such image submitted, at the commit time due to
another bug in cmds-check return value, it doesn't detect it until that
bug is fixed.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
cmds-check.c | 14 ++++++++++----
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/cmds-check.c b/cmds-check.c
index fd176b76..802d179f 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -10730,13 +10730,20 @@ static int check_extent_item(struct btrfs_fs_info *fs_info,
}
end = (unsigned long)ei + item_size;
- if (ptr >= end) {
+next:
+ /* Reached extent item end normally */
+ if (ptr == end)
+ goto out;
+
+ /* Beyond extent item end, wrong item size */
+ if (ptr > end) {
err |= ITEM_SIZE_MISMATCH;
+ error("extent item at bytenr %llu slot %d has wrong size",
+ eb->start, slot);
goto out;
}
/* Now check every backref in this extent item */
-next:
iref = (struct btrfs_extent_inline_ref *)ptr;
type = btrfs_extent_inline_ref_type(eb, iref);
offset = btrfs_extent_inline_ref_offset(eb, iref);
@@ -10773,8 +10780,7 @@ next:
}
ptr += btrfs_extent_inline_ref_size(type);
- if (ptr < end)
- goto next;
+ goto next;
out:
return err;
--
2.11.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 6/9] btrfs-progs: tests: Move fsck-tests/015 to fuzz tests
2017-01-23 9:13 [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Qu Wenruo
` (4 preceding siblings ...)
2017-01-23 9:13 ` [PATCH 5/9] btrfs-progs: lowmem check: Fix extent item size false alert Qu Wenruo
@ 2017-01-23 9:13 ` Qu Wenruo
2017-01-23 9:13 ` [PATCH 7/9] btrfs-progs: fsck-tests: Make 013 compatible with lowmem mode Qu Wenruo
` (4 subsequent siblings)
10 siblings, 0 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-23 9:13 UTC (permalink / raw)
To: linux-btrfs; +Cc: calestyo, chris, dsterba
The test case fsck-tests/015-check-bad-memory-access can't be repair by
btrfs check, and it's a fortunate bug makes original mode to forget the
error code from extent tree, making original mode pass it.
So fuzz-tests is more suitable for it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
.../images}/bko-97171-btrfs-image.raw.txt | 0
.../images}/bko-97171-btrfs-image.raw.xz | Bin
2 files changed, 0 insertions(+), 0 deletions(-)
rename tests/{fsck-tests/015-check-bad-memory-access => fuzz-tests/images}/bko-97171-btrfs-image.raw.txt (100%)
rename tests/{fsck-tests/015-check-bad-memory-access => fuzz-tests/images}/bko-97171-btrfs-image.raw.xz (100%)
diff --git a/tests/fsck-tests/015-check-bad-memory-access/bko-97171-btrfs-image.raw.txt b/tests/fuzz-tests/images/bko-97171-btrfs-image.raw.txt
similarity index 100%
rename from tests/fsck-tests/015-check-bad-memory-access/bko-97171-btrfs-image.raw.txt
rename to tests/fuzz-tests/images/bko-97171-btrfs-image.raw.txt
diff --git a/tests/fsck-tests/015-check-bad-memory-access/bko-97171-btrfs-image.raw.xz b/tests/fuzz-tests/images/bko-97171-btrfs-image.raw.xz
similarity index 100%
rename from tests/fsck-tests/015-check-bad-memory-access/bko-97171-btrfs-image.raw.xz
rename to tests/fuzz-tests/images/bko-97171-btrfs-image.raw.xz
--
2.11.0
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 7/9] btrfs-progs: fsck-tests: Make 013 compatible with lowmem mode
2017-01-23 9:13 [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Qu Wenruo
` (5 preceding siblings ...)
2017-01-23 9:13 ` [PATCH 6/9] btrfs-progs: tests: Move fsck-tests/015 to fuzz tests Qu Wenruo
@ 2017-01-23 9:13 ` Qu Wenruo
2017-01-23 9:13 ` [PATCH 8/8] btrfs-progs: fsck: Fix lowmem mode override to allow it skip repair work Qu Wenruo
` (3 subsequent siblings)
10 siblings, 0 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-23 9:13 UTC (permalink / raw)
To: linux-btrfs; +Cc: calestyo, chris, dsterba
fsck-tests/013-extent-tree-rebuild uses "--init-extent-tree", which
implies "--repair".
But the test script doesn't specify "--repair" for lowmem mode test to
detect it.
Add it so lowmem mode test can be happy with it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
tests/fsck-tests/013-extent-tree-rebuild/test.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/fsck-tests/013-extent-tree-rebuild/test.sh b/tests/fsck-tests/013-extent-tree-rebuild/test.sh
index 37bdcd9c..08c1e50e 100755
--- a/tests/fsck-tests/013-extent-tree-rebuild/test.sh
+++ b/tests/fsck-tests/013-extent-tree-rebuild/test.sh
@@ -36,7 +36,7 @@ test_extent_tree_rebuild()
$SUDO_HELPER $TOP/btrfs check $TEST_DEV >& /dev/null && \
_fail "btrfs check should detect failure"
- run_check $SUDO_HELPER $TOP/btrfs check --init-extent-tree $TEST_DEV
+ run_check $SUDO_HELPER $TOP/btrfs check --repair --init-extent-tree $TEST_DEV
run_check $SUDO_HELPER $TOP/btrfs check $TEST_DEV
}
--
2.11.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 8/8] btrfs-progs: fsck: Fix lowmem mode override to allow it skip repair work
2017-01-23 9:13 [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Qu Wenruo
` (6 preceding siblings ...)
2017-01-23 9:13 ` [PATCH 7/9] btrfs-progs: fsck-tests: Make 013 compatible with lowmem mode Qu Wenruo
@ 2017-01-23 9:13 ` Qu Wenruo
2017-01-23 9:13 ` [PATCH 8/9] btrfs-progs: fsck-tests: Add new test case for partly written prealloc extent Qu Wenruo
` (2 subsequent siblings)
10 siblings, 0 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-23 9:13 UTC (permalink / raw)
To: linux-btrfs; +Cc: calestyo, chris, dsterba, Lu Fengqi
From: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Current common.local doesn't handle lowmem mode well.
It passes "--mode=lowmem" alone with "--repair", making it unable to
check lowmem mode.
It's caused by the following bugs:
1) Wrong variable in test/common.local
We should check TEST_ARGS_CHECK, not TEST_CHECK, which is not defined
so we never return 1.
2) Wrong parameter passed to _cmd_spec() in test/common
This prevents us from grepping the correct parameters.
Fix it.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
---
tests/common | 8 ++++----
tests/common.local | 2 +-
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/tests/common b/tests/common
index 51c2e267..7ad436e3 100644
--- a/tests/common
+++ b/tests/common
@@ -106,7 +106,7 @@ run_check()
ins=$(_get_spec_ins "$@")
spec=$(($ins-1))
cmd=$(eval echo "\${$spec}")
- spec=$(_cmd_spec "$cmd")
+ spec=$(_cmd_spec "${@:$spec}")
set -- "${@:1:$(($ins-1))}" $spec "${@: $ins}"
echo "############### $@" >> "$RESULTS" 2>&1
if [[ $TEST_LOG =~ tty ]]; then echo "CMD: $@" > /dev/tty; fi
@@ -128,7 +128,7 @@ run_check_stdout()
ins=$(_get_spec_ins "$@")
spec=$(($ins-1))
cmd=$(eval echo "\${$spec}")
- spec=$(_cmd_spec "$cmd")
+ spec=$(_cmd_spec "${@:$spec}")
set -- "${@:1:$(($ins-1))}" $spec "${@: $ins}"
echo "############### $@" >> "$RESULTS" 2>&1
if [[ $TEST_LOG =~ tty ]]; then echo "CMD(stdout): $@" > /dev/tty; fi
@@ -152,7 +152,7 @@ run_mayfail()
ins=$(_get_spec_ins "$@")
spec=$(($ins-1))
cmd=$(eval echo "\${$spec}")
- spec=$(_cmd_spec "$cmd")
+ spec=$(_cmd_spec "${@:$spec}")
set -- "${@:1:$(($ins-1))}" $spec "${@: $ins}"
echo "############### $@" >> "$RESULTS" 2>&1
if [[ $TEST_LOG =~ tty ]]; then echo "CMD(mayfail): $@" > /dev/tty; fi
@@ -188,7 +188,7 @@ run_mustfail()
ins=$(_get_spec_ins "$@")
spec=$(($ins-1))
cmd=$(eval echo "\${$spec}")
- spec=$(_cmd_spec "$cmd")
+ spec=$(_cmd_spec "${@:$spec}")
set -- "${@:1:$(($ins-1))}" $spec "${@: $ins}"
echo "############### $@" >> "$RESULTS" 2>&1
if [[ $TEST_LOG =~ tty ]]; then echo "CMD(mustfail): $@" > /dev/tty; fi
diff --git a/tests/common.local b/tests/common.local
index 9f567c27..4f56bb08 100644
--- a/tests/common.local
+++ b/tests/common.local
@@ -17,7 +17,7 @@ TEST_ARGS_CHECK=--mode=lowmem
# break tests
_skip_spec()
{
- if echo "$TEST_CHECK" | grep -q 'mode=lowmem' &&
+ if echo "$TEST_ARGS_CHECK" | grep -q 'mode=lowmem' &&
echo "$@" | grep -q -- '--repair'; then
return 0
fi
--
2.11.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 8/9] btrfs-progs: fsck-tests: Add new test case for partly written prealloc extent
2017-01-23 9:13 [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Qu Wenruo
` (7 preceding siblings ...)
2017-01-23 9:13 ` [PATCH 8/8] btrfs-progs: fsck: Fix lowmem mode override to allow it skip repair work Qu Wenruo
@ 2017-01-23 9:13 ` Qu Wenruo
2017-01-23 9:13 ` [PATCH 9/9] btrfs-progs: fsck: Fix lowmem mode override to allow it skip repair work Qu Wenruo
2017-01-24 16:54 ` [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Christoph Anton Mitterer
10 siblings, 0 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-23 9:13 UTC (permalink / raw)
To: linux-btrfs; +Cc: calestyo, chris, dsterba
This is a bug found in lowmem mode, which reports false alert for partly
written prealloc extent.
Reported-by: Chris Murphy <chris@colorremedies.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
tests/fsck-tests/020-extent-ref-cases/test.sh | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/tests/fsck-tests/020-extent-ref-cases/test.sh b/tests/fsck-tests/020-extent-ref-cases/test.sh
index 5dc5e55d..91340671 100755
--- a/tests/fsck-tests/020-extent-ref-cases/test.sh
+++ b/tests/fsck-tests/020-extent-ref-cases/test.sh
@@ -18,6 +18,7 @@
source $TOP/tests/common
check_prereq btrfs
+check_global_prereq xfs_io
for img in *.img *.raw.xz
do
@@ -28,3 +29,17 @@ do
run_check $TOP/btrfs check "$image"
rm -f "$image"
done
+
+# Extra test case for partly written prealloc extents.
+test_prealloc_written()
+{
+ run_check $SUDO_HELPER $TOP/mkfs.btrfs -f $TEST_DEV
+
+ run_check_mount_test_dev
+ xfs_io -f -c "falloc 0 128k" -c "syncfs" $TEST_MNT/tmpfile
+ xfs_io -c "pwrite 0 64k" $TEST_MNT/tmpfile
+
+ run_check_umount_test_dev
+
+ run_check $TOP/btrfs check $TEST_DEV
+}
--
2.11.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 9/9] btrfs-progs: fsck: Fix lowmem mode override to allow it skip repair work
2017-01-23 9:13 [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Qu Wenruo
` (8 preceding siblings ...)
2017-01-23 9:13 ` [PATCH 8/9] btrfs-progs: fsck-tests: Add new test case for partly written prealloc extent Qu Wenruo
@ 2017-01-23 9:13 ` Qu Wenruo
2017-01-24 16:54 ` [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Christoph Anton Mitterer
10 siblings, 0 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-23 9:13 UTC (permalink / raw)
To: linux-btrfs; +Cc: calestyo, chris, dsterba, Lu Fengqi
From: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Current common.local doesn't handle lowmem mode well.
It passes "--mode=lowmem" alone with "--repair", making it unable to
check lowmem mode.
It's caused by the following bugs:
1) Wrong variable in test/common.local
We should check TEST_ARGS_CHECK, not TEST_CHECK, which is not defined
so we never return 1.
2) Wrong parameter passed to _cmd_spec() in test/common
This prevents us from grepping the correct parameters.
Fix it.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
---
tests/common | 8 ++++----
tests/common.local | 2 +-
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/tests/common b/tests/common
index 51c2e267..7ad436e3 100644
--- a/tests/common
+++ b/tests/common
@@ -106,7 +106,7 @@ run_check()
ins=$(_get_spec_ins "$@")
spec=$(($ins-1))
cmd=$(eval echo "\${$spec}")
- spec=$(_cmd_spec "$cmd")
+ spec=$(_cmd_spec "${@:$spec}")
set -- "${@:1:$(($ins-1))}" $spec "${@: $ins}"
echo "############### $@" >> "$RESULTS" 2>&1
if [[ $TEST_LOG =~ tty ]]; then echo "CMD: $@" > /dev/tty; fi
@@ -128,7 +128,7 @@ run_check_stdout()
ins=$(_get_spec_ins "$@")
spec=$(($ins-1))
cmd=$(eval echo "\${$spec}")
- spec=$(_cmd_spec "$cmd")
+ spec=$(_cmd_spec "${@:$spec}")
set -- "${@:1:$(($ins-1))}" $spec "${@: $ins}"
echo "############### $@" >> "$RESULTS" 2>&1
if [[ $TEST_LOG =~ tty ]]; then echo "CMD(stdout): $@" > /dev/tty; fi
@@ -152,7 +152,7 @@ run_mayfail()
ins=$(_get_spec_ins "$@")
spec=$(($ins-1))
cmd=$(eval echo "\${$spec}")
- spec=$(_cmd_spec "$cmd")
+ spec=$(_cmd_spec "${@:$spec}")
set -- "${@:1:$(($ins-1))}" $spec "${@: $ins}"
echo "############### $@" >> "$RESULTS" 2>&1
if [[ $TEST_LOG =~ tty ]]; then echo "CMD(mayfail): $@" > /dev/tty; fi
@@ -188,7 +188,7 @@ run_mustfail()
ins=$(_get_spec_ins "$@")
spec=$(($ins-1))
cmd=$(eval echo "\${$spec}")
- spec=$(_cmd_spec "$cmd")
+ spec=$(_cmd_spec "${@:$spec}")
set -- "${@:1:$(($ins-1))}" $spec "${@: $ins}"
echo "############### $@" >> "$RESULTS" 2>&1
if [[ $TEST_LOG =~ tty ]]; then echo "CMD(mustfail): $@" > /dev/tty; fi
diff --git a/tests/common.local b/tests/common.local
index 9f567c27..4f56bb08 100644
--- a/tests/common.local
+++ b/tests/common.local
@@ -17,7 +17,7 @@ TEST_ARGS_CHECK=--mode=lowmem
# break tests
_skip_spec()
{
- if echo "$TEST_CHECK" | grep -q 'mode=lowmem' &&
+ if echo "$TEST_ARGS_CHECK" | grep -q 'mode=lowmem' &&
echo "$@" | grep -q -- '--repair'; then
return 0
fi
--
2.11.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-01-23 9:13 [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Qu Wenruo
` (9 preceding siblings ...)
2017-01-23 9:13 ` [PATCH 9/9] btrfs-progs: fsck: Fix lowmem mode override to allow it skip repair work Qu Wenruo
@ 2017-01-24 16:54 ` Christoph Anton Mitterer
2017-01-25 0:44 ` Qu Wenruo
10 siblings, 1 reply; 29+ messages in thread
From: Christoph Anton Mitterer @ 2017-01-24 16:54 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs; +Cc: chris, dsterba
[-- Attachment #1: Type: text/plain, Size: 6669 bytes --]
Hey Qu.
I was giving your patches a try, again on the very same fs (which saw
however writes in the meantime), from my initial report.
btrfs-progs v4.9 WITHOUT patch:
*******************************
# btrfs check /dev/nbd0 ; echo $?
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used err is 0
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
0
# btrfs check --mode=lowmem /dev/nbd0 ; echo $?
checking extents
ERROR: block group[74117545984 1073741824] used 1073741824 but extent items used 0
ERROR: block group[239473786880 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[500393050112 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[581997428736 1073741824] used 1073741824 but extent items used 0
ERROR: block group[626557714432 1073741824] used 1073741824 but extent items used 0
ERROR: block group[668433645568 1073741824] used 1073741824 but extent items used 0
ERROR: block group[948680261632 1073741824] used 1073741824 but extent items used 0
ERROR: block group[982503129088 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1039411445760 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1054443831296 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[1190809042944 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1279392743424 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1481256206336 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1620842643456 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[1914511032320 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[3055361720320 1073741824] used 1073741824 but extent items used 0
ERROR: block group[3216422993920 1073741824] used 1073741824 but extent items used 0
ERROR: block group[3670615785472 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[3801612288000 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[3828455833600 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[4250973241344 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4261710659584 1073741824] used 1073741824 but extent items used 1074266112
ERROR: block group[4392707162112 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4558063403008 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4607455526912 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4635372814336 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4640204652544 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4642352136192 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[4681006841856 1073741824] used 1073741824 but extent items used 0
ERROR: block group[5063795802112 1073741824] used 1073741824 but extent items used 0
ERROR: block group[5171169984512 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[5216267141120 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[5290355326976 1073741824] used 1073741824 but extent items used 0
ERROR: block group[5445511020544 1073741824] used 1073741824 but extent items used 1074266112
ERROR: block group[6084387405824 1073741824] used 1073741824 but extent items used 0
ERROR: block group[6104788500480 1073741824] used 1073741824 but extent items used 0
ERROR: block group[6878956355584 1073741824] used 1073741824 but extent items used 0
ERROR: block group[6997067956224 1073741824] used 1073741824 but extent items used 0
ERROR: block group[7702516334592 1073741824] used 1073741824 but extent items used 0
ERROR: block group[8051482427392 1073741824] used 1073741824 but extent items used 1084751872
ERROR: block group[8116980678656 1073741824] used 1073741824 but extent items used 0
ERROR: errors found in extent allocation tree or chunk allocation
checking free space cache
checking fs roots
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used err is -5
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
1
=> so the fs would still show the symptoms
Then, with no RW mount to the fs in between, 4.9 with the following of
your patches:
0001-btrfs-progs-lowmem-check-Fix-wrong-block-group-check.patch
0003-btrfs-progs-fsck-Output-verbose-error-when-fsck-foun.patch
0004-btrfs-progs-lowmem-check-Fix-false-alert-in-checking.patch
0005-btrfs-progs-lowmem-check-Fix-extent-item-size-false-.patch
the others were anyway just tests related, AFAICS.
btrfs-progs v4.9 WITH patches:
****************************
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used, no error found
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
0
=> bon!
# btrfs check --mode=lowmem /dev/nbd0 ; echo $?
checking extents
checking free space cache
checking fs roots
ERROR: root 6031 EXTENT_DATA[277 524288] datasum missing
ERROR: errors found in fs roots
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used, error(s) found
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
1
=> some new errors :-(
Not sure though, whether this is an issue in your patch or a new
problem due to my writes (I did just another round of incremental send
-p/receive)
Any ideas?
Best,
Chris.
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5930 bytes --]
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-01-24 16:54 ` [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Christoph Anton Mitterer
@ 2017-01-25 0:44 ` Qu Wenruo
2017-01-25 0:46 ` Christoph Anton Mitterer
0 siblings, 1 reply; 29+ messages in thread
From: Qu Wenruo @ 2017-01-25 0:44 UTC (permalink / raw)
To: Christoph Anton Mitterer, linux-btrfs; +Cc: chris, dsterba
At 01/25/2017 12:54 AM, Christoph Anton Mitterer wrote:
> Hey Qu.
>
> I was giving your patches a try, again on the very same fs (which saw
> however writes in the meantime), from my initial report.
>
> btrfs-progs v4.9 WITHOUT patch:
> *******************************
> # btrfs check /dev/nbd0 ; echo $?
> checking extents
> checking free space cache
> checking fs roots
> checking csums
> checking root refs
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used err is 0
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 0
>
> # btrfs check --mode=lowmem /dev/nbd0 ; echo $?
> checking extents
> ERROR: block group[74117545984 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[239473786880 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[500393050112 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[581997428736 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[626557714432 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[668433645568 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[948680261632 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[982503129088 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1039411445760 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1054443831296 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[1190809042944 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1279392743424 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1481256206336 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1620842643456 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[1914511032320 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[3055361720320 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[3216422993920 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[3670615785472 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[3801612288000 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[3828455833600 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[4250973241344 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4261710659584 1073741824] used 1073741824 but extent items used 1074266112
> ERROR: block group[4392707162112 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4558063403008 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4607455526912 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4635372814336 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4640204652544 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4642352136192 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[4681006841856 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[5063795802112 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[5171169984512 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[5216267141120 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[5290355326976 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[5445511020544 1073741824] used 1073741824 but extent items used 1074266112
> ERROR: block group[6084387405824 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[6104788500480 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[6878956355584 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[6997067956224 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[7702516334592 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[8051482427392 1073741824] used 1073741824 but extent items used 1084751872
> ERROR: block group[8116980678656 1073741824] used 1073741824 but extent items used 0
> ERROR: errors found in extent allocation tree or chunk allocation
> checking free space cache
> checking fs roots
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used err is -5
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 1
>
> => so the fs would still show the symptoms
>
>
> Then, with no RW mount to the fs in between, 4.9 with the following of
> your patches:
> 0001-btrfs-progs-lowmem-check-Fix-wrong-block-group-check.patch
> 0003-btrfs-progs-fsck-Output-verbose-error-when-fsck-foun.patch
> 0004-btrfs-progs-lowmem-check-Fix-false-alert-in-checking.patch
> 0005-btrfs-progs-lowmem-check-Fix-extent-item-size-false-.patch
> the others were anyway just tests related, AFAICS.
>
> btrfs-progs v4.9 WITH patches:
> ****************************
> checking extents
> checking free space cache
> checking fs roots
> checking csums
> checking root refs
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used, no error found
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 0
>
> => bon!
>
> # btrfs check --mode=lowmem /dev/nbd0 ; echo $?
> checking extents
> checking free space cache
> checking fs roots
> ERROR: root 6031 EXTENT_DATA[277 524288] datasum missing
There must be some hidden corner case I didn't cover.
Considering lowmem mode check functions are quite independent to each
other, I think I could find out the reason just using the error output.
Thanks for the test,
Qu
> ERROR: errors found in fs roots
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used, error(s) found
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 1
>
> => some new errors :-(
> Not sure though, whether this is an issue in your patch or a new
> problem due to my writes (I did just another round of incremental send
> -p/receive)
>
>
> Any ideas?
>
>
> Best,
> Chris.
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-01-25 0:44 ` Qu Wenruo
@ 2017-01-25 0:46 ` Christoph Anton Mitterer
2017-01-25 4:16 ` Qu Wenruo
0 siblings, 1 reply; 29+ messages in thread
From: Christoph Anton Mitterer @ 2017-01-25 0:46 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs; +Cc: chris, dsterba
[-- Attachment #1: Type: text/plain, Size: 228 bytes --]
On Wed, 2017-01-25 at 08:44 +0800, Qu Wenruo wrote:
> Thanks for the test,
You're welcome... I'm happy if I can help :)
Just tell me once you think you found something, and I'll repeat the
testing.
Cheers,
Chr
is.
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5930 bytes --]
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-01-25 0:46 ` Christoph Anton Mitterer
@ 2017-01-25 4:16 ` Qu Wenruo
2017-01-25 4:40 ` Christoph Anton Mitterer
2017-01-26 2:50 ` Christoph Anton Mitterer
0 siblings, 2 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-25 4:16 UTC (permalink / raw)
To: Christoph Anton Mitterer, linux-btrfs; +Cc: chris, dsterba
At 01/25/2017 08:46 AM, Christoph Anton Mitterer wrote:
> On Wed, 2017-01-25 at 08:44 +0800, Qu Wenruo wrote:
>> Thanks for the test,
>
> You're welcome... I'm happy if I can help :)
>
> Just tell me once you think you found something, and I'll repeat the
> testing.
>
>
> Cheers,
> Chr
> is.
New patches are out now.
Although I just updated
0001-btrfs-progs-lowmem-check-Fix-wrong-block-group-check.patch to fix
all similar bugs.
You could get it from github:
https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes
Unfortunately, I didn't find the cause of the remaining error of that
missing csum.
And considering the size of your fs, btrfs-image is not possible, so I'm
afraid you need to test the patches every time it updates.
Sorry for that,
Qu
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-01-25 4:16 ` Qu Wenruo
@ 2017-01-25 4:40 ` Christoph Anton Mitterer
2017-01-26 2:50 ` Christoph Anton Mitterer
1 sibling, 0 replies; 29+ messages in thread
From: Christoph Anton Mitterer @ 2017-01-25 4:40 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs; +Cc: chris, dsterba
[-- Attachment #1: Type: text/plain, Size: 661 bytes --]
On Wed, 2017-01-25 at 12:16 +0800, Qu Wenruo wrote:
> New patches are out now.
>
> Although I just updated
> 0001-btrfs-progs-lowmem-check-Fix-wrong-block-group-check.patch to
> fix
> all similar bugs.
>
> You could get it from github:
> https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes
Sure, will take a while, though (hopefully get it done tomorrow)
> Unfortunately, I didn't find the cause of the remaining error of
> that
> missing csum.
> And considering the size of your fs, btrfs-image is not possible, so
> I'm
> afraid you need to test the patches every time it updates.
No worries :-)
Cheers,
Chris.
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5930 bytes --]
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-01-25 4:16 ` Qu Wenruo
2017-01-25 4:40 ` Christoph Anton Mitterer
@ 2017-01-26 2:50 ` Christoph Anton Mitterer
2017-01-26 3:10 ` Qu Wenruo
1 sibling, 1 reply; 29+ messages in thread
From: Christoph Anton Mitterer @ 2017-01-26 2:50 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs; +Cc: chris, dsterba
[-- Attachment #1: Type: text/plain, Size: 6261 bytes --]
On Wed, 2017-01-25 at 12:16 +0800, Qu Wenruo wrote:
> https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes
Just finished trying your new patches.
Same game as last time, applied to 4.9, no RW mount between the runs.
btrfs-progs v4.9 WITHOUT patch:
*******************************
# btrfs check /dev/nbd0 ; echo $?
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used err is 0
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
0
# btrfs check --mode=lowmem /dev/nbd0 ; echo $?
checking extents
ERROR: block group[74117545984 1073741824] used 1073741824 but extent items used 0
ERROR: block group[239473786880 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[500393050112 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[581997428736 1073741824] used 1073741824 but extent items used 0
ERROR: block group[626557714432 1073741824] used 1073741824 but extent items used 0
ERROR: block group[668433645568 1073741824] used 1073741824 but extent items used 0
ERROR: block group[948680261632 1073741824] used 1073741824 but extent items used 0
ERROR: block group[982503129088 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1039411445760 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1054443831296 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[1190809042944 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1279392743424 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1481256206336 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1620842643456 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[1914511032320 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[3055361720320 1073741824] used 1073741824 but extent items used 0
ERROR: block group[3216422993920 1073741824] used 1073741824 but extent items used 0
ERROR: block group[3670615785472 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[3801612288000 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[3828455833600 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[4250973241344 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4261710659584 1073741824] used 1073741824 but extent items used 1074266112
ERROR: block group[4392707162112 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4558063403008 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4607455526912 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4635372814336 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4640204652544 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4642352136192 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[4681006841856 1073741824] used 1073741824 but extent items used 0
ERROR: block group[5063795802112 1073741824] used 1073741824 but extent items used 0
ERROR: block group[5171169984512 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[5216267141120 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[5290355326976 1073741824] used 1073741824 but extent items used 0
ERROR: block group[5445511020544 1073741824] used 1073741824 but extent items used 1074266112
ERROR: block group[6084387405824 1073741824] used 1073741824 but extent items used 0
ERROR: block group[6104788500480 1073741824] used 1073741824 but extent items used 0
ERROR: block group[6878956355584 1073741824] used 1073741824 but extent items used 0
ERROR: block group[6997067956224 1073741824] used 1073741824 but extent items used 0
ERROR: block group[7702516334592 1073741824] used 1073741824 but extent items used 0
ERROR: block group[8051482427392 1073741824] used 1073741824 but extent items used 1084751872
ERROR: block group[8116980678656 1073741824] used 1073741824 but extent items used 0
ERROR: errors found in extent allocation tree or chunk allocation
checking free space cache
checking fs roots
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used err is -5
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
1
btrfs-progs v4.9 WITH patches:
******************************
# btrfs check /dev/nbd0 ; echo $?
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used, no error found
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
0
# btrfs check --mode=lowmem /dev/nbd0 ; echo $?
checking extents
checking free space cache
checking fs roots
ERROR: root 6031 EXTENT_DATA[277 524288] datasum missing
ERROR: errors found in fs roots
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used, error(s) found
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
1
=> so unfortunately what seems to be the very same error than before
Just tell me if you find something new :)
Cheers,
Chris.
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5930 bytes --]
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-01-26 2:50 ` Christoph Anton Mitterer
@ 2017-01-26 3:10 ` Qu Wenruo
2017-01-26 3:30 ` Christoph Anton Mitterer
2017-01-26 23:31 ` Christoph Anton Mitterer
0 siblings, 2 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-01-26 3:10 UTC (permalink / raw)
To: Christoph Anton Mitterer, linux-btrfs; +Cc: Qu Wenruo
Thanks for your patient and the test!
At 01/26/2017 10:50 AM, Christoph Anton Mitterer wrote:
> On Wed, 2017-01-25 at 12:16 +0800, Qu Wenruo wrote:
>> https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes
>
> Just finished trying your new patches.
>
> Same game as last time, applied to 4.9, no RW mount between the runs.
>
>
> btrfs-progs v4.9 WITHOUT patch:
> *******************************
> # btrfs check /dev/nbd0 ; echo $?
> checking extents
> checking free space cache
> checking fs roots
> checking csums
> checking root refs
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used err is 0
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 0
>
>
> # btrfs check --mode=lowmem /dev/nbd0 ; echo $?
> checking extents
> ERROR: block group[74117545984 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[239473786880 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[500393050112 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[581997428736 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[626557714432 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[668433645568 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[948680261632 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[982503129088 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1039411445760 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1054443831296 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[1190809042944 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1279392743424 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1481256206336 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1620842643456 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[1914511032320 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[3055361720320 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[3216422993920 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[3670615785472 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[3801612288000 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[3828455833600 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[4250973241344 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4261710659584 1073741824] used 1073741824 but extent items used 1074266112
> ERROR: block group[4392707162112 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4558063403008 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4607455526912 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4635372814336 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4640204652544 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4642352136192 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[4681006841856 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[5063795802112 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[5171169984512 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[5216267141120 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[5290355326976 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[5445511020544 1073741824] used 1073741824 but extent items used 1074266112
> ERROR: block group[6084387405824 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[6104788500480 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[6878956355584 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[6997067956224 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[7702516334592 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[8051482427392 1073741824] used 1073741824 but extent items used 1084751872
> ERROR: block group[8116980678656 1073741824] used 1073741824 but extent items used 0
> ERROR: errors found in extent allocation tree or chunk allocation
> checking free space cache
> checking fs roots
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used err is -5
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 1
In fact, the result without patches is not really needed for current stage.
Feel free to skip them until the patched ones passed.
Which should save you some time.
>
> btrfs-progs v4.9 WITH patches:
> ******************************
> # btrfs check /dev/nbd0 ; echo $?
> checking extents
> checking free space cache
> checking fs roots
> checking csums
> checking root refs
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used, no error found
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 0
Same for original mode.
>
> # btrfs check --mode=lowmem /dev/nbd0 ; echo $?
> checking extents
> checking free space cache
> checking fs roots
> ERROR: root 6031 EXTENT_DATA[277 524288] datasum missing
> ERROR: errors found in fs roots
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used, error(s) found
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 1
>
> => so unfortunately what seems to be the very same error than before
>
> Just tell me if you find something new :)
>
>
> Cheers,
> Chris.
>
Would you please try lowmem_tests branch of my repo?
That branch contains a special debug output for the case you
encountered, which should help to debug the case.
pecial debug output for the case you encountered, which
Thanks,
Qu
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-01-26 3:10 ` Qu Wenruo
@ 2017-01-26 3:30 ` Christoph Anton Mitterer
2017-01-26 23:31 ` Christoph Anton Mitterer
1 sibling, 0 replies; 29+ messages in thread
From: Christoph Anton Mitterer @ 2017-01-26 3:30 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 924 bytes --]
On Thu, 2017-01-26 at 11:10 +0800, Qu Wenruo wrote:
> In fact, the result without patches is not really needed for current
> stage.
>
> Feel free to skip them until the patched ones passed.
> Which should save you some time.
Well the idea is, that if I do further writes in the meantime (by
adding new backup data), then things in the fs could change (I blindly
assume) in such a way, that the false positive isn't triggered any more
- not because a patch would finally have it fixed, but simply because
things on the fs changed...
That's why I repeated it always so far - just to see that the issues
would be still there.
> Would you please try lowmem_tests branch of my repo?
>
> That branch contains a special debug output for the case you
> encountered, which should help to debug the case.
> pecial debug output for the case you encountered, which
Sure, tomorrow.
Best wishes,
Chris.
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5930 bytes --]
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-01-26 3:10 ` Qu Wenruo
2017-01-26 3:30 ` Christoph Anton Mitterer
@ 2017-01-26 23:31 ` Christoph Anton Mitterer
2017-01-29 4:27 ` Qu Wenruo
1 sibling, 1 reply; 29+ messages in thread
From: Christoph Anton Mitterer @ 2017-01-26 23:31 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 50025 bytes --]
On Thu, 2017-01-26 at 11:10 +0800, Qu Wenruo wrote:
> Would you please try lowmem_tests branch of my repo?
>
> That branch contains a special debug output for the case you
> encountered, which should help to debug the case.
> pecial debug output for the case you encountered, which
Here the output with your patches (again, not having applied the
unnecessary fs-tests patches):
In the output below I've replaced filenames with "[snip..snap]" and
exchanged some of the xattr values.
In case you should need their original values for testing, just tell me
and I send them to you off-list.
btrfs-progs v4.9 WITH patches:
******************************
# btrfs check --mode=lowmem /dev/nbd0 ; echo $?
checking extents
checking free space cache
checking fs roots
ERROR: root 6031 EXTENT_DATA[277 524288] datasum missing, have: 36864, expect: 45056 ret: 0
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
=== fs tree leaf dump: slot: 136 ===
leaf 5960902508544 items 191 free space 60 generation 2775 owner 6031
fs uuid 326d292d-f97b-43ca-b1e8-c722d3474719
chunk uuid 5da7e818-7f0b-43c1-b465-fdfaa52da633
item 0 key (274 EXTENT_DATA 7733248) itemoff 16230 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807724716032 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 1 key (274 EXTENT_DATA 7864320) itemoff 16177 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807724834816 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 2 key (274 EXTENT_DATA 7995392) itemoff 16124 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807724953600 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 3 key (274 EXTENT_DATA 8126464) itemoff 16071 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807725076480 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 4 key (274 EXTENT_DATA 8257536) itemoff 16018 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807725199360 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 5 key (274 EXTENT_DATA 8388608) itemoff 15965 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807725318144 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 6 key (274 EXTENT_DATA 8519680) itemoff 15912 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807725432832 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 7 key (274 EXTENT_DATA 8650752) itemoff 15859 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807725551616 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 8 key (274 EXTENT_DATA 8781824) itemoff 15806 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807725670400 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 9 key (274 EXTENT_DATA 8912896) itemoff 15753 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807725789184 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 10 key (274 EXTENT_DATA 9043968) itemoff 15700 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807725912064 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 11 key (274 EXTENT_DATA 9175040) itemoff 15647 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807726026752 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 12 key (274 EXTENT_DATA 9306112) itemoff 15594 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807726141440 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 13 key (274 EXTENT_DATA 9437184) itemoff 15541 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807726260224 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 14 key (274 EXTENT_DATA 9568256) itemoff 15488 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807726379008 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 15 key (274 EXTENT_DATA 9699328) itemoff 15435 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807726501888 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 16 key (274 EXTENT_DATA 9830400) itemoff 15382 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807726620672 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 17 key (274 EXTENT_DATA 9961472) itemoff 15329 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807726739456 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 18 key (274 EXTENT_DATA 10092544) itemoff 15276 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807726858240 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 19 key (274 EXTENT_DATA 10223616) itemoff 15223 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807726977024 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 20 key (274 EXTENT_DATA 10354688) itemoff 15170 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807727099904 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 21 key (274 EXTENT_DATA 10485760) itemoff 15117 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807727218688 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 22 key (274 EXTENT_DATA 10616832) itemoff 15064 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807727341568 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 23 key (274 EXTENT_DATA 10747904) itemoff 15011 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807727460352 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 24 key (274 EXTENT_DATA 10878976) itemoff 14958 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807727575040 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 25 key (274 EXTENT_DATA 11010048) itemoff 14905 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807727693824 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 26 key (274 EXTENT_DATA 11141120) itemoff 14852 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807727812608 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 27 key (274 EXTENT_DATA 11272192) itemoff 14799 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807727931392 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 28 key (274 EXTENT_DATA 11403264) itemoff 14746 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807728054272 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 29 key (274 EXTENT_DATA 11534336) itemoff 14693 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807728177152 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 30 key (274 EXTENT_DATA 11665408) itemoff 14640 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807728291840 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 31 key (274 EXTENT_DATA 11796480) itemoff 14587 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807728406528 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 32 key (274 EXTENT_DATA 11927552) itemoff 14534 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807728525312 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 33 key (274 EXTENT_DATA 12058624) itemoff 14481 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807728648192 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 34 key (274 EXTENT_DATA 12189696) itemoff 14428 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807728771072 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 35 key (274 EXTENT_DATA 12320768) itemoff 14375 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807728893952 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 36 key (274 EXTENT_DATA 12451840) itemoff 14322 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807729016832 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 37 key (274 EXTENT_DATA 12582912) itemoff 14269 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807729131520 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 38 key (274 EXTENT_DATA 12713984) itemoff 14216 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807729246208 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 39 key (274 EXTENT_DATA 12845056) itemoff 14163 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807729364992 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 40 key (274 EXTENT_DATA 12976128) itemoff 14110 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807729483776 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 41 key (274 EXTENT_DATA 13107200) itemoff 14057 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807729606656 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 42 key (274 EXTENT_DATA 13238272) itemoff 14004 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807729725440 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 43 key (274 EXTENT_DATA 13369344) itemoff 13951 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807729840128 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 44 key (274 EXTENT_DATA 13500416) itemoff 13898 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807729954816 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 45 key (274 EXTENT_DATA 13631488) itemoff 13845 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807730069504 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 46 key (274 EXTENT_DATA 13762560) itemoff 13792 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807730188288 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 47 key (274 EXTENT_DATA 13893632) itemoff 13739 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807730311168 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 48 key (274 EXTENT_DATA 14024704) itemoff 13686 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807730429952 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 49 key (274 EXTENT_DATA 14155776) itemoff 13633 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807730548736 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 50 key (274 EXTENT_DATA 14286848) itemoff 13580 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807730663424 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 51 key (274 EXTENT_DATA 14417920) itemoff 13527 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807730782208 nr 45056
extent data offset 0 nr 53248 ram 53248
extent compression 1 (zlib)
item 52 key (275 INODE_ITEM 0) itemoff 13367 itemsize 160
inode generation 2775 transid 2775 size 136 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0xa(none)
atime 1471659949.935831165 (2016-08-20 04:25:49)
ctime 1473986193.679587164 (2016-09-16 02:36:33)
mtime 1471659958.979930414 (2016-08-20 04:25:58)
otime 1473986193.595586129 (2016-09-16 02:36:33)
item 53 key (275 INODE_REF 268) itemoff 13340 itemsize 27
inode ref index 4 namelen 17 name: [snip..snap]
item 54 key (275 DIR_ITEM 332901471) itemoff 13261 itemsize 79
location key (276 INODE_ITEM 0) type FILE
transid 2775 data_len 0 name_len 49
name: [snip..snap]
item 55 key (275 DIR_ITEM 2370206775) itemoff 13212 itemsize 49
location key (277 INODE_ITEM 0) type FILE
transid 2775 data_len 0 name_len 19
name: [snip..snap]
item 56 key (275 DIR_INDEX 2) itemoff 13133 itemsize 79
location key (276 INODE_ITEM 0) type FILE
transid 2775 data_len 0 name_len 49
name: [snip..snap]
item 57 key (275 DIR_INDEX 3) itemoff 13084 itemsize 49
location key (277 INODE_ITEM 0) type FILE
transid 2775 data_len 0 name_len 19
name: [snip..snap]
item 58 key (276 INODE_ITEM 0) itemoff 12924 itemsize 160
inode generation 2775 transid 2775 size 9185394 nbytes 9187328
block group 0 mode 100644 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0x23(none)
atime 1471659943.443759931 (2016-08-20 04:25:43)
ctime 1473986193.663586968 (2016-09-16 02:36:33)
mtime 1471659958.975930370 (2016-08-20 04:25:58)
otime 1473986193.595586129 (2016-09-16 02:36:33)
item 59 key (276 INODE_REF 275) itemoff 12865 itemsize 59
inode ref index 2 namelen 49 name: [snip..snap]
item 60 key (276 XATTR_ITEM 1330394165) itemoff 12683 itemsize 182
location key (0 UNKNOWN.0 0) type XATTR
transid 2775 data_len 128 name_len 24
name: user.scinet.hash.SHA-512
data 54a9b51b6914470bb549457c63250c59bc60ed4aa954bde3fc199313cfbbbda2d318e896652bb8d2971be18f0d2afaba1c48e08efa7472208f950aba2c3287ca
item 61 key (276 EXTENT_DATA 0) itemoff 12630 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807730827264 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 62 key (276 EXTENT_DATA 131072) itemoff 12577 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807730954240 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 63 key (276 EXTENT_DATA 262144) itemoff 12524 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807731081216 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 64 key (276 EXTENT_DATA 393216) itemoff 12471 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807731204096 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 65 key (276 EXTENT_DATA 524288) itemoff 12418 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807731326976 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 66 key (276 EXTENT_DATA 655360) itemoff 12365 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807731445760 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 67 key (276 EXTENT_DATA 786432) itemoff 12312 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807731568640 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 68 key (276 EXTENT_DATA 917504) itemoff 12259 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807731695616 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 69 key (276 EXTENT_DATA 1048576) itemoff 12206 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807731822592 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 70 key (276 EXTENT_DATA 1179648) itemoff 12153 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807731949568 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 71 key (276 EXTENT_DATA 1310720) itemoff 12100 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807732072448 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 72 key (276 EXTENT_DATA 1441792) itemoff 12047 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807732195328 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 73 key (276 EXTENT_DATA 1572864) itemoff 11994 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807732314112 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 74 key (276 EXTENT_DATA 1703936) itemoff 11941 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807732436992 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 75 key (276 EXTENT_DATA 1835008) itemoff 11888 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807732559872 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 76 key (276 EXTENT_DATA 1966080) itemoff 11835 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807732682752 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 77 key (276 EXTENT_DATA 2097152) itemoff 11782 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807732805632 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 78 key (276 EXTENT_DATA 2228224) itemoff 11729 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807732924416 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 79 key (276 EXTENT_DATA 2359296) itemoff 11676 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807733047296 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 80 key (276 EXTENT_DATA 2490368) itemoff 11623 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807733170176 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 81 key (276 EXTENT_DATA 2621440) itemoff 11570 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807733293056 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 82 key (276 EXTENT_DATA 2752512) itemoff 11517 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807733411840 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 83 key (276 EXTENT_DATA 2883584) itemoff 11464 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807733538816 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 84 key (276 EXTENT_DATA 3014656) itemoff 11411 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807733665792 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 85 key (276 EXTENT_DATA 3145728) itemoff 11358 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807733792768 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 86 key (276 EXTENT_DATA 3276800) itemoff 11305 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807733915648 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 87 key (276 EXTENT_DATA 3407872) itemoff 11252 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807734030336 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 88 key (276 EXTENT_DATA 3538944) itemoff 11199 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807734153216 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 89 key (276 EXTENT_DATA 3670016) itemoff 11146 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807734272000 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 90 key (276 EXTENT_DATA 3801088) itemoff 11093 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807734390784 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 91 key (276 EXTENT_DATA 3932160) itemoff 11040 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807734509568 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 92 key (276 EXTENT_DATA 4063232) itemoff 10987 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807734624256 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 93 key (276 EXTENT_DATA 4194304) itemoff 10934 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807734747136 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 94 key (276 EXTENT_DATA 4325376) itemoff 10881 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807734865920 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 95 key (276 EXTENT_DATA 4456448) itemoff 10828 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807734988800 nr 114688
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 96 key (276 EXTENT_DATA 4587520) itemoff 10775 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807735103488 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 97 key (276 EXTENT_DATA 4718592) itemoff 10722 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807735222272 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 98 key (276 EXTENT_DATA 4849664) itemoff 10669 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807735341056 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 99 key (276 EXTENT_DATA 4980736) itemoff 10616 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807735459840 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 100 key (276 EXTENT_DATA 5111808) itemoff 10563 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807735582720 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 101 key (276 EXTENT_DATA 5242880) itemoff 10510 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807735705600 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 102 key (276 EXTENT_DATA 5373952) itemoff 10457 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807735828480 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 103 key (276 EXTENT_DATA 5505024) itemoff 10404 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807735951360 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 104 key (276 EXTENT_DATA 5636096) itemoff 10351 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807736074240 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 105 key (276 EXTENT_DATA 5767168) itemoff 10298 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807736193024 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 106 key (276 EXTENT_DATA 5898240) itemoff 10245 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807736315904 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 107 key (276 EXTENT_DATA 6029312) itemoff 10192 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807736438784 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 108 key (276 EXTENT_DATA 6160384) itemoff 10139 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807736557568 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 109 key (276 EXTENT_DATA 6291456) itemoff 10086 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807736680448 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 110 key (276 EXTENT_DATA 6422528) itemoff 10033 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807736807424 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 111 key (276 EXTENT_DATA 6553600) itemoff 9980 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807736930304 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 112 key (276 EXTENT_DATA 6684672) itemoff 9927 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807737053184 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 113 key (276 EXTENT_DATA 6815744) itemoff 9874 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807737176064 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 114 key (276 EXTENT_DATA 6946816) itemoff 9821 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807737298944 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 115 key (276 EXTENT_DATA 7077888) itemoff 9768 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807737417728 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 116 key (276 EXTENT_DATA 7208960) itemoff 9715 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807737536512 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 117 key (276 EXTENT_DATA 7340032) itemoff 9662 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807737659392 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 118 key (276 EXTENT_DATA 7471104) itemoff 9609 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807737778176 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 119 key (276 EXTENT_DATA 7602176) itemoff 9556 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807737896960 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 120 key (276 EXTENT_DATA 7733248) itemoff 9503 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807738015744 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 121 key (276 EXTENT_DATA 7864320) itemoff 9450 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807738134528 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 122 key (276 EXTENT_DATA 7995392) itemoff 9397 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807738257408 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 123 key (276 EXTENT_DATA 8126464) itemoff 9344 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807738376192 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 124 key (276 EXTENT_DATA 8257536) itemoff 9291 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807738499072 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 125 key (276 EXTENT_DATA 8388608) itemoff 9238 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807738617856 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 126 key (276 EXTENT_DATA 8519680) itemoff 9185 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807738740736 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 127 key (276 EXTENT_DATA 8650752) itemoff 9132 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807738863616 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 128 key (276 EXTENT_DATA 8781824) itemoff 9079 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807738982400 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 129 key (276 EXTENT_DATA 8912896) itemoff 9026 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807739101184 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 130 key (276 EXTENT_DATA 9043968) itemoff 8973 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807739219968 nr 118784
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 131 key (276 EXTENT_DATA 9175040) itemoff 8920 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807739338752 nr 8192
extent data offset 0 nr 12288 ram 12288
extent compression 1 (zlib)
item 132 key (277 INODE_ITEM 0) itemoff 8760 itemsize 160
inode generation 2775 transid 2775 size 565664 nbytes 569344
block group 0 mode 100644 links 1 uid 1000 gid 1000 rdev 0
sequence 8 flags 0xb(NOCOMPRESS)
atime 1471659953.967875410 (2016-08-20 04:25:53)
ctime 1473986193.679587164 (2016-09-16 02:36:33)
mtime 1471659957.51909254 (2016-08-20 04:25:57)
otime 1473986193.663586968 (2016-09-16 02:36:33)
item 133 key (277 INODE_REF 275) itemoff 8731 itemsize 29
inode ref index 3 namelen 19 name: [snip..snap]
item 134 key (277 XATTR_ITEM 1330394165) itemoff 8549 itemsize 182
location key (0 UNKNOWN.0 0) type XATTR
transid 2775 data_len 128 name_len 24
name: user.scinet.hash.SHA-512
data a6e286c88c1b290294e5367513e670c3d50456d81ba59486d00fa15509042e464116c0f061711d9fdd0c94ed642106868660b7ae984713b498d28a87732822b7
item 135 key (277 EXTENT_DATA 0) itemoff 8496 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807739346944 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 136 key (277 EXTENT_DATA 524288) itemoff 8443 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6805694144512 nr 36864
extent data offset 0 nr 45056 ram 45056
extent compression 1 (zlib)
item 137 key (278 INODE_ITEM 0) itemoff 8283 itemsize 160
inode generation 2775 transid 2775 size 140 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0x9(none)
atime 1471659969.700048076 (2016-08-20 04:26:09)
ctime 1473986193.803588694 (2016-09-16 02:36:33)
mtime 1471659997.828356911 (2016-08-20 04:26:37)
otime 1473986193.679587164 (2016-09-16 02:36:33)
item 138 key (278 INODE_REF 268) itemoff 8247 itemsize 36
inode ref index 5 namelen 26 name: [snip..snap]
item 139 key (278 DIR_ITEM 3019993287) itemoff 8147 itemsize 100
location key (279 INODE_ITEM 0) type FILE
transid 2775 data_len 0 name_len 70
name: [snip..snap]
item 140 key (278 DIR_INDEX 2) itemoff 8047 itemsize 100
location key (279 INODE_ITEM 0) type FILE
transid 2775 data_len 0 name_len 70
name: [snip..snap]
item 141 key (279 INODE_ITEM 0) itemoff 7887 itemsize 160
inode generation 2775 transid 2775 size 18033093 nbytes 18034688
block group 0 mode 100644 links 1 uid 1000 gid 1000 rdev 0
sequence 8 flags 0xb(NOCOMPRESS)
atime 1471659959.995941565 (2016-08-20 04:25:59)
ctime 1473986193.803588694 (2016-09-16 02:36:33)
mtime 1471659997.824356866 (2016-08-20 04:26:37)
otime 1473986193.679587164 (2016-09-16 02:36:33)
item 142 key (279 INODE_REF 278) itemoff 7807 itemsize 80
inode ref index 2 namelen 70 name: [snip..snap]
item 143 key (279 XATTR_ITEM 1330394165) itemoff 7625 itemsize 182
location key (0 UNKNOWN.0 0) type XATTR
data 3da14e71e11761b072cf710044fa3403276d7d540618407a104209c10345b0b9295d24c6624e902b174cde2a63aaa07b0fa36ee1af39c05517cf85300bc70b92
transid 2775 data_len 128 name_len 24
name: user.scinet.hash.SHA-512
item 144 key (279 EXTENT_DATA 0) itemoff 7572 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807739871232 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 145 key (279 EXTENT_DATA 131072) itemoff 7519 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807739998208 nr 393216
extent data offset 0 nr 393216 ram 393216
extent compression 0 (none)
item 146 key (279 EXTENT_DATA 524288) itemoff 7466 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807740391424 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 147 key (279 EXTENT_DATA 1048576) itemoff 7413 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807740915712 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 148 key (279 EXTENT_DATA 1179648) itemoff 7360 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807741042688 nr 393216
extent data offset 0 nr 393216 ram 393216
extent compression 0 (none)
item 149 key (279 EXTENT_DATA 1572864) itemoff 7307 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807741435904 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 150 key (279 EXTENT_DATA 1703936) itemoff 7254 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807741562880 nr 393216
extent data offset 0 nr 393216 ram 393216
extent compression 0 (none)
item 151 key (279 EXTENT_DATA 2097152) itemoff 7201 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807741956096 nr 122880
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 152 key (279 EXTENT_DATA 2228224) itemoff 7148 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807742078976 nr 393216
extent data offset 0 nr 393216 ram 393216
extent compression 0 (none)
item 153 key (279 EXTENT_DATA 2621440) itemoff 7095 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807742472192 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 154 key (279 EXTENT_DATA 3145728) itemoff 7042 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807742996480 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 155 key (279 EXTENT_DATA 3670016) itemoff 6989 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807743520768 nr 126976
extent data offset 0 nr 131072 ram 131072
extent compression 1 (zlib)
item 156 key (279 EXTENT_DATA 3801088) itemoff 6936 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807743647744 nr 393216
extent data offset 0 nr 393216 ram 393216
extent compression 0 (none)
item 157 key (279 EXTENT_DATA 4194304) itemoff 6883 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807744040960 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 158 key (279 EXTENT_DATA 4718592) itemoff 6830 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807744565248 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 159 key (279 EXTENT_DATA 5242880) itemoff 6777 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807745089536 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 160 key (279 EXTENT_DATA 5767168) itemoff 6724 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807745613824 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 161 key (279 EXTENT_DATA 6291456) itemoff 6671 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807746138112 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 162 key (279 EXTENT_DATA 6815744) itemoff 6618 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807746662400 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 163 key (279 EXTENT_DATA 7340032) itemoff 6565 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807747186688 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 164 key (279 EXTENT_DATA 7864320) itemoff 6512 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807747710976 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 165 key (279 EXTENT_DATA 8388608) itemoff 6459 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807748235264 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 166 key (279 EXTENT_DATA 8912896) itemoff 6406 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807748759552 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 167 key (279 EXTENT_DATA 9437184) itemoff 6353 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807749283840 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 168 key (279 EXTENT_DATA 9961472) itemoff 6300 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807749808128 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 169 key (279 EXTENT_DATA 10485760) itemoff 6247 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807750332416 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 170 key (279 EXTENT_DATA 11010048) itemoff 6194 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807750856704 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 171 key (279 EXTENT_DATA 11534336) itemoff 6141 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807751380992 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 172 key (279 EXTENT_DATA 12058624) itemoff 6088 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807751905280 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 173 key (279 EXTENT_DATA 12582912) itemoff 6035 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807752429568 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 174 key (279 EXTENT_DATA 13107200) itemoff 5982 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807752953856 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 175 key (279 EXTENT_DATA 13631488) itemoff 5929 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807753478144 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 176 key (279 EXTENT_DATA 14155776) itemoff 5876 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807754002432 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 177 key (279 EXTENT_DATA 14680064) itemoff 5823 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807754526720 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 178 key (279 EXTENT_DATA 15204352) itemoff 5770 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807755051008 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 179 key (279 EXTENT_DATA 15728640) itemoff 5717 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807755575296 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 180 key (279 EXTENT_DATA 16252928) itemoff 5664 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807756099584 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 181 key (279 EXTENT_DATA 16777216) itemoff 5611 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807756623872 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 182 key (279 EXTENT_DATA 17301504) itemoff 5558 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807757148160 nr 524288
extent data offset 0 nr 524288 ram 524288
extent compression 0 (none)
item 183 key (279 EXTENT_DATA 17825792) itemoff 5505 itemsize 53
generation 2775 type 1 (regular)
extent data disk byte 6807757672448 nr 208896
extent data offset 0 nr 208896 ram 208896
extent compression 0 (none)
item 184 key (280 INODE_ITEM 0) itemoff 5345 itemsize 160
inode generation 2775 transid 2775 size 202 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0x1a(none)
atime 1471659982.48183634 (2016-08-20 04:26:22)
ctime 1473986194.95592295 (2016-09-16 02:36:34)
mtime 1471660037.872796836 (2016-08-20 04:27:17)
otime 1473986193.803588694 (2016-09-16 02:36:33)
item 185 key (280 INODE_REF 268) itemoff 5317 itemsize 28
inode ref index 6 namelen 18 name: [snip..snap]
item 186 key (280 DIR_ITEM 38578281) itemoff 5232 itemsize 85
location key (281 INODE_ITEM 0) type FILE
transid 2775 data_len 0 name_len 55
name: [snip..snap]
item 187 key (280 DIR_ITEM 2341083723) itemoff 5156 itemsize 76
location key (282 INODE_ITEM 0) type FILE
transid 2775 data_len 0 name_len 46
name: [snip..snap]
item 188 key (280 DIR_INDEX 2) itemoff 5071 itemsize 85
location key (281 INODE_ITEM 0) type FILE
transid 2775 data_len 0 name_len 55
name: J[snip..snap]
item 189 key (280 DIR_INDEX 3) itemoff 4995 itemsize 76
location key (282 INODE_ITEM 0) type FILE
transid 2775 data_len 0 name_len 46
name: [snip..snap]
item 190 key (281 INODE_ITEM 0) itemoff 4835 itemsize 160
inode generation 2775 transid 2775 size 28025786 nbytes 28028928
block group 0 mode 100644 links 1 uid 1000 gid 1000 rdev 0
sequence 8 flags 0x1f(NOCOMPRESS)
atime 1471659973.56084916 (2016-08-20 04:26:13)
ctime 1473986194.23591408 (2016-09-16 02:36:34)
mtime 1471660037.868796793 (2016-08-20 04:27:17)
otime 1473986193.803588694 (2016-09-16 02:36:33)
=== csum tree dump ===
leaf 4983028940800 items 22 free space 6853 generation 3093 owner 7
fs uuid 326d292d-f97b-43ca-b1e8-c722d3474719
chunk uuid 5da7e818-7f0b-43c1-b465-fdfaa52da633
item 0 key (EXTENT_CSUM EXTENT_CSUM 6805691457536) itemoff 16279 itemsize 4
extent csum item
item 1 key (EXTENT_CSUM EXTENT_CSUM 6805691461632) itemoff 16275 itemsize 4
extent csum item
item 2 key (EXTENT_CSUM EXTENT_CSUM 6805691465728) itemoff 16263 itemsize 12
extent csum item
item 3 key (EXTENT_CSUM EXTENT_CSUM 6805691478016) itemoff 16231 itemsize 32
extent csum item
item 4 key (EXTENT_CSUM EXTENT_CSUM 6805691510784) itemoff 16223 itemsize 8
extent csum item
item 5 key (EXTENT_CSUM EXTENT_CSUM 6805691518976) itemoff 16199 itemsize 24
extent csum item
item 6 key (EXTENT_CSUM EXTENT_CSUM 6805691543552) itemoff 16183 itemsize 16
extent csum item
item 7 key (EXTENT_CSUM EXTENT_CSUM 6805691559936) itemoff 15159 itemsize 1024
extent csum item
item 8 key (EXTENT_CSUM EXTENT_CSUM 6805692608512) itemoff 15151 itemsize 8
extent csum item
item 9 key (EXTENT_CSUM EXTENT_CSUM 6805692616704) itemoff 15055 itemsize 96
extent csum item
item 10 key (EXTENT_CSUM EXTENT_CSUM 6805692715008) itemoff 14903 itemsize 152
extent csum item
item 11 key (EXTENT_CSUM EXTENT_CSUM 6805693132800) itemoff 14519 itemsize 384
extent csum item
item 12 key (EXTENT_CSUM EXTENT_CSUM 6805693526016) itemoff 14391 itemsize 128
extent csum item
item 13 key (EXTENT_CSUM EXTENT_CSUM 6805693919232) itemoff 14135 itemsize 256
extent csum item
item 14 key (EXTENT_CSUM EXTENT_CSUM 6805694443520) itemoff 12727 itemsize 1408
extent csum item
item 15 key (EXTENT_CSUM EXTENT_CSUM 6805695885312) itemoff 11811 itemsize 916
extent csum item
item 16 key (EXTENT_CSUM EXTENT_CSUM 6805696823296) itemoff 11007 itemsize 804
extent csum item
item 17 key (EXTENT_CSUM EXTENT_CSUM 6805697646592) itemoff 10211 itemsize 796
extent csum item
item 18 key (EXTENT_CSUM EXTENT_CSUM 6805698461696) itemoff 9799 itemsize 412
extent csum item
item 19 key (EXTENT_CSUM EXTENT_CSUM 6805698883584) itemoff 9259 itemsize 540
extent csum item
item 20 key (EXTENT_CSUM EXTENT_CSUM 6805699436544) itemoff 8827 itemsize 432
extent csum item
item 21 key (EXTENT_CSUM EXTENT_CSUM 6805699878912) itemoff 7403 itemsize 1424
extent csum item
=== csum tree dump ===
leaf 4983028940800 items 22 free space 6853 generation 3093 owner 7
fs uuid 326d292d-f97b-43ca-b1e8-c722d3474719
chunk uuid 5da7e818-7f0b-43c1-b465-fdfaa52da633
item 0 key (EXTENT_CSUM EXTENT_CSUM 6805691457536) itemoff 16279 itemsize 4
extent csum item
item 1 key (EXTENT_CSUM EXTENT_CSUM 6805691461632) itemoff 16275 itemsize 4
extent csum item
item 2 key (EXTENT_CSUM EXTENT_CSUM 6805691465728) itemoff 16263 itemsize 12
extent csum item
item 3 key (EXTENT_CSUM EXTENT_CSUM 6805691478016) itemoff 16231 itemsize 32
extent csum item
item 4 key (EXTENT_CSUM EXTENT_CSUM 6805691510784) itemoff 16223 itemsize 8
extent csum item
item 5 key (EXTENT_CSUM EXTENT_CSUM 6805691518976) itemoff 16199 itemsize 24
extent csum item
item 6 key (EXTENT_CSUM EXTENT_CSUM 6805691543552) itemoff 16183 itemsize 16
extent csum item
item 7 key (EXTENT_CSUM EXTENT_CSUM 6805691559936) itemoff 15159 itemsize 1024
extent csum item
item 8 key (EXTENT_CSUM EXTENT_CSUM 6805692608512) itemoff 15151 itemsize 8
extent csum item
item 9 key (EXTENT_CSUM EXTENT_CSUM 6805692616704) itemoff 15055 itemsize 96
extent csum item
item 10 key (EXTENT_CSUM EXTENT_CSUM 6805692715008) itemoff 14903 itemsize 152
extent csum item
item 11 key (EXTENT_CSUM EXTENT_CSUM 6805693132800) itemoff 14519 itemsize 384
extent csum item
item 12 key (EXTENT_CSUM EXTENT_CSUM 6805693526016) itemoff 14391 itemsize 128
extent csum item
item 13 key (EXTENT_CSUM EXTENT_CSUM 6805693919232) itemoff 14135 itemsize 256
extent csum item
item 14 key (EXTENT_CSUM EXTENT_CSUM 6805694443520) itemoff 12727 itemsize 1408
extent csum item
item 15 key (EXTENT_CSUM EXTENT_CSUM 6805695885312) itemoff 11811 itemsize 916
extent csum item
item 16 key (EXTENT_CSUM EXTENT_CSUM 6805696823296) itemoff 11007 itemsize 804
extent csum item
item 17 key (EXTENT_CSUM EXTENT_CSUM 6805697646592) itemoff 10211 itemsize 796
extent csum item
item 18 key (EXTENT_CSUM EXTENT_CSUM 6805698461696) itemoff 9799 itemsize 412
extent csum item
item 19 key (EXTENT_CSUM EXTENT_CSUM 6805698883584) itemoff 9259 itemsize 540
extent csum item
item 20 key (EXTENT_CSUM EXTENT_CSUM 6805699436544) itemoff 8827 itemsize 432
extent csum item
item 21 key (EXTENT_CSUM EXTENT_CSUM 6805699878912) itemoff 7403 itemsize 1424
extent csum item
ERROR: errors found in fs roots
found 7519512838144 bytes used, error(s) found
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
1
Cheers,
Chris.
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3575 bytes --]
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-01-26 23:31 ` Christoph Anton Mitterer
@ 2017-01-29 4:27 ` Qu Wenruo
2017-01-30 3:07 ` Christoph Anton Mitterer
0 siblings, 1 reply; 29+ messages in thread
From: Qu Wenruo @ 2017-01-29 4:27 UTC (permalink / raw)
To: Christoph Anton Mitterer, Qu Wenruo, linux-btrfs
On 01/27/2017 07:31 AM, Christoph Anton Mitterer wrote:
> On Thu, 2017-01-26 at 11:10 +0800, Qu Wenruo wrote:
>> Would you please try lowmem_tests branch of my repo?
>>
>> That branch contains a special debug output for the case you
>> encountered, which should help to debug the case.
>> pecial debug output for the case you encountered, which
>
> Here the output with your patches (again, not having applied the
> unnecessary fs-tests patches):
>
> In the output below I've replaced filenames with "[snip..snap]" and
> exchanged some of the xattr values.
> In case you should need their original values for testing, just tell me
> and I send them to you off-list.
Sorry for the late reply, in Chinese New Year vacation.
>
> btrfs-progs v4.9 WITH patches:
> ******************************
> # btrfs check --mode=lowmem /dev/nbd0 ; echo $?
> checking extents
> checking free space cache
> checking fs roots
> ERROR: root 6031 EXTENT_DATA[277 524288] datasum missing, have: 36864, expect: 45056 ret: 0
Thanks a lot for your debug output!
This shows the full reason of the problem! And it can be fixed easily now.
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> === fs tree leaf dump: slot: 136 ===
> leaf 5960902508544 items 191 free space 60 generation 2775 owner 6031
> fs uuid 326d292d-f97b-43ca-b1e8-c722d3474719
> chunk uuid 5da7e818-7f0b-43c1-b465-fdfaa52da633
<snipped>
> item 136 key (277 EXTENT_DATA 524288) itemoff 8443 itemsize 53
> generation 2775 type 1 (regular)
> extent data disk byte 6805694144512 nr 36864
> extent data offset 0 nr 45056 ram 45056
> extent compression 1 (zlib)
I just forgot that, we must handle plain extent and compressed extent
separately.
For compressed case, we should not use extent offset and nr_bytes, but
use the full extent.
I'll update the patchset soon to address it.
Thanks again for your detailed output and patience,
Qu
> === csum tree dump ===
> leaf 4983028940800 items 22 free space 6853 generation 3093 owner 7
> fs uuid 326d292d-f97b-43ca-b1e8-c722d3474719
> chunk uuid 5da7e818-7f0b-43c1-b465-fdfaa52da633
> item 0 key (EXTENT_CSUM EXTENT_CSUM 6805691457536) itemoff 16279 itemsize 4
> extent csum item
> item 1 key (EXTENT_CSUM EXTENT_CSUM 6805691461632) itemoff 16275 itemsize 4
> extent csum item
> item 2 key (EXTENT_CSUM EXTENT_CSUM 6805691465728) itemoff 16263 itemsize 12
> extent csum item
> item 3 key (EXTENT_CSUM EXTENT_CSUM 6805691478016) itemoff 16231 itemsize 32
> extent csum item
> item 4 key (EXTENT_CSUM EXTENT_CSUM 6805691510784) itemoff 16223 itemsize 8
> extent csum item
> item 5 key (EXTENT_CSUM EXTENT_CSUM 6805691518976) itemoff 16199 itemsize 24
> extent csum item
> item 6 key (EXTENT_CSUM EXTENT_CSUM 6805691543552) itemoff 16183 itemsize 16
> extent csum item
> item 7 key (EXTENT_CSUM EXTENT_CSUM 6805691559936) itemoff 15159 itemsize 1024
> extent csum item
> item 8 key (EXTENT_CSUM EXTENT_CSUM 6805692608512) itemoff 15151 itemsize 8
> extent csum item
> item 9 key (EXTENT_CSUM EXTENT_CSUM 6805692616704) itemoff 15055 itemsize 96
> extent csum item
> item 10 key (EXTENT_CSUM EXTENT_CSUM 6805692715008) itemoff 14903 itemsize 152
> extent csum item
> item 11 key (EXTENT_CSUM EXTENT_CSUM 6805693132800) itemoff 14519 itemsize 384
> extent csum item
> item 12 key (EXTENT_CSUM EXTENT_CSUM 6805693526016) itemoff 14391 itemsize 128
> extent csum item
> item 13 key (EXTENT_CSUM EXTENT_CSUM 6805693919232) itemoff 14135 itemsize 256
> extent csum item
> item 14 key (EXTENT_CSUM EXTENT_CSUM 6805694443520) itemoff 12727 itemsize 1408
> extent csum item
> item 15 key (EXTENT_CSUM EXTENT_CSUM 6805695885312) itemoff 11811 itemsize 916
> extent csum item
> item 16 key (EXTENT_CSUM EXTENT_CSUM 6805696823296) itemoff 11007 itemsize 804
> extent csum item
> item 17 key (EXTENT_CSUM EXTENT_CSUM 6805697646592) itemoff 10211 itemsize 796
> extent csum item
> item 18 key (EXTENT_CSUM EXTENT_CSUM 6805698461696) itemoff 9799 itemsize 412
> extent csum item
> item 19 key (EXTENT_CSUM EXTENT_CSUM 6805698883584) itemoff 9259 itemsize 540
> extent csum item
> item 20 key (EXTENT_CSUM EXTENT_CSUM 6805699436544) itemoff 8827 itemsize 432
> extent csum item
> item 21 key (EXTENT_CSUM EXTENT_CSUM 6805699878912) itemoff 7403 itemsize 1424
> extent csum item
> ERROR: errors found in fs roots
> found 7519512838144 bytes used, error(s) found
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 1
>
>
> Cheers,
> Chris.
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-01-29 4:27 ` Qu Wenruo
@ 2017-01-30 3:07 ` Christoph Anton Mitterer
2017-02-01 1:06 ` Qu Wenruo
0 siblings, 1 reply; 29+ messages in thread
From: Christoph Anton Mitterer @ 2017-01-30 3:07 UTC (permalink / raw)
To: Qu Wenruo, Qu Wenruo, linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 322 bytes --]
On Sun, 2017-01-29 at 12:27 +0800, Qu Wenruo wrote:
> Sorry for the late reply, in Chinese New Year vacation.
No worries... and happy new year then ;)
> I'll update the patchset soon to address it.
Just tell me and I re-check.
> Thanks again for your detailed output and patience,
Thanks as well :)
Chris.
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3575 bytes --]
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-01-30 3:07 ` Christoph Anton Mitterer
@ 2017-02-01 1:06 ` Qu Wenruo
2017-02-01 22:03 ` Christoph Anton Mitterer
0 siblings, 1 reply; 29+ messages in thread
From: Qu Wenruo @ 2017-02-01 1:06 UTC (permalink / raw)
To: Christoph Anton Mitterer, Qu Wenruo, linux-btrfs
At 01/30/2017 11:07 AM, Christoph Anton Mitterer wrote:
> On Sun, 2017-01-29 at 12:27 +0800, Qu Wenruo wrote:
>> Sorry for the late reply, in Chinese New Year vacation.
> No worries... and happy new year then ;)
Happy new year!
>
>
>> I'll update the patchset soon to address it.
> Just tell me and I re-check.
Updated, since I'm confident with this update, I'm not using the
lowmem_tests branch, but lowmem_fixes branch.
(lowmem_tests branch is now deleted in my repo)
https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes
Which is also rebased to latest v4.9.1.
Thanks,
Qu
>
>
>> Thanks again for your detailed output and patience,
> Thanks as well :)
>
> Chris.
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-02-01 1:06 ` Qu Wenruo
@ 2017-02-01 22:03 ` Christoph Anton Mitterer
2017-02-02 0:25 ` Qu Wenruo
0 siblings, 1 reply; 29+ messages in thread
From: Christoph Anton Mitterer @ 2017-02-01 22:03 UTC (permalink / raw)
To: Qu Wenruo; +Cc: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 6407 bytes --]
On Wed, 2017-02-01 at 09:06 +0800, Qu Wenruo wrote:
> https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes
>
> Which is also rebased to latest v4.9.1.
Same game as last time, applied to 4.9, no RW mount between the runs.
btrfs-progs v4.9 WITHOUT patch:
*******************************
# btrfs check /dev/nbd0 ; echo $?
checking extents
137
=> would be nice if btrfs-progrs could give a message on why it failed,
i.e. "not enough memory" or so.
# btrfs check --mode=lowmem /dev/nbd0 ; echo $?
checking extents
ERROR: block group[74117545984 1073741824] used 1073741824 but extent items used 0
ERROR: block group[239473786880 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[500393050112 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[581997428736 1073741824] used 1073741824 but extent items used 0
ERROR: block group[626557714432 1073741824] used 1073741824 but extent items used 0
ERROR: block group[668433645568 1073741824] used 1073741824 but extent items used 0
ERROR: block group[948680261632 1073741824] used 1073741824 but extent items used 0
ERROR: block group[982503129088 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1039411445760 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1054443831296 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[1190809042944 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1279392743424 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1481256206336 1073741824] used 1073741824 but extent items used 0
ERROR: block group[1620842643456 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[1914511032320 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[3055361720320 1073741824] used 1073741824 but extent items used 0
ERROR: block group[3216422993920 1073741824] used 1073741824 but extent items used 0
ERROR: block group[3670615785472 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[3801612288000 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[3828455833600 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[4250973241344 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4261710659584 1073741824] used 1073741824 but extent items used 1074266112
ERROR: block group[4392707162112 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4558063403008 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4607455526912 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4635372814336 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4640204652544 1073741824] used 1073741824 but extent items used 0
ERROR: block group[4642352136192 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[4681006841856 1073741824] used 1073741824 but extent items used 0
ERROR: block group[5063795802112 1073741824] used 1073741824 but extent items used 0
ERROR: block group[5171169984512 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[5216267141120 1073741824] used 1073741824 but extent items used 1207959552
ERROR: block group[5290355326976 1073741824] used 1073741824 but extent items used 0
ERROR: block group[5445511020544 1073741824] used 1073741824 but extent items used 1074266112
ERROR: block group[6084387405824 1073741824] used 1073741824 but extent items used 0
ERROR: block group[6104788500480 1073741824] used 1073741824 but extent items used 0
ERROR: block group[6878956355584 1073741824] used 1073741824 but extent items used 0
ERROR: block group[6997067956224 1073741824] used 1073741824 but extent items used 0
ERROR: block group[7702516334592 1073741824] used 1073741824 but extent items used 0
ERROR: block group[8051482427392 1073741824] used 1073741824 but extent items used 1084751872
ERROR: block group[8116980678656 1073741824] used 1073741824 but extent items used 0
ERROR: errors found in extent allocation tree or chunk allocation
checking free space cache
checking fs roots
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used err is -5
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
1
=> error still occurs *without* patch
=> increased VM memory here
# btrfs check /dev/nbd0 ; echo $?
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used err is 0
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
0
=> same as before
btrfs-progs v4.9 WITH patches:
******************************
# btrfs check /dev/nbd0 ; echo $?
checking extents
checking free space
cache
checking fs roots
checking csums
checking root refs
Checking
filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found
7519512838144 bytes used, no error found
total csum bytes: 7330834320
tot
al tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent
tree bytes: 1020149760
btree space waste bytes: 925714197
file data
blocks allocated: 7509228494848
referenced 7630551511040
0
=> same as
before
# btrfs check --mode=lowmem /dev/nbd0 ; echo $?
checking extents
checking free space cache
checking fs roots
ERROR: errors found in fs roots
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used, error(s) found
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
1
=> Unfortunately :-(
Cheers,
Chris.
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3575 bytes --]
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-02-01 22:03 ` Christoph Anton Mitterer
@ 2017-02-02 0:25 ` Qu Wenruo
2017-02-02 2:12 ` Qu Wenruo
0 siblings, 1 reply; 29+ messages in thread
From: Qu Wenruo @ 2017-02-02 0:25 UTC (permalink / raw)
To: Christoph Anton Mitterer; +Cc: linux-btrfs
Thanks for your test.
At 02/02/2017 06:03 AM, Christoph Anton Mitterer wrote:
> On Wed, 2017-02-01 at 09:06 +0800, Qu Wenruo wrote:
>> https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes
>>
>> Which is also rebased to latest v4.9.1.
>
> Same game as last time, applied to 4.9, no RW mount between the runs.
>
>
> btrfs-progs v4.9 WITHOUT patch:
> *******************************
> # btrfs check /dev/nbd0 ; echo $?
> checking extents
> 137
>
> => would be nice if btrfs-progrs could give a message on why it failed,
> i.e. "not enough memory" or so.
>
>
> # btrfs check --mode=lowmem /dev/nbd0 ; echo $?
> checking extents
> ERROR: block group[74117545984 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[239473786880 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[500393050112 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[581997428736 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[626557714432 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[668433645568 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[948680261632 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[982503129088 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1039411445760 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1054443831296 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[1190809042944 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1279392743424 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1481256206336 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[1620842643456 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[1914511032320 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[3055361720320 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[3216422993920 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[3670615785472 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[3801612288000 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[3828455833600 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[4250973241344 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4261710659584 1073741824] used 1073741824 but extent items used 1074266112
> ERROR: block group[4392707162112 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4558063403008 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4607455526912 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4635372814336 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4640204652544 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[4642352136192 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[4681006841856 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[5063795802112 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[5171169984512 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[5216267141120 1073741824] used 1073741824 but extent items used 1207959552
> ERROR: block group[5290355326976 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[5445511020544 1073741824] used 1073741824 but extent items used 1074266112
> ERROR: block group[6084387405824 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[6104788500480 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[6878956355584 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[6997067956224 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[7702516334592 1073741824] used 1073741824 but extent items used 0
> ERROR: block group[8051482427392 1073741824] used 1073741824 but extent items used 1084751872
> ERROR: block group[8116980678656 1073741824] used 1073741824 but extent items used 0
> ERROR: errors found in extent allocation tree or chunk allocation
> checking free space cache
> checking fs roots
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used err is -5
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 1
> => error still occurs *without* patch
>
> => increased VM memory here
>
> # btrfs check /dev/nbd0 ; echo $?
> checking extents
> checking free space cache
> checking fs roots
> checking csums
> checking root refs
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used err is 0
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 0
> => same as before
>
>
>
>
> btrfs-progs v4.9 WITH patches:
> ******************************
> # btrfs check /dev/nbd0 ; echo $?
> checking extents
> checking free space
> cache
> checking fs roots
> checking csums
> checking root refs
> Checking
> filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found
> 7519512838144 bytes used, no error found
> total csum bytes: 7330834320
> tot
> al tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent
> tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data
> blocks allocated: 7509228494848
> referenced 7630551511040
> 0
> => same as
> before
>
> # btrfs check --mode=lowmem /dev/nbd0 ; echo $?
> checking extents
> checking free space cache
> checking fs roots
> ERROR: errors found in fs roots
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used, error(s) found
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 1
>
> => Unfortunately :-(
That's embarrassing now.
At least the file extent bug is gone.
But still some new error is exposed, even without error message now.
I'll keep digging.
Thanks,
Qu
>
>
>
> Cheers,
> Chris.
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-02-02 0:25 ` Qu Wenruo
@ 2017-02-02 2:12 ` Qu Wenruo
[not found] ` <1486098502.7443.3.camel@lmu.de>
0 siblings, 1 reply; 29+ messages in thread
From: Qu Wenruo @ 2017-02-02 2:12 UTC (permalink / raw)
To: Christoph Anton Mitterer; +Cc: linux-btrfs
At 02/02/2017 08:25 AM, Qu Wenruo wrote:
> Thanks for your test.
>
> At 02/02/2017 06:03 AM, Christoph Anton Mitterer wrote:
>> On Wed, 2017-02-01 at 09:06 +0800, Qu Wenruo wrote:
>>> https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes
>>>
>>> Which is also rebased to latest v4.9.1.
>>
>> Same game as last time, applied to 4.9, no RW mount between the runs.
>>
>>
[snipped]
>> => same as
>> before
>>
>> # btrfs check --mode=lowmem /dev/nbd0 ; echo $?
>> checking extents
>> checking free space cache
>> checking fs roots
>> ERROR: errors found in fs roots
>> Checking filesystem on /dev/nbd0
>> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
>> found 7519512838144 bytes used, error(s) found
>> total csum bytes: 7330834320
>> total tree bytes: 10902437888
>> total fs tree bytes: 2019704832
>> total extent tree bytes: 1020149760
>> btree space waste bytes: 925714197
>> file data blocks allocated: 7509228494848
>> referenced 7630551511040
>> 1
>>
>> => Unfortunately :-(
>
Would you please to check the latest lowmem_tests branch?
https://github.com/adam900710/btrfs-progs/tree/lowmem_tests
This time I added several debug output to catch the whole path where the
silent error happens, to help us to solve the problem.
Sorry for the extra trouble.
Thanks,
Qu
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
[not found] ` <1486098502.7443.3.camel@lmu.de>
@ 2017-02-03 6:20 ` Qu Wenruo
2017-02-03 23:01 ` Christoph Anton Mitterer
0 siblings, 1 reply; 29+ messages in thread
From: Qu Wenruo @ 2017-02-03 6:20 UTC (permalink / raw)
To: Christoph Anton Mitterer, btrfs
At 02/03/2017 01:08 PM, Christoph Anton Mitterer wrote:
> Hey Qu.
>
> I'm sending this off-list for privacy reasons of the contained file-
> names / hash sums.
> It doesn't contained anything particularly secret, nevertheless, please
> try to avoid spreading it too far around and delete it once you don't
> need it anymore :)
>
> On Thu, 2017-02-02 at 10:12 +0800, Qu Wenruo wrote:
>> Would you please to check the latest lowmem_tests branch?
>> https://github.com/adam900710/btrfs-progs/tree/lowmem_tests
>
> Invocation was as usual:
> # btrfs check --mode=lowmem /dev/nbd0 ; echo $?
>
> stdout/err output in the attachment
Great thanks for that!
Although I made a mistake in the debug patch, the output info is still
good enough for me to catch the bug in inline extent check.
I also added missing error message output for other places I found, and
updated the branch, the name remains as "lowmem_tests"
Please try it.
Thanks,
Qu
>
>
>> Sorry for the extra trouble.
> No worries :)
>
>
> Cheers,
> Chris.
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-02-03 6:20 ` Qu Wenruo
@ 2017-02-03 23:01 ` Christoph Anton Mitterer
2017-02-06 1:25 ` Qu Wenruo
0 siblings, 1 reply; 29+ messages in thread
From: Christoph Anton Mitterer @ 2017-02-03 23:01 UTC (permalink / raw)
To: Qu Wenruo, btrfs
[-- Attachment #1: Type: text/plain, Size: 1530 bytes --]
Hey Qu
On Fri, 2017-02-03 at 14:20 +0800, Qu Wenruo wrote:
> Great thanks for that!
You're welcome. :)
> I also added missing error message output for other places I found,
> and
> updated the branch, the name remains as "lowmem_tests"
>
> Please try it.
# btrfs check /dev/nbd0 ; echo $?
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used, no error found
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
0
# btrfs check /dev/nbd0 ; echo $?
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
Checking filesystem on /dev/nbd0
UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
found 7519512838144 bytes used, no error found
total csum bytes: 7330834320
total tree bytes: 10902437888
total fs tree bytes: 2019704832
total extent tree bytes: 1020149760
btree space waste bytes: 925714197
file data blocks allocated: 7509228494848
referenced 7630551511040
0
=> looks good this time :)
btw: In your commit messages, please change my email to
calestyo@scientia.net everywhere... I accidentally used my university
address (christoph.anton.mitterer@lmu.de) sometimes when sending mail.
Cheers,
Chris.
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5930 bytes --]
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
2017-02-03 23:01 ` Christoph Anton Mitterer
@ 2017-02-06 1:25 ` Qu Wenruo
0 siblings, 0 replies; 29+ messages in thread
From: Qu Wenruo @ 2017-02-06 1:25 UTC (permalink / raw)
To: Christoph Anton Mitterer, btrfs
At 02/04/2017 07:01 AM, Christoph Anton Mitterer wrote:
> Hey Qu
>
> On Fri, 2017-02-03 at 14:20 +0800, Qu Wenruo wrote:
>> Great thanks for that!
> You're welcome. :)
>
>
>> I also added missing error message output for other places I found,
>> and
>> updated the branch, the name remains as "lowmem_tests"
>>
>> Please try it.
> # btrfs check /dev/nbd0 ; echo $?
> checking extents
> checking free space cache
> checking fs roots
> checking csums
> checking root refs
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used, no error found
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 0
>
> # btrfs check /dev/nbd0 ; echo $?
> checking extents
> checking free space cache
> checking fs roots
> checking csums
> checking root refs
> Checking filesystem on /dev/nbd0
> UUID: 326d292d-f97b-43ca-b1e8-c722d3474719
> found 7519512838144 bytes used, no error found
> total csum bytes: 7330834320
> total tree bytes: 10902437888
> total fs tree bytes: 2019704832
> total extent tree bytes: 1020149760
> btree space waste bytes: 925714197
> file data blocks allocated: 7509228494848
> referenced 7630551511040
> 0
>
> => looks good this time :)
Great!
>
>
> btw: In your commit messages, please change my email to
> calestyo@scientia.net everywhere... I accidentally used my university
> address (christoph.anton.mitterer@lmu.de) sometimes when sending mail.
Not a problem.
I'll change the address and re-send the patchset to mail list.
Thanks,
Qu
>
> Cheers,
> Chris.
>
^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2017-02-06 1:25 UTC | newest]
Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-01-23 9:13 [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Qu Wenruo
2017-01-23 9:13 ` [PATCH 1/9] btrfs-progs: lowmem check: Fix wrong block group check when search slot points to slot beyong leaf Qu Wenruo
2017-01-23 9:13 ` [PATCH 2/9] btrfs-progs: fsck-test: Add test image for lowmem mode block group false alert Qu Wenruo
2017-01-23 9:13 ` [PATCH 3/9] btrfs-progs: fsck: Output verbose error when fsck found a bug Qu Wenruo
2017-01-23 9:13 ` [PATCH 4/9] btrfs-progs: lowmem check: Fix false alert in checking data extent pointing to prealloc extent Qu Wenruo
2017-01-23 9:13 ` [PATCH 5/9] btrfs-progs: lowmem check: Fix extent item size false alert Qu Wenruo
2017-01-23 9:13 ` [PATCH 6/9] btrfs-progs: tests: Move fsck-tests/015 to fuzz tests Qu Wenruo
2017-01-23 9:13 ` [PATCH 7/9] btrfs-progs: fsck-tests: Make 013 compatible with lowmem mode Qu Wenruo
2017-01-23 9:13 ` [PATCH 8/8] btrfs-progs: fsck: Fix lowmem mode override to allow it skip repair work Qu Wenruo
2017-01-23 9:13 ` [PATCH 8/9] btrfs-progs: fsck-tests: Add new test case for partly written prealloc extent Qu Wenruo
2017-01-23 9:13 ` [PATCH 9/9] btrfs-progs: fsck: Fix lowmem mode override to allow it skip repair work Qu Wenruo
2017-01-24 16:54 ` [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update Christoph Anton Mitterer
2017-01-25 0:44 ` Qu Wenruo
2017-01-25 0:46 ` Christoph Anton Mitterer
2017-01-25 4:16 ` Qu Wenruo
2017-01-25 4:40 ` Christoph Anton Mitterer
2017-01-26 2:50 ` Christoph Anton Mitterer
2017-01-26 3:10 ` Qu Wenruo
2017-01-26 3:30 ` Christoph Anton Mitterer
2017-01-26 23:31 ` Christoph Anton Mitterer
2017-01-29 4:27 ` Qu Wenruo
2017-01-30 3:07 ` Christoph Anton Mitterer
2017-02-01 1:06 ` Qu Wenruo
2017-02-01 22:03 ` Christoph Anton Mitterer
2017-02-02 0:25 ` Qu Wenruo
2017-02-02 2:12 ` Qu Wenruo
[not found] ` <1486098502.7443.3.camel@lmu.de>
2017-02-03 6:20 ` Qu Wenruo
2017-02-03 23:01 ` Christoph Anton Mitterer
2017-02-06 1:25 ` Qu Wenruo
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.