btrfs, broken design?

btrfs, broken design?

[Benoît Thiébault]

Hi everyone,

I am very interested in the features provided by btrfs.

I know it is still under active development and thus do not consider using it yet "in production", but the Wikipedia page describing btrfs contains a very frightening sentence:
"Edward Shiskin, one of the Reiser4 developers now working for Redhat, got asked in Q2/2010 to look more detailled into Btrfs and judged that it has a broken design. This design may lead in some cases to out of space problems."

I have read the linked mail exchange in the mailing list but it is very technical and I am not quite sure I understood everything. Could you please tell me if the problem has been fixed since?

Is there a planned date for the final release of btrfs?

Kind regards,

Ben

Re: btrfs, broken design?

[Chris Mason]

Excerpts from Beno=C3=AEt Thi=C3=A9bault's message of 2011-01-20 16:06:=
21 -0500:
> Hi everyone,
>=20
> I am very interested in the features provided by btrfs.
>=20
> I know it is still under active development and thus do not consider =
using it yet "in production", but the Wikipedia page describing btrfs c=
ontains a very frightening sentence:
> "Edward Shiskin, one of the Reiser4 developers now working for Redhat=
, got asked in Q2/2010 to look more detailled into Btrfs and judged tha=
t it has a broken design. This design may lead in some cases to out of =
space problems."
>=20
> I have read the linked mail exchange in the mailing list but it is ve=
ry technical and I am not quite sure I understood everything. Could you=
 please tell me if the problem has been fixed since?

There was a bug fixed as part of that discussion, and I think I also
better described the way the tree balancing works to Edward.

>=20
> Is there a planned date for the final release of btrfs?

A final release?  We'll keep improving things for a long time.  The
biggest missing feature today is btrfsck, which I'm working on full tim=
e
right now.

-chris
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs, broken design?

[Benoît Thiébault]

Thanks for your answer

Le 20 janv. 2011 =E0 22:20, Chris Mason a =E9crit :

> There was a bug fixed as part of that discussion, and I think I also
> better described the way the tree balancing works to Edward.

Maybe the wikipedia article should be modified then, because it is not =
very reinsuring :-)

> A final release?  We'll keep improving things for a long time.  The
> biggest missing feature today is btrfsck, which I'm working on full t=
ime
> right now.
>=20
> -chris

Still on the wikipedia page, it's written "Btrfs 1.0 (with finalized on=
-disk format) was originally slated for a late 2008 release,[5] but a s=
table release has not been made as of January 2011.", which is also ver=
y confusing.

According to you, is the version of btrfs in 2.6.37 ready for productio=
n? I mean, are there still chances that I may loose all my data if I us=
e btrfs?

Last question, do you know when the RAID-5 like capabilities will be av=
ailable?

Sorry to ask so many questions, I fully understand you and your team ar=
e working very hard on the project, but I was very confused by the Wiki=
pedia article.

Kind regards,

Ben--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs, broken design?

[Chester]

Btrfs has its own wiki page at https://btrfs.wiki.kernel.org which you
may find more helpful than what is on wikipedia.

2011/1/20 Beno=EEt Thi=E9bault <benoit.thiebault@gmail.com>:
> Thanks for your answer
>
> Le 20 janv. 2011 =E0 22:20, Chris Mason a =E9crit :
>
>> There was a bug fixed as part of that discussion, and I think I also
>> better described the way the tree balancing works to Edward.
>
> Maybe the wikipedia article should be modified then, because it is no=
t very reinsuring :-)
>
>> A final release? =A0We'll keep improving things for a long time. =A0=
The
>> biggest missing feature today is btrfsck, which I'm working on full =
time
>> right now.
>>
>> -chris
>
> Still on the wikipedia page, it's written "Btrfs 1.0 (with finalized =
on-disk format) was originally slated for a late 2008 release,[5] but a=
 stable release has not been made as of January 2011.", which is also v=
ery confusing.
>
> According to you, is the version of btrfs in 2.6.37 ready for product=
ion? I mean, are there still chances that I may loose all my data if I =
use btrfs?
>
> Last question, do you know when the RAID-5 like capabilities will be =
available?
>
> Sorry to ask so many questions, I fully understand you and your team =
are working very hard on the project, but I was very confused by the Wi=
kipedia article.
>
> Kind regards,
>
> Ben--
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs, broken design?

[Benoît Thiébault]

Ok, thanks, I will read the project wiki more carefully then :-)
Beware however that Wikipedia is the first place to look for informatio=
n for a lot of people (whether this is a good practice or not) and it c=
urrently does not provide a very good advertisement to btrfs.

Kind regards

Le 21 janv. 2011 =E0 07:46, Chester a =E9crit :

> Btrfs has its own wiki page at https://btrfs.wiki.kernel.org which yo=
u
> may find more helpful than what is on wikipedia.
>=20
> 2011/1/20 Beno=EEt Thi=E9bault <benoit.thiebault@gmail.com>:
>> Thanks for your answer
>>=20
>> Le 20 janv. 2011 =E0 22:20, Chris Mason a =E9crit :
>>=20
>>> There was a bug fixed as part of that discussion, and I think I als=
o
>>> better described the way the tree balancing works to Edward.
>>=20
>> Maybe the wikipedia article should be modified then, because it is n=
ot very reinsuring :-)
>>=20
>>> A final release?  We'll keep improving things for a long time.  The
>>> biggest missing feature today is btrfsck, which I'm working on full=
 time
>>> right now.
>>>=20
>>> -chris
>>=20
>> Still on the wikipedia page, it's written "Btrfs 1.0 (with finalized=
 on-disk format) was originally slated for a late 2008 release,[5] but =
a stable release has not been made as of January 2011.", which is also =
very confusing.
>>=20
>> According to you, is the version of btrfs in 2.6.37 ready for produc=
tion? I mean, are there still chances that I may loose all my data if I=
 use btrfs?
>>=20
>> Last question, do you know when the RAID-5 like capabilities will be=
 available?
>>=20
>> Sorry to ask so many questions, I fully understand you and your team=
 are working very hard on the project, but I was very confused by the W=
ikipedia article.
>>=20
>> Kind regards,
>>=20
>> Ben--
>> To unsubscribe from this list: send the line "unsubscribe linux-btrf=
s" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>=20
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs, broken design?

[Hubert Kario]

On Friday 21 of January 2011 09:11:57 Beno=EEt Thi=E9bault wrote:
> Ok, thanks, I will read the project wiki more carefully then :-)
> Beware however that Wikipedia is the first place to look for informat=
ion
> for a lot of people (whether this is a good practice or not) and it
> currently does not provide a very good advertisement to btrfs.

It's still aimed more at experts than even to advanced users, though th=
is will=20
change when btrfsck will become avaiable.
=20
> Le 21 janv. 2011 =E0 07:46, Chester a =E9crit :
> > Btrfs has its own wiki page at https://btrfs.wiki.kernel.org which =
you
> > may find more helpful than what is on wikipedia.
> >=20
> > 2011/1/20 Beno=EEt Thi=E9bault <benoit.thiebault@gmail.com>:
> >> Thanks for your answer
> >>=20
> >> Le 20 janv. 2011 =E0 22:20, Chris Mason a =E9crit :
> >>> There was a bug fixed as part of that discussion, and I think I a=
lso
> >>> better described the way the tree balancing works to Edward.
> >>=20
> >> Maybe the wikipedia article should be modified then, because it is=
 not
> >> very reinsuring :-)
> >>=20
> >>> A final release?  We'll keep improving things for a long time.  T=
he
> >>> biggest missing feature today is btrfsck, which I'm working on fu=
ll
> >>> time right now.
> >>>=20
> >>> -chris
> >>=20
> >> Still on the wikipedia page, it's written "Btrfs 1.0 (with finaliz=
ed
> >> on-disk format) was originally slated for a late 2008 release,[5] =
but a
> >> stable release has not been made as of January 2011.", which is al=
so
> >> very confusing.
> >>=20
> >> According to you, is the version of btrfs in 2.6.37 ready for
> >> production? I mean, are there still chances that I may loose all m=
y
> >> data if I use btrfs?
> >>=20
> >> Last question, do you know when the RAID-5 like capabilities will =
be
> >> available?
> >>=20
> >> Sorry to ask so many questions, I fully understand you and your te=
am are
> >> working very hard on the project, but I was very confused by the
> >> Wikipedia article.
> >>=20
> >> Kind regards,
> >>=20
> >> Ben--
> >> To unsubscribe from this list: send the line "unsubscribe linux-bt=
rfs"
> >> in the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >=20
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-btr=
fs" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
>=20
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--=20
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawer=F3w 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

version (was: btrfs, broken design?)

[Helmut Hullen]

Hallo, Chris,

Du meintest am 20.01.11:

>> Is there a planned date for the final release of btrfs?

> A final release?  We'll keep improving things for a long time.  The
> biggest missing feature today is btrfsck, which I'm working on full
> time right now.

Could it be possible to tell somewhere the actual version?

Sometimes I download via git,

 <git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstable.git>


and I never have found which version that is; "version.sh" tells  
something wrong.

And I never have seen somethin like "Changelog" - that would be fine  
too.

Viele Gruesse!
Helmut

Re: version (was: btrfs, broken design?)

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1485 bytes --]

On Fri, Jan 21, 2011 at 10:54:00AM +0100, Helmut Hullen wrote:
> Hallo, Chris,
> 
> Du meintest am 20.01.11:
> 
> >> Is there a planned date for the final release of btrfs?
> 
> > A final release?  We'll keep improving things for a long time.  The
> > biggest missing feature today is btrfsck, which I'm working on full
> > time right now.
> 
> Could it be possible to tell somewhere the actual version?
> 
> Sometimes I download via git,
> 
>  <git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstable.git>

> and I never have found which version that is; "version.sh" tells  
> something wrong.

$ git log

   then look at the revision ID of the top commit -- that's the
closest thing to a version number we've got. When you build from the
git repository, the version number that the tools report will be
something like 0.19-36-g70c6c10. This means that it's 36 commits on
from the version tagged as 0.19, and the last commit was g70c6c10.

   (This is for the userspace tools, of course -- the kernel has a
similar numbering scheme, if you're not building from Linus's tagged
versions).

> And I never have seen somethin like "Changelog" - that would be fine  
> too.

$ git log

   :)

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
       --- He's playing Schubert.  I think Schubert is losing. ---       

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: version

[Helmut Hullen]

Hallo, Hugo,

Du meintest am 21.01.11:

>> Could it be possible to tell somewhere the actual version?
>>
>> Sometimes I download via git,
>>
>>  <git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-un
>>  stable.git>

>> and I never have found which version that is; "version.sh" tells
>> something wrong.

> $ git log

Ok - last changes from October 6, 2010

Viele Gruesse!
Helmut

Re: version (was: btrfs, broken design?)

[Diego Calleja]

On Viernes, 21 de Enero de 2011 10:54:00 Helmut Hullen escribi=F3:

> And I never have seen somethin like "Changelog" - that would be fine =
=20
> too.

Check the wiki, I keep that updated: https://btrfs.wiki.kernel.org/inde=
x.php/Main_Page#News
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: version

[Helmut Hullen]

Hallo, Diego,

Du meintest am 21.01.11:

>> And I never have seen somethin like "Changelog" - that would be fine
>> too.

> Check the wiki, I keep that updated:
> https://btrfs.wiki.kernel.org/index.php/Main_Page#News

Thank you - that's more simple for me than first cloning via "git clone"  
and then running "git log".

Viele Gruesse!
Helmut

Re: version

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 971 bytes --]

On Fri, Jan 21, 2011 at 03:55:00PM +0100, Helmut Hullen wrote:
> Hallo, Diego,
> 
> Du meintest am 21.01.11:
> 
> >> And I never have seen somethin like "Changelog" - that would be fine
> >> too.
> 
> > Check the wiki, I keep that updated:
> > https://btrfs.wiki.kernel.org/index.php/Main_Page#News
> 
> Thank you - that's more simple for me than first cloning via "git clone"  
> and then running "git log".

   Once you've done a git clone, you can keep it up to date by running
"git pull". This is pretty efficient, and will only transfer the
changes since the last time you retrieved it.

   Sadly, it won't help you with reading the output of "git log". :)

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- But somewhere along the line, it seems / That pimp became ---    
                       cool,  and punk mainstream.                       

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: version

[Helmut Hullen]

Hallo, Hugo,

Du meintest am 21.01.11:

>>> Check the wiki, I keep that updated:
>>> https://btrfs.wiki.kernel.org/index.php/Main_Page#News
>>
>> Thank you - that's more simple for me than first cloning via "git
>> clone" and then running "git log".

>    Once you've done a git clone, you can keep it up to date by
> running "git pull". This is pretty efficient, and will only transfer
> the changes since the last time you retrieved it.

That's true (surely!). But my major interest is only to run btrfs  
(without those nasty problems like ENOSPC), not to develop it.

For developping I've enough other playing grounds (too much ...).

>    Sadly, it won't help you with reading the output of "git log". :)

That's a minor problem - just 3 minutes to find the way git wants.

Viele Gruesse!
Helmut

Re: version

[Chris Samuel]

On 22/01/11 01:55, Helmut Hullen wrote:

> Thank you - that's more simple for me than first cloning via
> "git clone" and then running "git log".

Ah, but the repo you were asked to clone was for the user tools,
not for the kernel code that implements the filesystem itself
which is what that Wiki page is for.

cheers!
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

Re: version

[Helmut Hullen]

Hallo, Chris,

Du meintest am 24.01.11:

>> Thank you - that's more simple for me than first cloning via
>> "git clone" and then running "git log".

> Ah, but the repo you were asked to clone was for the user tools,
> not for the kernel code that implements the filesystem itself
> which is what that Wiki page is for.

Yes - I've seen that small difference ... but I've installed 2.6.37.

It includes btrfs changes (from the btrfs-unstable branch) until 2010- 
12-14.

My other problem: where has the ENOSPC problem be cured? Is it a kernel  
problem?

Viele Gruesse!
Helmut

Re: version

[Johannes Hirte]

On Monday 24 January 2011 09:33:00 Helmut Hullen wrote:
> Hallo, Chris,
> 
> Du meintest am 24.01.11:
> >> Thank you - that's more simple for me than first cloning via
> >> "git clone" and then running "git log".
> > 
> > Ah, but the repo you were asked to clone was for the user tools,
> > not for the kernel code that implements the filesystem itself
> > which is what that Wiki page is for.
> 
> Yes - I've seen that small difference ... but I've installed 2.6.37.
> 
> It includes btrfs changes (from the btrfs-unstable branch) until 2010-
> 12-14.
> 
> My other problem: where has the ENOSPC problem be cured? Is it a kernel
> problem?

Which one? And yes, ENOSPC is kernel related.

regards,
  Johannes

Re: version

[Helmut Hullen]

Hallo, Johannes,

Du meintest am 24.01.11:

>> My other problem: where has the ENOSPC problem be cured? Is it a
>> kernel problem?

> Which one? And yes, ENOSPC is kernel related.

The problem:

I create 1 btrfs device with 2 TByte
I add another device with 40 GByte

I fill the btrfs "device" with 30 GByte and then run "balance".

That results in


# btrfs filesystem show
Label: none  uuid: 121ae2ed-f572-43e6-8855-cd66ad534401
	Total devices 2 FS bytes used 94.68GB
	devid    2 size 37.27GB used 37.00GB path /dev/sdf
	devid    1 size 1.82TB used 63.04GB path /dev/sde

Label: 'MM2'  uuid: ad7c0668-316c-4a79-ba00-3b505b9d99b4
	Total devices 2 FS bytes used 1.15TB
	devid    1 size 1.81TB used 591.14GB path /dev/sde2
	*** Some devices missing

Btrfs Btrfs v0.19

# btrfs filesystem df /mnt/btr
Data, RAID0: total=3D4.00GB, used=3D2.00GB
Data: total=3D94.01GB, used=3D92.58GB
System, DUP: total=3D8.00MB, used=3D16.00KB
System: total=3D4.00MB, used=3D0.00
Metadata, DUP: total=3D1.00GB, used=3D111.87MB
Metadata: total=3D8.00MB, used=3D0.00

# df -t btrfs
=46ilesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sde     btrfs   1992597264  99396500 1891304076   5% /mnt/btr

# fdisk -l

# btrfs filesystem show
Label: none  uuid: 121ae2ed-f572-43e6-8855-cd66ad534401
	Total devices 2 FS bytes used 93.68GB
	devid    2 size 37.27GB used 36.12GB path /dev/sdf
	devid    1 size 1.82TB used 93.16GB path /dev/sde

Label: 'MM2'  uuid: ad7c0668-316c-4a79-ba00-3b505b9d99b4
	Total devices 2 FS bytes used 1.15TB
	devid    1 size 1.81TB used 591.14GB path /dev/sde2
	*** Some devices missing

Btrfs Btrfs v0.19

# btrfs filesystem df /mnt/btr
Data, RAID0: total=3D64.00GB, used=3D31.90GB
Data: total=3D63.01GB, used=3D61.67GB
System, DUP: total=3D8.00MB, used=3D20.00KB
System: total=3D4.00MB, used=3D0.00
Metadata, RAID1: total=3D128.00MB, used=3D104.71MB
Metadata, DUP: total=3D1.00GB, used=3D6.15MB
Metadata: total=3D8.00MB, used=3D0.00

# df -t btrfs
=46ilesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sde     btrfs   1992597264  98348716 1892087652   5% /mnt/btr

# ---------------------------------------------------

Regard the difference between "df" and "btrfs filesystem df".

When I copy some more files then the system tells "no space left on ...=
"

Yesterday I've run a similar experiment with 40 and 8 GByte - same bad =
=20
behaviour:


# btrfs filesystem show
Label: none  uuid: da2f178e-a364-4494-a9e5-a7762ed81d10
	Total devices 2 FS bytes used 3.83GB
	devid    1 size 7.51GB used 6.13GB path /dev/sdb
	devid    2 size 37.27GB used 6.13GB path /dev/sdc

Btrfs Btrfs v0.19

# btrfs filesystem df /mnt/btr
Data, RAID0: total=3D12.00GB, used=3D3.83GB
System, RAID1: total=3D8.00MB, used=3D4.00KB
System: total=3D4.00MB, used=3D0.00
Metadata, RAID1: total=3D128.00MB, used=3D4.47MB

# df -t btrfs
Dateisystem   Typ    1K-Bl=F6cke   Benutzt Verf=FCgbar Ben% Eingeh=E4ng=
t auf
/dev/sdb     btrfs    46963224   4021964  42667804   9% /mnt/btr


Viele Gruesse!
Helmut
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: version

[Chris Samuel]

On 25/01/11 08:39, Helmut Hullen wrote:

> Regard the difference between "df" and "btrfs filesystem df".

I suspect this is fixed in 2.6.38 with the following commit.

BE WARNED: there are some fairly hairy changes to the pathname
lookup code to replace the BKL with RCU (not specific to btrfs)
and so if you are tempted to try it (currently 2.6.38-rc2) only
do so on a system that you don't care about data on and/or have
very good incremental backups of which you trust...


commit 6d07bcec969af335d4e35b3921131b7929bd634e
Author: Miao Xie <miaox@cn.fujitsu.com>
Date:   Wed Jan 5 10:07:31 2011 +0000

    btrfs: fix wrong free space information of btrfs

    When we store data by raid profile in btrfs with two or more
different size
    disks, df command shows there is some free space in the filesystem,
but the
    user can not write any data in fact, df command shows the wrong free
space
    information of btrfs.

[...]

    It is because btrfs cannot allocate chunks when one of the pairing
disks has
    no space, the free space on the other disks can not be used for
ever, and should
    be subtracted from the total space, but btrfs doesn't subtract this
space from
    the total. It is strange to the user.


-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

Re: version

[Chris Mason]

Excerpts from Chris Samuel's message of 2011-01-24 17:46:06 -0500:
> On 25/01/11 08:39, Helmut Hullen wrote:
> 
> > Regard the difference between "df" and "btrfs filesystem df".
> 
> I suspect this is fixed in 2.6.38 with the following commit.
> 
> BE WARNED: there are some fairly hairy changes to the pathname
> lookup code to replace the BKL with RCU (not specific to btrfs)
> and so if you are tempted to try it (currently 2.6.38-rc2) only
> do so on a system that you don't care about data on and/or have
> very good incremental backups of which you trust...

Yes, and you can pull in these commits on top of 2.6.37 or 2.6.36 by
pulling in the btrfs-unstable git tree.  But I do agree they should fix
it.

-chris

Re: version

[Helmut Hullen]

Hallo, Chris,

Du meintest am 25.01.11:

>> Regard the difference between "df" and "btrfs filesystem df".

> I suspect this is fixed in 2.6.38 with the following commit.

[...]

Looks good - I'll take a try.

Viele Gruesse!
Helmut

Re: version

[Helmut Hullen]

Hallo, Chris,

Du meintest am 25.01.11:

>> Regard the difference between "df" and "btrfs filesystem df".

> I suspect this is fixed in 2.6.38 with the following commit.

> BE WARNED: there are some fairly hairy changes to the pathname
> lookup code to replace the BKL with RCU (not specific to btrfs)
> and so if you are tempted to try it (currently 2.6.38-rc2) only
> do so on a system that you don't care about data on and/or have
> very good incremental backups of which you trust...


> commit 6d07bcec969af335d4e35b3921131b7929bd634e
> Author: Miao Xie <miaox@cn.fujitsu.com>
> Date:   Wed Jan 5 10:07:31 2011 +0000

>     btrfs: fix wrong free space information of btrfs

I've tried 2.6.38-rc2 - new problems.

        mkfs.btrfs /dev/sdb
        mount /dev/sdb /mnt/btr
        btrfs device add /dev/sdc

        cp <dir_with_6_GByte> /mnt/btr

leads to

-----------

# btrfs filesystem show
Label: none  uuid: 4a8c2c48-6c0e-4a97-8286-d1f7d930f9a8
	Total devices 2 FS bytes used 6.53GB
	devid    1 size 7.51GB used 805.50MB path /dev/sdb
	devid    2 size 37.27GB used 7.00GB path /dev/sdc

Btrfs Btrfs v0.19

# btrfs filesystem df /mnt/btr
Data: total=3D7.01GB, used=3D6.52GB
System, DUP: total=3D8.00MB, used=3D4.00KB
System: total=3D4.00MB, used=3D0.00
Metadata, DUP: total=3D384.75MB, used=3D9.84MB
Metadata: total=3D8.00MB, used=3D0.00

# df -t btrfs
Dateisystem   Typ    1K-Bl=F6cke   Benutzt Verf=FCgbar Ben% Eingeh=E4ng=
t auf
/dev/sdb     btrfs    46963224   6861804  39303836  15% /mnt/btr

-----------

And then:

        btrfs filesystem balance /mnt/btr

crashes with the "dmesg" lines

--------------------- dmesg -------------------

bio too big device sdc (256 > 240)
bio too big device sdc (256 > 240)
bio too big device sdc (256 > 240)
bio too big device sdc (256 > 240)
------------[ cut here ]------------
kernel BUG at fs/btrfs/volumes.c:2097!
invalid opcode: 0000 [#1]
last sysfs file: /sys/devices/pci0000:00/0000:00:07.1/host1/target1:0:0=
/1:0:0:0/block/sdb/dev
Modules linked in: sg nf_nat_ftp nf_conntrack_ftp ipt_MASQUERADE iptabl=
e_nat nf_nat xt_DSCP xt_multiport xt_recent nf_conntrack_ipv4 nf_defrag=
_ipv4 xt_state nf_conntrack xt_tcpudp ipt_REJECT iptable_filter iptable=
_mangle ip_tables xt_iprange x_tables nfsd exportfs 8139too 8139cp sava=
gefb fb_ddc i2c_algo_bit vgastate i2c_piix4 piix e100 mii intel_agp int=
el_gtt agpgart cmd64x video thermal_sys ac battery yenta_socket pcmcia_=
rsrc pcmcia pcmcia_core thinkpad_acpi hwmon rfkill nvram fuse

Pid: 16501, comm: btrfs Not tainted 2.6.38-rc2-ODS #1 26478EG/26478EG
EIP: 0060:[<c1235264>] EFLAGS: 00010282 CPU: 0
EIP is at btrfs_balance+0x2d4/0x2e0
EAX: fffffffb EBX: cd570000 ECX: d7cd4090 EDX: 00000000
ESI: d097c070 EDI: cf9f3c00 EBP: d37dde9c ESP: d37dde38
 DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068
Process btrfs (pid: 16501, ti=3Dd37dc000 task=3Dcd5d6f00 task.ti=3Dd37d=
c000)
Stack:
 99cc0000 00000001 000000e4 00100000 00000000 00000100 cd56e000 cf9f67d=
8
 aea58000 00000001 00000000 99cc0000 00000001 0100a3c8 00000000 00e4000=
0
 0199cc00 00000000 00000001 e4000000 ffffffff ffffffff cfa50700 ffffffe=
a
Call Trace:
 [<c123bcf1>] btrfs_ioctl+0x2e1/0x9d0
 [<c123ba10>] ? btrfs_ioctl+0x0/0x9d0
 [<c10c3f65>] do_vfs_ioctl+0x85/0x590
 [<c10206db>] ? do_page_fault+0x17b/0x380
 [<c10b554b>] ? do_sys_open+0xdb/0x110
 [<c10c44f7>] sys_ioctl+0x87/0x90
 [<c1753d0c>] syscall_call+0x7/0xb
Code: 1b ff ff ff 89 f0 e8 cc 75 fb ff 8b 55 b4 8b 82 10 01 00 00 05 74=
 19 00 00 e8 09 dc 51 00 e9 70 fd ff ff 31 db eb dd 85 c0 74 9d <0f> 0b=
 0f 0b 0f 0b 0f 0b 0f 0b 66 90 55 89 e5 56 53 83 ec 34 3e
EIP: [<c1235264>] btrfs_balance+0x2d4/0x2e0 SS:ESP 0068:d37dde38
---[ end trace 8dcdbc0f75858a35 ]---

---------------------------------------


Viele Gruesse!
Helmut
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: version (was: btrfs, broken design?)

[Erik Logtenberg]

On 01/21/2011 03:32 PM, Diego Calleja wrote:
> On Viernes, 21 de Enero de 2011 10:54:00 Helmut Hullen escribi=F3:
>=20
>> And I never have seen somethin like "Changelog" - that would be fine=
 =20
>> too.
>=20
> Check the wiki, I keep that updated: https://btrfs.wiki.kernel.org/in=
dex.php/Main_Page#News


I like the Changelog on the wiki [1] very much, since it shows the most
important changes in easily understandable language. Unfortunately, the
most recent change is 2.6.35 (august 2010), while we are currently at
2.6.37 (stable) and 2.6.38-rc2 (mainline). Especially 2.6.38(-rc2)
contains many interesting new btrfs features and lots of important
fixes. It would be very nice if the Changelog could list/explain those.

[1] https://btrfs.wiki.kernel.org/index.php/Changelog

The News [2] section on the Main page does mention one more version,
2.6.37. But the news section is less elaborate than the Changelog and
also I notice that 2.6.36 is not mentioned in the News section. Still,
2.6.36 does contain all kinds of btrfs-related changes.

[2] https://btrfs.wiki.kernel.org/index.php/Main_Page#News

Diego, pls don't read anything negative in my comments, I enjoy and
respect your work very much! If you could find time to add those latest
changes to the wiki, it would be greatly appreciated.

Kind regards,

Erik.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: version (was: btrfs, broken design?)

[Diego Calleja]

On Mi=E9rcoles, 26 de Enero de 2011 11:13:20 Erik Logtenberg escribi=F3=
:
> Diego, pls don't read anything negative in my comments, I enjoy and
> respect your work very much! If you could find time to add those late=
st
> changes to the wiki, it would be greatly appreciated.

Thanks for your suggestion, I've updated the Changelog and removed the =
old
items from the news section.

2.6.36 didn't had many btrfs changes, there was no new features.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

2.6.38-rc2 oops's when rebalancing on different size drives (was Re: version)

[Chris Samuel]

On 26/01/11 01:37, Helmut Hullen wrote:

> bio too big device sdc (256 > 240)
> bio too big device sdc (256 > 240)
> bio too big device sdc (256 > 240)
> bio too big device sdc (256 > 240)

Oh dear, those are errors from the block layer, looks like
btrfs is doing something wrong there.. :-(

> ------------[ cut here ]------------
> kernel BUG at fs/btrfs/volumes.c:2097!

It looks like btrfs isn't handling errors coming back from the
block layer - at that point it's just called btrfs_relocate_chunk()..

So my guess is that the rebalancing code is naive and assumes
the drives are the same size - but I can't quite follow what
the code above that BUG_ON() is doing to verify that..

Chris M. ?

cheers!
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

Re: version

[Chris Mason]

Excerpts from Helmut Hullen's message of 2011-01-25 09:37:00 -0500:
> crashes with the "dmesg" lines
> 
> --------------------- dmesg -------------------
> 
> bio too big device sdc (256 > 240)
> bio too big device sdc (256 > 240)
> bio too big device sdc (256 > 240)
> bio too big device sdc (256 > 240)
> ------------[ cut here ]------------
> kernel BUG at fs/btrfs/volumes.c:2097!

Ugh, this one is an old friend I thought I had fixed up.  The two
devices have different limits on the max size of the bio, and we're
using one that is too large.

I'll get it fixed for the next rc.

-chris

Re: version

[Helmut Hullen]

Hallo, Chris,

Du meintest am 27.01.11:

>> crashes with the "dmesg" lines
>>
>> --------------------- dmesg -------------------
>>
>> bio too big device sdc (256 > 240)
>> bio too big device sdc (256 > 240)
>> bio too big device sdc (256 > 240)
>> bio too big device sdc (256 > 240)
>> ------------[ cut here ]------------
>> kernel BUG at fs/btrfs/volumes.c:2097!

> Ugh, this one is an old friend I thought I had fixed up.  The two
> devices have different limits on the max size of the bio, and we're
> using one that is too large.

> I'll get it fixed for the next rc.

Thank you!
The one device is a CF card with 8 GB, the other is a SSD with 40 GByte.  
Maybe it's strange mixing them ... but I don't dare to test my 2 TByte  
disks.

Viele Gruesse!
Helmut

Re: version

[Helmut Hullen]

Hallo, Chris,

Du meintest am 27.01.11:

>> crashes with the "dmesg" lines
>>
>> --------------------- dmesg -------------------
>>
>> bio too big device sdc (256 > 240)
>> bio too big device sdc (256 > 240)
>> bio too big device sdc (256 > 240)
>> bio too big device sdc (256 > 240)
>> ------------[ cut here ]------------
>> kernel BUG at fs/btrfs/volumes.c:2097!

> Ugh, this one is an old friend I thought I had fixed up.  The two
> devices have different limits on the max size of the bio, and we're
> using one that is too large.

> I'll get it fixed for the next rc.

Seems to be not related to SSD or CFdisk; I've run the same commands  
with two real HDs and got the same error messages.

That problem appears with Kernel 2.6.38-rc2, it doesn't appear with  
Kernel 2.6.37 (but there I've still the ENOSPC problem).

Viele Gruesse!
Helmut

no space left (was: version)

[Helmut Hullen]

Hallo, Chris,

Du meintest am 27.01.11:

>> crashes with the "dmesg" lines
>>
>> --------------------- dmesg -------------------
>>
>> bio too big device sdc (256 > 240)
>> bio too big device sdc (256 > 240)
>> bio too big device sdc (256 > 240)
>> bio too big device sdc (256 > 240)
>> ------------[ cut here ]------------
>> kernel BUG at fs/btrfs/volumes.c:2097!

> Ugh, this one is an old friend I thought I had fixed up.  The two
> devices have different limits on the max size of the bio, and we're
> using one that is too large.

> I'll get it fixed for the next rc.

Sorry - the problem seems to exist in kernel 2.6.37 too.

/dev/sdb1 with about 8 GByte
/dev/sdc1 with about 12 GByte

    mkfs.btrfs /dev/sdb1
    mount /dev/sdb1 /mnt/btr
    cp <dir with 6 GByte>
    btrfs device add /dev/sdc1 /mnt/btr

looks good.

    btrfs filesystem balance /mnt/btr

seems to look good.

But then (at about 14:34)

    cp <dir with 2 Gbyte> /mnt/btr

sends the following lines to "/var/log/messages":

-------------- var/log/messages ----------------------

Jan 27 14:08:15 ElNath kernel: device fsid e144bfc5cb117745- 
187a9339eb8c0daa devid 1 transid 7 /dev/sdb1
Jan 27 14:24:04 ElNath kernel: end_request: I/O error, dev fd0, sector 0

Jan 27 14:25:34 ElNath kernel: btrfs: relocating block group 7330660352 flags 1
Jan 27 14:25:35 ElNath last message repeated 2 times
Jan 27 14:25:35 ElNath kernel: btrfs: relocating block group 12582912 flags 1
Jan 27 14:25:36 ElNath kernel: btrfs: found 19 extents
Jan 27 14:25:36 ElNath kernel: btrfs: found 19 extents
Jan 27 14:25:36 ElNath kernel: btrfs: relocating block group 4194304 flags 4
Jan 27 14:26:37 ElNath kernel: end_request: I/O error, dev fd0, sector 0

Jan 27 14:34:28 ElNath kernel: bio too big device sdc1 (256 > 240)
Jan 27 14:34:59 ElNath last message repeated 2474 times
Jan 27 14:35:02 ElNath last message repeated 357 times
Jan 27 14:35:04 ElNath squid[3493]: NETDB state saved; 0 entries, 239 msec

but the system seems to copy all files, without any "no space left".

Does this error depend on the sequence (?) of the commands?

Earlier I always typed

        mkfs.btrfs /dev/sdb1
        mount /dev/sdb1 /mnt/btr
        btrfs device add /mnt/sdc1 /mnt/btr
        cp ...
        btrfs filesystem balance /mnt/btr

and then copying more files led to "no space left".

Viele Gruesse!
Helmut