From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from resqmta-ch2-07v.sys.comcast.net ([69.252.207.39]:34202 "EHLO resqmta-ch2-07v.sys.comcast.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730946AbeG1W0p (ORCPT ); Sat, 28 Jul 2018 18:26:45 -0400 Date: Sat, 28 Jul 2018 16:50:49 -0400 (EDT) From: jkexcel Reply-To: jkexcel To: linux-btrfs@vger.kernel.org Message-ID: <1438689914.6655.1532811050018@connect.xfinity.com> Subject: btrfs-convert missing in btrfs-tools v4.15.1 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_6654_1787011089.1532811050017" Sender: linux-btrfs-owner@vger.kernel.org List-ID: ------=_Part_6654_1787011089.1532811050017 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit I'm an end user trying to use btrfs-convert but when I installed btrfs-tools and its dependency btrfs-progs on kubuntu 18.04, the installation was successful, and it shows that v4.15.1-1build1 was installed. However, when using the command # brtfs-convert /dev/sda4 (with the drive unmounted) the resulting error appears "command not found" I also tried command "btrfs convert" in case this was folded into the main tool, but this also failed. 1. Is btrfs-convert still available? 2. Where can I find it? 3. Has btrfs-convert been replaced? what is it's new name? 4. Is it safe to use a downgraded version of btrfs-tools ie: 4.14 or older? ------=_Part_6654_1787011089.1532811050017 MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

I'm an end user trying to use btrfs-co= nvert but when I installed btrfs-tools and its dependency btrfs-progs on ku= buntu 18.04, the installation was successful, and it shows that v4.15.1-1bu= ild1 was installed.


Howe= ver, when using the command  # brtfs-convert  /dev/sda4 (with the= drive unmounted) the resulting error appears "command not found"


I also tried command "= ;btrfs convert" in case this was folded into the main tool, but this al= so failed.


1. Is btrfs-c= onvert still available?

2. Where can I find it?

<= p style=3D"color: rgb(51, 51, 51); font-family: helvetica,arial,sans-serif;= font-size: 12pt;">3. Has btrfs-convert been replaced? what is it's new= name?

4. Is it safe to use a downgraded version of = btrfs-tools ie: 4.14 or older?

=20 ------=_Part_6654_1787011089.1532811050017-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f196.google.com ([209.85.220.196]:46075 "EHLO mail-qk0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731302AbeG1XCo (ORCPT ); Sat, 28 Jul 2018 19:02:44 -0400 Received: by mail-qk0-f196.google.com with SMTP id c192-v6so5553056qkg.12 for ; Sat, 28 Jul 2018 14:34:51 -0700 (PDT) Date: Sat, 28 Jul 2018 17:34:49 -0400 From: Nicholas D Steeves To: jkexcel Cc: linux-btrfs@vger.kernel.org Subject: Re: btrfs-convert missing in btrfs-tools v4.15.1 Message-ID: <20180728213448.GA16016@DigitalMercury.dynalias.net> References: <1438689914.6655.1532811050018@connect.xfinity.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="YZ5djTAD1cGYuMQK" In-Reply-To: <1438689914.6655.1532811050018@connect.xfinity.com> Sender: linux-btrfs-owner@vger.kernel.org List-ID: --YZ5djTAD1cGYuMQK Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Resending because I forgot to CC list. Hi jkexcel, On 28 July 2018 at 16:50, jkexcel wrote: > > I'm an end user trying to use btrfs-convert but when I installed > btrfs-tools and its dependency btrfs-progs on kubuntu 18.04, the > installation was successful, and it shows that v4.15.1-1build1 was > installed. > > However, when using the command # brtfs-convert /dev/sda4 (with the > drive unmounted) the resulting error appears "command not found" > I also tried command "btrfs convert" in case this was folded into the > main tool, but this also failed. > > 1. Is btrfs-convert still available? > > 2. Where can I find it? > > 3. Has btrfs-convert been replaced? what is it's new name? > > 4. Is it safe to use a downgraded version of btrfs-tools ie: 4.14 or > older? You can blame me for that. In Debian several users had reported release-critical issues in btrfs-convert, so I submitted a patch to disable it for the forseable future, eg: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=864798 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=854489 Also, please consider the official status "As of 4.0 kernels this feature is not often used or well tested anymore, and there have been some reports that the conversion doesn't work reliably. Feel free to try it out, but make sure you have backups" ( https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3 ). I'm happy to hear it is still disabled in Ubuntu, where many more users would be affected. IIRC OpenSUSE LEAP and SLED 15 reenabled it (it was previously disabled there), so maybe it needs specific kernel versions or patches to not trigger RC bugs, and/or very specific btrfs-progs versions, and/or very specific e2fslibs, and/or specific combinations? While I very much look forward to the day when btrfs-convert can be relied on in Debian, I don't think we're there yet. Please take this as an opportunity to test that your backups are restorable, mkfs.btrfs, and then restore from backup. P.S. I have no idea if Ubuntu has additional btrfs support. Cheers, Nicholas --YZ5djTAD1cGYuMQK Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEE4qYmHjkArtfNxmcIWogwR199EGEFAltc4XgACgkQWogwR199 EGGyVhAAxVHKYyDTMjk92lHj7fqQPP0m66PEzzMYa7PPc6hjb0pAkuIo1KqsyzJr cVsgbVHlAuAoIM9De0JWiSz1YMPbgbiNHHfNZuRYb4Qv2DG/wUfp3XEmcvSO8eXo GP+y/f3tqRJ+w++LfNPqEjFFfQGYcw1OA345thfsu1FAVxUU/z6YGjTgkY4jaFCt 9aP1Om5K0ABd/gWcgYdVxf7BuNhGRyQyYLIGOzb+4auFTY/IEou2Bj8+zeZRNyQ7 1P3IB4UwYHIViKlpcUuU+mQAp0kzZtrg6LC/LDPd2q1ttDUvFpsWnbUTuoo3b5RF 6EyYRXy8qaNNmlfjg7YJg5JmPmiwmyH2nmlRX8w+UAb1BIuZm3hZ5ixKJtQVP/+/ rCEFEAVWo9XuypapyNE8uUm2RRvdZrvILUQkKsZ+VDnEeH+6cVfeA4QMCbgNaB2Z 5QRncgwVhljdj4nW7Og9P/xCqdaPr+8BzyhCCaIcai3LPt8RaOF3LWvDT7xvFlhM 0xaG4l3WmqNJ1NEAwd17fmNx3/4mJPJVGdWn7wp/YY74gUm4InHlzPvVcKRYm9/Q mgfeX8vaDQXLK/kobIrWKWcKblSd3zZWrQ36RhdpUavHC05HoVId2LpMucInP3Tt /iON4CpSsK8AFVcdJ3hyRJhykMIS+DDyppntfmzCHyY8TOL9qY0= =T107 -----END PGP SIGNATURE----- --YZ5djTAD1cGYuMQK-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net ([212.227.15.15]:59283 "EHLO mout.gmx.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731734AbeG2BMc (ORCPT ); Sat, 28 Jul 2018 21:12:32 -0400 Subject: Re: btrfs-convert missing in btrfs-tools v4.15.1 To: Nicholas D Steeves , jkexcel Cc: linux-btrfs@vger.kernel.org References: <1438689914.6655.1532811050018@connect.xfinity.com> <20180728213448.GA16016@DigitalMercury.dynalias.net> From: Qu Wenruo Message-ID: <6c37e7ef-5669-7464-c72a-69a2074ebfb1@gmx.com> Date: Sun, 29 Jul 2018 07:44:05 +0800 MIME-Version: 1.0 In-Reply-To: <20180728213448.GA16016@DigitalMercury.dynalias.net> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="1nM0niQEXS38HPmQD4lx8IaQRIE5oTwyH" Sender: linux-btrfs-owner@vger.kernel.org List-ID: This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --1nM0niQEXS38HPmQD4lx8IaQRIE5oTwyH Content-Type: multipart/mixed; boundary="yBdBXgQG2w6Fb3HIvMJ3MV92AhyGcKH9K"; protected-headers="v1" From: Qu Wenruo To: Nicholas D Steeves , jkexcel Cc: linux-btrfs@vger.kernel.org Message-ID: <6c37e7ef-5669-7464-c72a-69a2074ebfb1@gmx.com> Subject: Re: btrfs-convert missing in btrfs-tools v4.15.1 References: <1438689914.6655.1532811050018@connect.xfinity.com> <20180728213448.GA16016@DigitalMercury.dynalias.net> In-Reply-To: <20180728213448.GA16016@DigitalMercury.dynalias.net> --yBdBXgQG2w6Fb3HIvMJ3MV92AhyGcKH9K Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2018=E5=B9=B407=E6=9C=8829=E6=97=A5 05:34, Nicholas D Steeves wrote: > Resending because I forgot to CC list. >=20 > Hi jkexcel, >=20 > On 28 July 2018 at 16:50, jkexcel wrote: >> >> I'm an end user trying to use btrfs-convert but when I installed >> btrfs-tools and its dependency btrfs-progs on kubuntu 18.04, the >> installation was successful, and it shows that v4.15.1-1build1 was >> installed. >> >> However, when using the command # brtfs-convert /dev/sda4 (with the >> drive unmounted) the resulting error appears "command not found" >> I also tried command "btrfs convert" in case this was folded into the >> main tool, but this also failed. >> >> 1. Is btrfs-convert still available? >> >> 2. Where can I find it? >> >> 3. Has btrfs-convert been replaced? what is it's new name? >> >> 4. Is it safe to use a downgraded version of btrfs-tools ie: 4.14 or >> older? >=20 > You can blame me for that. In Debian several users had reported > release-critical issues in btrfs-convert, so I submitted a patch to > disable it for the forseable future, eg: >=20 > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=3D864798 > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=3D854489 Both report looks pretty old (4.7~4.9). In fact, just in v4.10 we had a big rework for convert. It should work much better after that. Furthermore, we have newer (but smaller) fixes to remove a lot of BUG_ON(), and do graceful exit for ENOSPC case since then. And the design of btrfs-convert (at least for the latest btrfs-convert) is to ensure until everything goes well, we won't touch any bytes of the ext* fs (in fact we open the ext* fs in RO mode). So it at least won't corrupt the ext* fs. >=20 > Also, please consider the official status "As of 4.0 kernels this featu= re > is not often used or well tested anymore, and there have been some repo= rts > that the conversion doesn't work reliably. Feel free to try it out, but= > make sure you have backups" ( > https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3 ). The wiki page looks needs to be updated. Both btrfs-convert and base btrfs-progs are improving, especially after v4.10 btrfs-convert goes through a big rework and works well so far, and even added support for reiserfs under the same framwork. So IMHO it's at least worth trying. Thanks, Qu >=20 > I'm happy to hear it is still disabled in Ubuntu, where many more > users would be affected. IIRC OpenSUSE LEAP and SLED 15 reenabled it > (it was previously disabled there), so maybe it needs specific kernel > versions or patches to not trigger RC bugs, and/or very specific > btrfs-progs versions, and/or very specific e2fslibs, and/or specific > combinations? While I very much look forward to the day when > btrfs-convert can be relied on in Debian, I don't think we're there > yet. Please take this as an opportunity to test that your backups are > restorable, mkfs.btrfs, and then restore from backup. P.S. I have no > idea if Ubuntu has additional btrfs support. >=20 > Cheers, > Nicholas >=20 --yBdBXgQG2w6Fb3HIvMJ3MV92AhyGcKH9K-- --1nM0niQEXS38HPmQD4lx8IaQRIE5oTwyH Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEELd9y5aWlW6idqkLhwj2R86El/qgFAltc/8YACgkQwj2R86El /qgsRQf+NZc8/iQguMEsCeECM8zE+uVkIACIwshDammVeuWv0IfaYqzHuS7cbpnE W/0TKYUQ5xdUPhk1fRKBNF8/okFXvzdQFVFDUNaDaQzbYNFiTSbmTsmJT9Hjc/KP d/NK0AweZn3UlaHWjlQ2970whFMoCFDuME6LAk0tGtathxLT6vKmNr+FqeK5oezq bqQpnC9qIFfBbe5tcfOKJJ3kBDtaPk/wDon/mWl8u9QC5/kplfbH0d5mRgLxczVD zMgZjnLHBVsLKt4VictGS7JTgogR6t9lYXgN0wV5jq1LG6Qro816qUimtvBuyXYe V7xK2W9i9EZMmj1fcwZjF35cB8oAKA== =jsYH -----END PGP SIGNATURE----- --1nM0niQEXS38HPmQD4lx8IaQRIE5oTwyH-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-f67.google.com ([209.85.214.67]:32963 "EHLO mail-it0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726955AbeHWVqQ (ORCPT ); Thu, 23 Aug 2018 17:46:16 -0400 Received: by mail-it0-f67.google.com with SMTP id j198-v6so2631827ita.0 for ; Thu, 23 Aug 2018 11:15:22 -0700 (PDT) Date: Thu, 23 Aug 2018 14:15:18 -0400 From: Nicholas D Steeves To: dsterba@suse.cz, jkexcel , linux-btrfs@vger.kernel.org Subject: Re: btrfs-convert missing in btrfs-tools v4.15.1 Message-ID: <20180823181516.4oq4yu7xzzueligq@navis> References: <1438689914.6655.1532811050018@connect.xfinity.com> <20180728213448.GA16016@DigitalMercury.dynalias.net> <20180809115046.GX3218@twin.jikos.cz> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="o2vgc72v7i2vjxie" In-Reply-To: <20180809115046.GX3218@twin.jikos.cz> Sender: linux-btrfs-owner@vger.kernel.org List-ID: --o2vgc72v7i2vjxie Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi everyone, Sorry for the delay replying, I've been busy with other work. Reply follows inline. P.S. sorry about the table in this email that is 99 columns wide. On Thu, Aug 09, 2018 at 01:50:46PM +0200, David Sterba wrote: > On Sat, Jul 28, 2018 at 05:34:49PM -0400, Nicholas D Steeves wrote: > > On 28 July 2018 at 16:50, jkexcel wrote: > > > I'm an end user trying to use btrfs-convert but when I installed > > > btrfs-tools and its dependency btrfs-progs on kubuntu 18.04, the > > > installation was successful, and it shows that v4.15.1-1build1 was > > > installed. > > > > > > However, when using the command # brtfs-convert /dev/sda4 (with the > > > drive unmounted) the resulting error appears "command not found" > > > I also tried command "btrfs convert" in case this was folded into the > > > main tool, but this also failed. > > > > > > 1. Is btrfs-convert still available? > > > > > > 2. Where can I find it? > > > > > > 3. Has btrfs-convert been replaced? what is it's new name? > > > > > > 4. Is it safe to use a downgraded version of btrfs-tools ie: 4.14 or > > > older? > >=20 > > You can blame me for that. In Debian several users had reported > > release-critical issues in btrfs-convert, so I submitted a patch to > > disable it for the forseable future, eg: > >=20 > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=3D864798 > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=3D854489 >=20 > The reports are against version 4.7.3 released one year before the time > of the bug reports. The fix for the reported bug happened in 4.10, that > was half a year before the bug. Debian stable will always have an old version, which will be in use for two to four years. Btrfs-progs 4.7.3 will be in use in Debian 9 until at least 2021, possibly longer. Btw, I strongly believe Debian 9 should have shipped with btrfs-progs 4.9.1, but alas the primary maintainer didn't upload it on time. For "buster" (Debian 10), which will probably be released in mid 2019, the newest possible btrfs-progs version that could be included is whatever is current at the end of January 2019. Exceptions are sometimes granted for an unblock of a newer version. For example, if an LTS kernel won't be released until March, and the release, technical committee, and kernel team decide that's the version we want, then a preapproved exception will be granted. At that time an argument can be made for preapproval of a newer btrfs-progs as well. That said, I try to keep a backported newer version of btrfs-progs for the stable Debian release reasonably up-to-date (backports are available to users on a per-package opt-in basis). That's the channel for feature enablement. Also, my apologies, at the moment this backport is very much out of date--it stalled while investigating which packages would be broken by the library reorganisation; although, from what I can tell that would only be snapper. > > Also, please consider the official status "As of 4.0 kernels this featu= re > > is not often used or well tested anymore, and there have been some repo= rts > > that the conversion doesn't work reliably. Feel free to try it out, but > > make sure you have backups" ( > > https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3 ). >=20 > Sorry that this you take it as official status. The wiki is open to > edits and such claims appear there from time to time. I've removed it, > it's been there since 2015 when it possibly was accurate but it's not > anymore. Is there a more authoritative and up-to-date location for various statuses? It would be nice to have something in btrfs-progs as a table like this, or exportable in some kind of human-friendly format: +-------+-----------------------------------------+------------+-----------= -------+---------------+ |Feature|1st mainline version where feature |LTS kernel 1|LTS kernel = 2 |LTS kernel 3 | | |appeared |eg: 4.4 |eg: 4.9 = |eg: 4.14 | +-------+-----------------------------------------+------------+-----------= -------+---------------+ |convert|Assume -progs and kernel |exp? |mostly? = |stable? | |from |vessions are strongly | | amend = | | |ext3/4 |associated, for simplicity. | | status wi= th: | | | | | |4.9.z:testi= ng | | +-------+-----------------------------------------+------------+-----------= -------+---------------+ |foo |3.16:danger||exp||mostly||testing||stable|exp |mostly = |testing | +-------+-----------------------------------------+------------+-----------= -------+---------------+ Ideally something that could be trivially exported to the format the btrfs.wiki.kernel.org wiki uses. The trick is to make it convenient to maintain... Other than CSV, I'm not sure what would work well for non-emacs users. When the oldest LTS kernel is considered depreciated by the btrfs upstream community, "depreciated" is added to the column header, and when a 4th LTS kernel becomes available the oldest LTS column is dropped and a new empty column is appended. Oh, there could also be a column for current mainline version, but I don't want to maintain that column, and I suspect no one else would either. It would also be nice to add various optimisations to the "Feature" column as they become available. Ideally it would also be nice to have an auto-generated bug matrix, but I digress. If there is not yet a canonical/official table for various btrfs features' statuses then I'd be happy to start work on one--given that the official upstream wiki is nonauthoritative... At the moment 1) I'd use either a table like the one above, something from org-mode, or a CSV. 2) I'd fill it in from the wiki, with versions informed by the bugs I'm aware of 3) I would always err on the side of caution, because I believe that the most important thing for btrfs right now are incident-free success stories, and overly optimistic recommendations will not provide these. 4) I'd submit the patch for discussion and review. > > I'm happy to hear it is still disabled in Ubuntu, where many more > > users would be affected. IIRC OpenSUSE LEAP and SLED 15 reenabled it > > (it was previously disabled there), so maybe it needs specific kernel > > versions or patches to not trigger RC bugs, and/or very specific > > btrfs-progs versions, and/or very specific e2fslibs, and/or specific > > combinations? While I very much look forward to the day when > > btrfs-convert can be relied on in Debian, I don't think we're there > > yet. >=20 > So if there's no way to update package to newer version or nobody who > backports fixes, then it's better not to ship the tool. There's nothing > close to the 'specific version of X', besides using more up to date > versions. Alternatively, there could have been a separate package with > the convert tool, with a warning about the known issues. Briefly, in Debian, backported fixes (to the stable version) are only approved for security and release critical bugs, and never for feature enablement. Supposing btrfs-convert had been its own package, this package would have been dropped from testing if a targeted fix couldn't be found and uploaded within a week or two. During the freeze entry of new upstream versions is blocked, and reentry of dropped packages is also blocked. Once Debian has KVM-isolated machines for running continuous integration, the plan is to enable self-tests in CI. Generally regressions will also block migration of new versions, so that's yet another way Debian ends up with old versions--unless the issues are fixed...but at least this will occur during the general development phase of the release cycle, where there is a lot more time to fix issues :-) > It's in my interest to ship all tools in distros, but there's also only > that much what the upstream community can do. If you're going to > reconsider the status of btrfs-convert in Debian, please let me know. Yes, I'd be happy to advocate for its reinclusion if the answer to 4/5 of the following questions is "yes". Does SUSE now recommend the use of btrfs-convert to its enterprise customers? The following is a frustrating criteria, but: Can a random desktop user run btrfs-convert against their ext4 rootfs and expect the operation to succeed? Is btrfs-convert now sufficiently trusted that it can be recommended with the same degree of confidence as a backup, mkfs.btrfs, then restore to new filesystem approach? Does the user of a btrfs volume created with btrfs-convert have an equal or lesser probability of encountering bugs compared to a one who used mkfs.btrfs? Sincerely, Nicholas --o2vgc72v7i2vjxie Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEE4qYmHjkArtfNxmcIWogwR199EGEFAlt++bEACgkQWogwR199 EGFWWBAAktgorgiYjpf7DqhCYlj9+mGnO5XPQ3xIXbgl4sAN/1pIqjNNqqCvPlV+ l7oQBll2yTLYJ3PzOU8QN4y5RH67+8ZvONlkdPikwa0dFj05WtUqAskqGNazCLIP okjQLS6PRXMqBUn3iFF3dHPcGdiD1wp09WBaQOfuzD+reLoNfaWAqJMguMieEVdj xpJC4FyxtgA/P5CiTXZ9Ztyzmv/GejghJVM/SC6HOaVPi04LIwic9NC0SFOmx3/n OYB5MzNl0gVEdT2DuOH/LNrkw++0vOczRIwHLpgfXuXsAhPJRJEevByFy4p7hbGR YRUJMZVMFw+RH2NeR5OFMtoL4vmA4Y2+VUwSqLweANsaMlgNOWnQkG5W375FX3sx RKI4NKeGP5axEDvRwapA+MbfdfFeGLqLarEBDtZM7ElXr9cOFH7EYGs3RyZRlh30 Ph7o5g1hVF4ffmhnPVIkCg+5ouplphz1pRg13q499/ALQHGHeMq7nhtBcY74foyx NStaP3YhhB2HU4nOByBQ5Hv1J6uBWGJ5HEzWEjpJKR+eEJGDc1fGiwbiiJ7nHmGc ezRa3HQv/Y4yi51ONzOSDpegj6RlfN6aFjGbe3SDB5jYvVeLdiPFzAXSrQdd9tQQ J5gqqLKHb9OPv5EKh8aOXrWpaTEZDWl4PEm0xSKgOnDaIMVEnkk= =TeAC -----END PGP SIGNATURE----- --o2vgc72v7i2vjxie-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-f67.google.com ([209.85.214.67]:50842 "EHLO mail-it0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726780AbeHWV6i (ORCPT ); Thu, 23 Aug 2018 17:58:38 -0400 Received: by mail-it0-f67.google.com with SMTP id j81-v6so8886236ite.0 for ; Thu, 23 Aug 2018 11:27:42 -0700 (PDT) Date: Thu, 23 Aug 2018 14:27:38 -0400 From: Nicholas D Steeves To: Qu Wenruo Cc: jkexcel , linux-btrfs@vger.kernel.org Subject: Re: btrfs-convert missing in btrfs-tools v4.15.1 Message-ID: <20180823182737.g2o7377e7torldtk@navis> References: <1438689914.6655.1532811050018@connect.xfinity.com> <20180728213448.GA16016@DigitalMercury.dynalias.net> <6c37e7ef-5669-7464-c72a-69a2074ebfb1@gmx.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="hplpz22xm64uklka" In-Reply-To: <6c37e7ef-5669-7464-c72a-69a2074ebfb1@gmx.com> Sender: linux-btrfs-owner@vger.kernel.org List-ID: --hplpz22xm64uklka Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Qu, On Sun, Jul 29, 2018 at 07:44:05AM +0800, Qu Wenruo wrote: >=20 >=20 > On 2018=E5=B9=B407=E6=9C=8829=E6=97=A5 05:34, Nicholas D Steeves wrote: > > Resending because I forgot to CC list. > >=20 > > Hi jkexcel, > >=20 > > On 28 July 2018 at 16:50, jkexcel wrote: > >> > >> I'm an end user trying to use btrfs-convert but when I installed > >> btrfs-tools and its dependency btrfs-progs on kubuntu 18.04, the > >> installation was successful, and it shows that v4.15.1-1build1 was > >> installed. > >> > >> However, when using the command # brtfs-convert /dev/sda4 (with the > >> drive unmounted) the resulting error appears "command not found" > >> I also tried command "btrfs convert" in case this was folded into the > >> main tool, but this also failed. > >> > >> 1. Is btrfs-convert still available? > >> > >> 2. Where can I find it? > >> > >> 3. Has btrfs-convert been replaced? what is it's new name? > >> > >> 4. Is it safe to use a downgraded version of btrfs-tools ie: 4.14 or > >> older? > >=20 > > You can blame me for that. In Debian several users had reported > > release-critical issues in btrfs-convert, so I submitted a patch to > > disable it for the forseable future, eg: > >=20 > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=3D864798 > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=3D854489 >=20 > Both report looks pretty old (4.7~4.9). >=20 > In fact, just in v4.10 we had a big rework for convert. > It should work much better after that. >=20 > Furthermore, we have newer (but smaller) fixes to remove a lot of > BUG_ON(), and do graceful exit for ENOSPC case since then. >=20 > And the design of btrfs-convert (at least for the latest btrfs-convert) > is to ensure until everything goes well, we won't touch any bytes of the > ext* fs (in fact we open the ext* fs in RO mode). > So it at least won't corrupt the ext* fs. >=20 > >=20 > > Also, please consider the official status "As of 4.0 kernels this featu= re > > is not often used or well tested anymore, and there have been some repo= rts > > that the conversion doesn't work reliably. Feel free to try it out, but > > make sure you have backups" ( > > https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3 ). >=20 > The wiki page looks needs to be updated. >=20 > Both btrfs-convert and base btrfs-progs are improving, especially after > v4.10 btrfs-convert goes through a big rework and works well so far, and > even added support for reiserfs under the same framwork. >=20 > So IMHO it's at least worth trying. >=20 > Thanks, > Qu Thank you for sharing the cut-off where success became more likely :-) Debian 9 could have had 4.9.1 at the newest, so it wouldn't have had btrfs-convert. So it sounds like btrfs-convert could have been enabled for the experimental suite (which is almost only used by Debian developers and not users) for 4.10. Looking at the changelog it seems we might have had to disable it again before 4.14.1 or 4.16. I'm happy we're having this conversation now, because the time to give it another try is probably sometime in the next month :-) See my long email to David for the caveats. Cheers, Nicholas --hplpz22xm64uklka Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEE4qYmHjkArtfNxmcIWogwR199EGEFAlt+/JcACgkQWogwR199 EGETxw/+O3FeuaH/ofJNIV2JrwV9dMrTwVUicXNwzAJVzKT3ROLLB1Xn2xnx3gNR PoA4vOP1rq0bfDZEOCvi88JCo/uPUCtboo517QmmbxFKLwUcFeuo5XJOq0xoAnVO HS81/GIvjTo3Kk9hPEk8paLzcV5DKC7FcTcm3uYI0PCp7CNEiyfA5yDP4eTui7sL KDQmdOsiP02Na7Xz4+xzaFJuJSoIFoDzvIQCnwm0dWs+lunEmuuDKh6WRkrohyQR bf+D5bBQ1xMd6Ic154gcpLOKqL9rvhejIgGl9d2YLOruavsOAZduxIG1cILiM3oM lZi+GEfKBakwLy+kZfeHS1NMR1T/kOmWSHi1aDmFvjXdQAvN1IyawcxIzcCRVOxZ jnjB3l7ALEDJRM1Iv/L5TruRXBFu0JlttH4br4Wl3EWkOdqIbq5ZjoyuhECtgeoG /7V5qmt4oBMAvFxWAGwJHRnU8yj+yy3nRjNvp3iHJQqlEKcdZs3C8BmR2XYsncvk hXotykOf9Vrr+yy+uglFpT7VJxW71rl7o+4IVGNtOU2T8o8Rb0Kjv5Qe7MS2ieKN qHoyaYCBQ+Dhznug6yO3p0kqLOLyTg8eULuv6if1pPiZ5ZdZXNUX1l0rJAs6Vf9I OCYWiZivHUeoHNW4wt62KlsXtioMWblJbGzmHMDFQbZgSPVfefs= =FFro -----END PGP SIGNATURE----- --hplpz22xm64uklka-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [195.159.176.226] ([195.159.176.226]:59270 "EHLO blaine.gmane.org" rhost-flags-FAIL-FAIL-OK-OK) by vger.kernel.org with ESMTP id S1726338AbeHXIzW (ORCPT ); Fri, 24 Aug 2018 04:55:22 -0400 Received: from list by blaine.gmane.org with local (Exim 4.84_2) (envelope-from ) id 1ft4W3-0005pn-0x for linux-btrfs@vger.kernel.org; Fri, 24 Aug 2018 07:20:11 +0200 To: linux-btrfs@vger.kernel.org From: Duncan <1i5t5.duncan@cox.net> Subject: Re: btrfs-convert missing in btrfs-tools v4.15.1 Date: Fri, 24 Aug 2018 05:20:04 +0000 (UTC) Message-ID: References: <1438689914.6655.1532811050018@connect.xfinity.com> <20180728213448.GA16016@DigitalMercury.dynalias.net> <20180809115046.GX3218@twin.jikos.cz> <20180823181516.4oq4yu7xzzueligq@navis> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Sender: linux-btrfs-owner@vger.kernel.org List-ID: Nicholas D Steeves posted on Thu, 23 Aug 2018 14:15:18 -0400 as excerpted: >> It's in my interest to ship all tools in distros, but there's also only >> that much what the upstream community can do. If you're going to >> reconsider the status of btrfs-convert in Debian, please let me know. > > Yes, I'd be happy to advocate for its reinclusion if the answer to 4/5 > of the following questions is "yes". Does SUSE now recommend the use of > btrfs-convert to its enterprise customers? The following is a > frustrating criteria, but: Can a random desktop user run btrfs-convert > against their ext4 rootfs and expect the operation to succeed? Is > btrfs-convert now sufficiently trusted that it can be recommended with > the same degree of confidence as a backup, mkfs.btrfs, then restore to > new filesystem approach? Does the user of a btrfs volume created with > btrfs-convert have an equal or lesser probability of encountering bugs > compared to a one who used mkfs.btrfs? Just a user and list regular here, and gentoo not debian, but for what it counts... I'd personally never consider or recommend a filesystem converter over the backup, mkfs-to-new-fs, restore-to-new-fs, method, for three reasons. 1) Regardless of how stable a filesystem converter is and what two filesystems the conversion is between, "things" /do/ occasionally happen, thus making it irresponsible to use or recommend use of such a converter without a suitably current and tested backup, "just in case." (This is of course a special case of the sysadmin's first rule of backups, that the true value of data is defined not by any arbitrary claims, but by the number of backups of that data it's considered worth the time/trouble/resources to make/have. If the data value is trivial enough, sure, don't bother with the backup, but if it's of /that/ low a value, so low it's not worth a backup even when doing something as theoretically risky as a filesystem conversion, why is it worth the time and trouble to bother converting it in the first place, instead of just blowing it away and starting clean?) 2) Once a backup is considered "strongly recommended", as we've just established that it should be in 1 regardless of the stability of the converter, just using the existing filesystem as that backup and starting fresh with a mkfs for the new filesystem and copying things over is simply put the easiest, simplest and cleanest method to change filesystems. 3) (Pretty much)[1] Regardless of the filesystems in question, a fresh mkfs and clean sequential transfer of files from the old-fs/backup to the new one is pretty well guaranteed to be better optimized than conversion from an existing filesystem of a different type, particularly one that has been in normal operation for awhile and thus has operational fragmentation of both data and free-space. That's in addition to being less bug-prone, even for a "stable" converter. Restating: So(1) doing a conversion without a backup is irresponsible, (2) the easiest backup and conversion method is directly using the old fs as the backup, and copying over to the freshly mkfs-ed new filesystem, and (3) a freshly mkfs-ed filesystem and sequential copy of files to it from backup, whether that be the old filesystem or not, is going to be more efficient and less bug-prone than an in-place conversion. Given the above, why would /anyone/ /sane/ consider using a converter? It simply doesn't make sense, even if the converter were as stable as the most stable filesystems we have. So as a distro btrfs package maintainer, do what you wish in terms of the converter, but were it me, I might actually consider replacing it with an executable that simply printed out some form of the above argument, with a pointer to the sources should they still be interested after having read that argument.[2] Then, if people really are determined to unnecessarily waste their time to get a less efficient filesystem, possibly risking their data in the process of getting it, they can always build the converter from sources themselves. --- [1] I debated omitting the qualifier as I know of no exceptions, but I'm not a filesystem expert and while I'm a bit skeptical, I suppose it's possible that they might exist. [2] There's actually btrfs precedent for this in the form of the executable built as fsck.btrfs, which does nothing (successfully) but possibly print a message referring people to btrfs check, if run in interactive mode. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx2.suse.de ([195.135.220.15]:41234 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1730090AbeHIOPZ (ORCPT ); Thu, 9 Aug 2018 10:15:25 -0400 Date: Thu, 9 Aug 2018 13:50:46 +0200 From: David Sterba To: Nicholas D Steeves Cc: jkexcel , linux-btrfs@vger.kernel.org Subject: Re: btrfs-convert missing in btrfs-tools v4.15.1 Message-ID: <20180809115046.GX3218@twin.jikos.cz> Reply-To: dsterba@suse.cz References: <1438689914.6655.1532811050018@connect.xfinity.com> <20180728213448.GA16016@DigitalMercury.dynalias.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii In-Reply-To: <20180728213448.GA16016@DigitalMercury.dynalias.net> Sender: linux-btrfs-owner@vger.kernel.org List-ID: On Sat, Jul 28, 2018 at 05:34:49PM -0400, Nicholas D Steeves wrote: > On 28 July 2018 at 16:50, jkexcel wrote: > > I'm an end user trying to use btrfs-convert but when I installed > > btrfs-tools and its dependency btrfs-progs on kubuntu 18.04, the > > installation was successful, and it shows that v4.15.1-1build1 was > > installed. > > > > However, when using the command # brtfs-convert /dev/sda4 (with the > > drive unmounted) the resulting error appears "command not found" > > I also tried command "btrfs convert" in case this was folded into the > > main tool, but this also failed. > > > > 1. Is btrfs-convert still available? > > > > 2. Where can I find it? > > > > 3. Has btrfs-convert been replaced? what is it's new name? > > > > 4. Is it safe to use a downgraded version of btrfs-tools ie: 4.14 or > > older? > > You can blame me for that. In Debian several users had reported > release-critical issues in btrfs-convert, so I submitted a patch to > disable it for the forseable future, eg: > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=864798 > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=854489 The reports are against version 4.7.3 released one year before the time of the bug reports. The fix for the reported bug happened in 4.10, that was half a year before the bug. > Also, please consider the official status "As of 4.0 kernels this feature > is not often used or well tested anymore, and there have been some reports > that the conversion doesn't work reliably. Feel free to try it out, but > make sure you have backups" ( > https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3 ). Sorry that this you take it as official status. The wiki is open to edits and such claims appear there from time to time. I've removed it, it's been there since 2015 when it possibly was accurate but it's not anymore. > I'm happy to hear it is still disabled in Ubuntu, where many more > users would be affected. IIRC OpenSUSE LEAP and SLED 15 reenabled it > (it was previously disabled there), so maybe it needs specific kernel > versions or patches to not trigger RC bugs, and/or very specific > btrfs-progs versions, and/or very specific e2fslibs, and/or specific > combinations? While I very much look forward to the day when > btrfs-convert can be relied on in Debian, I don't think we're there > yet. So if there's no way to update package to newer version or nobody who backports fixes, then it's better not to ship the tool. There's nothing close to the 'specific version of X', besides using more up to date versions. Alternatively, there could have been a separate package with the convert tool, with a warning about the known issues. It's in my interest to ship all tools in distros, but there's also only that much what the upstream community can do. If you're going to reconsider the status of btrfs-convert in Debian, please let me know.