* Encryption @ 2003-03-22 23:22 Pierre Abbat 2003-03-22 23:29 ` Encryption Yury Umanets 2003-03-23 3:38 ` Snapshots a la NetApp? kend 0 siblings, 2 replies; 33+ messages in thread From: Pierre Abbat @ 2003-03-22 23:22 UTC (permalink / raw) To: reiserfs-list How is filesystem encryption going to work? Or has anyone figured that out yet? phma -- .i toljundi do .ibabo mi'afra tu'a do .ibabo damba do .ibabo do jinga .icu'u la ma'atman. ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2003-03-22 23:22 Encryption Pierre Abbat @ 2003-03-22 23:29 ` Yury Umanets 2003-03-24 20:37 ` Encryption Hans Reiser 2003-03-23 3:38 ` Snapshots a la NetApp? kend 1 sibling, 1 reply; 33+ messages in thread From: Yury Umanets @ 2003-03-22 23:29 UTC (permalink / raw) To: Pierre Abbat; +Cc: reiserfs-list Pierre Abbat wrote: >How is filesystem encryption going to work? Or has anyone figured that out >yet? > >phma > > Don't worry, it is going up :) -- Yury Umanets ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2003-03-22 23:29 ` Encryption Yury Umanets @ 2003-03-24 20:37 ` Hans Reiser 2003-03-25 19:18 ` Encryption Edward Shushkin 0 siblings, 1 reply; 33+ messages in thread From: Hans Reiser @ 2003-03-24 20:37 UTC (permalink / raw) To: Yury Umanets; +Cc: Pierre Abbat, reiserfs-list, Edward Shishkin Yury Umanets wrote: > Pierre Abbat wrote: > >> How is filesystem encryption going to work? Or has anyone figured >> that out yet? >> >> phma >> >> > Don't worry, it is going up :) > Edward, please provide a serious answer including your design documents, so that the list can review your design and comment. Chances are that they will make at least one serious improvement to the design..... and it might be nice to get that improvement made before you have made the time investment required to debug..... -- Hans ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2003-03-24 20:37 ` Encryption Hans Reiser @ 2003-03-25 19:18 ` Edward Shushkin 0 siblings, 0 replies; 33+ messages in thread From: Edward Shushkin @ 2003-03-25 19:18 UTC (permalink / raw) To: Hans Reiser; +Cc: Yury Umanets, Pierre Abbat, reiserfs-list Hans Reiser wrote: > > Yury Umanets wrote: > > > Pierre Abbat wrote: > > > >> How is filesystem encryption going to work? Or has anyone figured > >> that out yet? > >> > >> phma > >> > >> > > Don't worry, it is going up :) > > > Edward, please provide a serious answer including your design documents, > so that the list can review your design and comment. > Chances are that > they will make at least one serious improvement to the design... Ok, it will be very nice. Edward. .. and > it might be nice to get that improvement made before you have made the > time investment required to debug..... > > -- > Hans ^ permalink raw reply [flat|nested] 33+ messages in thread
* Snapshots a la NetApp? 2003-03-22 23:22 Encryption Pierre Abbat 2003-03-22 23:29 ` Encryption Yury Umanets @ 2003-03-23 3:38 ` kend 2003-03-23 9:16 ` Heinz-Josef Claes ` (2 more replies) 1 sibling, 3 replies; 33+ messages in thread From: kend @ 2003-03-23 3:38 UTC (permalink / raw) To: reiserfs-list I'm just wondering if there's any development being done on NetApp-like snapshots. (This differs from LVM snapshots in that it's file-by-file, instead of volume based.) If not, has anyone given it any consideration? It would be a huge win for the RAID-in-a-box folks, and, speaking as a sysadmin, is something about which I dream frequently. Just curious... Ken D'Ambrosio Sr. SysAdmin, Xanoptix, Inc. ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Snapshots a la NetApp? 2003-03-23 3:38 ` Snapshots a la NetApp? kend @ 2003-03-23 9:16 ` Heinz-Josef Claes 2003-03-24 20:49 ` Hans Reiser 2003-03-23 11:18 ` Snapshots a la NetApp? Lars Marowsky-Bree 2003-03-23 12:44 ` Oleg Drokin 2 siblings, 1 reply; 33+ messages in thread From: Heinz-Josef Claes @ 2003-03-23 9:16 UTC (permalink / raw) To: kend; +Cc: reiserfs-list Am Son, 2003-03-23 um 04.38 schrieb kend@xanoptix.com: > I'm just wondering if there's any development being done on NetApp-like > snapshots. (This differs from LVM snapshots in that it's file-by-file, > instead of volume based.) If not, has anyone given it any consideration? > It would be a huge win for the RAID-in-a-box folks, and, speaking as a > sysadmin, is something about which I dream frequently. > > Just curious... > > Ken D'Ambrosio > Sr. SysAdmin, > Xanoptix, Inc. Hi, you can look at the URL below. It's a kind of snapshot inspired by NetApp, but has nothing to do with LVM or the filesystem. It also has a different behaviour. -- Heinz-Josef Claes hjclaes@web.de project: http://sourceforge.net/projects/storebackup -> snapshot-like backup to another disk ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Snapshots a la NetApp? 2003-03-23 9:16 ` Heinz-Josef Claes @ 2003-03-24 20:49 ` Hans Reiser 2003-03-24 21:42 ` Heinz-Josef Claes 2003-03-25 6:15 ` unsubscribe Voicu Liviu 0 siblings, 2 replies; 33+ messages in thread From: Hans Reiser @ 2003-03-24 20:49 UTC (permalink / raw) To: Heinz-Josef Claes; +Cc: kend, reiserfs-list Heinz-Josef Claes wrote: >Am Son, 2003-03-23 um 04.38 schrieb kend@xanoptix.com: > > >>I'm just wondering if there's any development being done on NetApp-like >>snapshots. (This differs from LVM snapshots in that it's file-by-file, >>instead of volume based.) If not, has anyone given it any consideration? >>It would be a huge win for the RAID-in-a-box folks, and, speaking as a >>sysadmin, is something about which I dream frequently. >> >>Just curious... >> >>Ken D'Ambrosio >>Sr. SysAdmin, >>Xanoptix, Inc. >> >> > >Hi, >you can look at the URL below. It's a kind of snapshot inspired by >NetApp, but has nothing to do with LVM or the filesystem. It also has a >different behaviour. > > > I didn't find the magic click that gave me the design doc for storebackup, could you post it to the list? -- Hans ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Snapshots a la NetApp? 2003-03-24 20:49 ` Hans Reiser @ 2003-03-24 21:42 ` Heinz-Josef Claes 2003-03-25 0:03 ` Tom Vier 2003-03-25 2:35 ` Hans Reiser 2003-03-25 6:15 ` unsubscribe Voicu Liviu 1 sibling, 2 replies; 33+ messages in thread From: Heinz-Josef Claes @ 2003-03-24 21:42 UTC (permalink / raw) To: Hans Reiser; +Cc: kend, reiserfs-list Am Mon, 2003-03-24 um 21.49 schrieb Hans Reiser: > Heinz-Josef Claes wrote: > > >Am Son, 2003-03-23 um 04.38 schrieb kend@xanoptix.com: > > > > > >>I'm just wondering if there's any development being done on NetApp-like > >>snapshots. (This differs from LVM snapshots in that it's file-by-file, > >>instead of volume based.) If not, has anyone given it any consideration? > >>It would be a huge win for the RAID-in-a-box folks, and, speaking as a > >>sysadmin, is something about which I dream frequently. > >> > >>Just curious... > >> > >>Ken D'Ambrosio > >>Sr. SysAdmin, > >>Xanoptix, Inc. > >> > >> > > > >Hi, > >you can look at the URL below. It's a kind of snapshot inspired by > >NetApp, but has nothing to do with LVM or the filesystem. It also has a > >different behaviour. > > > > > > > I didn't find the magic click that gave me the design doc for > storebackup, could you post it to the list? There is no design doc, because it's principal design is so simple :-). Here is an excerpt from the README file include in the tar file at sourceforge: GENERAL FUNCTIONALITY --------------------- storeBackup is a disk-to-disk backup tool for Linux. It should run on other Unix like machines. You can directly browse through the backuped files (locally, via NFS, via SAMBA or whatever). This gives the users the possibility to restore files absolutely easily and fast. He/She only has to copy (and possibly uncompress) the file. There is also a tool for easily restoring (sub) trees for the administrator. Every single backup of a specific time can be deleted without affecting the other existing backups. HOW DOES IT WORK? ----------------- storeBackup makes a backup of a directory to another direct reachable location. It does not care about where this location is (same disk, another disc, via NFS over the network). You should use another disk or better another computer to store the backup. The target directory must be on a Unix virtual file system which supports hard links. So backing up to a SAMBA share is not possible. Naturally, you can also mount the source directory via NFS and backup in a local filesystem. In this case, it's good to have a fast network. The backup(s) can be seen in a directory in the form date_time (yyyy.mm.dd_hh.mm.ss) which it creates. Implemented are several optimizations to reduce disk usage: - The files to backup are compressed (default bzip2) as discrete files in the backup. Files with definable suffixes (like .gz, which is part of the default value) will not be compressed. It is possible to avoid compression in general. - If a file with the same contents exists in the previous backup, the new backup will only be a hard link to the other one. (This mechanism depends on the contents, not on a file name or path!) If you rename a file or directory or move sub trees around, it will not cost additional space in the backup. - You can also check older backups than the last one for files with the same contents. But this is normally not worth the effort. You can also check backups of *other* machines (or backup series) for files with the same contents, which can be very efficient. - If a file with the same contents exists in the same tree to back up, it will be hard linked to the other one (and naturally to the older ones). As a result, only changes resulting in different files were stored (compressed) and require disk space. Normally, the required disk space for one backup a day for 30 days is less then the required space for the original. But this depends on the number of changes. There are several optimizations to improve performance. Normally, the first backup is much slower than the followings, because all the data has to be compressed and/or copied. storeBackup is able to take advantage of multiprocessor machines. From the second run, it should not be a problem to get more than 100 files/sec with a normal machine of today. This does not depend on the file size. There is are special files in the root of the created backup called .md5CheckSums.info and .md5CheckSums or .md5CheckSums.bz2 (default). You must not delete these files. They contain all information about the original files. You can use this information to write you own tools beside the existing to restore or to analyze the backups. When started, storeBackup will read .md5CheckSums and creates its own databases (dbm file) in $TMPDIR or --tmpdir (default is /tmp). If you back up a large number of files (some millions), the required space can be several dozens of megabytes. If you do not have enough memory to cache the dbm file, I recommend using a separate hard disk (if available) for better performance. LIMITATIONS ----------- - storeBackup can backup normal files, directories, symbolic links and named pipes. Other file types are not supported up to now and will generate a warning. - The permissions in the backup tree(s) are equal to the permissions in the original directory. Under special rare conditions it is possible, that a user cannot read one ore more of own his/her files in the backup. With the restore tool - storeBackupRecover.pl - everything is restored with the original permissions. - storeBackup uses hard links to save disk space. Linux with ext2 file system supports up to 32000, reiserfs up to 64535 hard links. If storeBackup needs more hard links, it will write a warning and store a new (compressed) copy of the file. If you use ext2 for the backup, you have to reserve enough (static) inodes! Hope this helps. As I wrote, it's not depending on the filesystem. It's depending on files, while NetApp is depending on blocks. Making a snapshot of a big database file with NetApp is a good idea. Making backups of big database files with storebackup costs you a lot of space. On the other side, if you change a normal text file, the block related algorithm of NetApp will not save much blocks. Compressing the whole file is much more efficient. And duplicated files, with exist in a great number in a "normal" file system containing the data of many users (I also didn't believe this :-)) are saved only once in the backup. If you want to backup big DB files on linux, there's a better tool for this use case which uses the algorithm of rsync and stores the original file and a series of deltas. If I remember well it's called rdiff. (Sorry, but I never used it.) -- Heinz-Josef Claes hjclaes@web.de project: http://sourceforge.net/projects/storebackup -> snapshot-like backup to another disk ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Snapshots a la NetApp? 2003-03-24 21:42 ` Heinz-Josef Claes @ 2003-03-25 0:03 ` Tom Vier 2003-03-26 21:11 ` Heinz-Josef Claes 2003-03-25 2:35 ` Hans Reiser 1 sibling, 1 reply; 33+ messages in thread From: Tom Vier @ 2003-03-25 0:03 UTC (permalink / raw) To: reiserfs-list fwiw, there's pdumpfs, which does much the same thing. it's much like plan9's dump (which writes snapshots to worm media). pdumpfs has worked great for me. i just use it for my home directory. note that many programs don't handle hardlinks well. specifically tar, cpio, and rsync (the last two use TONS of memory even if you have just a few months worth of snapshots). -- Tom Vier <tmv@comcast.net> DSA Key ID 0xE6CB97DA ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Snapshots a la NetApp? 2003-03-25 0:03 ` Tom Vier @ 2003-03-26 21:11 ` Heinz-Josef Claes 0 siblings, 0 replies; 33+ messages in thread From: Heinz-Josef Claes @ 2003-03-26 21:11 UTC (permalink / raw) To: Tom Vier; +Cc: reiserfs-list Am Die, 2003-03-25 um 01.03 schrieb Tom Vier: > fwiw, there's pdumpfs, which does much the same thing. it's much like > plan9's dump (which writes snapshots to worm media). pdumpfs has worked > great for me. i just use it for my home directory. note that many programs > don't handle hardlinks well. specifically tar, cpio, and rsync (the last two > use TONS of memory even if you have just a few months worth of snapshots). Thanks for the hint - I've never heard before about pdumpfs. I wrote storeBackup because I could not find such a tool. A very basic idea of both progs are the same: to use hard links. But the implementation differs. The difference is, that pdumpfs depends on paths and file names. StoreBackup depends on the *contents* of the files, compresses optional and can share files via hard links between independent backups of different trees (computers). The parameters are: storeBackup.pl -f configFile [-g | --print] or storeBackup.pl -s sourceDir -t targetDir [-T tmpdir] [-L lockFile] [--exceptDirs dir1,dir2,dir3] [--exceptDirsSep sep] [--precommand job] [--followLinks depth] [-c compress] [-u uncompress] [-p postfix] [--noCompress number] [--queueCompress number] [--noCopy number] [--queueCopy number] [--withUserGroupStat] [--userGroupStatFile filename] [--exceptSuffix suffixes] [--addExceptSuffix suffixes] [--contExceptDirsErr] [--compressMD5File yes|no] [--chmodMD5File] [-v] [-d level] [--keepAll timescale] [--keepWeekday entry] [--keepMinNumber number] [--keepOnlyLastOfDay timescale] [--nodelete] [--progressReport number] [--printDepth] [--ignoreReadError] [-l logFile [--plusLogStdout] [--withTime yes|no] [-m maxFilelen] [[-n noOfOldFiles] | [--saveLogs yes|no] [--compressWith compressprog]] [--logInBackupDir yes|no [--compressLogInBackupDir yes|no] [--logInBackupDirFileName logFile]] [otherBackupDirs ...] best regards, -- Heinz-Josef Claes hjclaes@web.de project: http://sourceforge.net/projects/storebackup -> snapshot-like backup to another disk ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Snapshots a la NetApp? 2003-03-24 21:42 ` Heinz-Josef Claes 2003-03-25 0:03 ` Tom Vier @ 2003-03-25 2:35 ` Hans Reiser 2003-03-26 14:15 ` Heinz-Josef Claes 1 sibling, 1 reply; 33+ messages in thread From: Hans Reiser @ 2003-03-25 2:35 UTC (permalink / raw) To: Heinz-Josef Claes; +Cc: kend, reiserfs-list, Alexander Lyamin Heinz-Josef Claes wrote: >Am Mon, 2003-03-24 um 21.49 schrieb Hans Reiser: > > >>Heinz-Josef Claes wrote: >> >> >> >>>Am Son, 2003-03-23 um 04.38 schrieb kend@xanoptix.com: >>> >>> >>> >>> >>>>I'm just wondering if there's any development being done on NetApp-like >>>>snapshots. (This differs from LVM snapshots in that it's file-by-file, >>>>instead of volume based.) If not, has anyone given it any consideration? >>>>It would be a huge win for the RAID-in-a-box folks, and, speaking as a >>>>sysadmin, is something about which I dream frequently. >>>> >>>>Just curious... >>>> >>>>Ken D'Ambrosio >>>>Sr. SysAdmin, >>>>Xanoptix, Inc. >>>> >>>> >>>> >>>> >>>Hi, >>>you can look at the URL below. It's a kind of snapshot inspired by >>>NetApp, but has nothing to do with LVM or the filesystem. It also has a >>>different behaviour. >>> >>> >>> >>> >>> >>I didn't find the magic click that gave me the design doc for >>storebackup, could you post it to the list? >> >> > >There is no design doc, because it's principal design is so simple :-). >Here is an excerpt from the README file include in the tar file at >sourceforge: > >GENERAL FUNCTIONALITY >--------------------- > >storeBackup is a disk-to-disk backup tool for Linux. It should run on >other Unix like machines. You can directly browse through the backuped >files (locally, via NFS, via SAMBA or whatever). This gives the users >the possibility to restore files absolutely easily and fast. He/She >only has to copy (and possibly uncompress) the file. There is also a >tool for easily restoring (sub) trees for the administrator. Every >single backup of a specific time can be deleted without affecting the >other existing backups. > > >HOW DOES IT WORK? >----------------- > >storeBackup makes a backup of a directory to another direct reachable >location. It does not care about where this location is (same disk, >another disc, via NFS over the network). You should use another disk >or better another computer to store the backup. The target directory >must be on a Unix virtual file system which supports hard links. So >backing up to a SAMBA share is not possible. Naturally, you can also >mount the source directory via NFS and backup in a local >filesystem. In this case, it's good to have a fast network. The >backup(s) can be seen in a directory in the form date_time >(yyyy.mm.dd_hh.mm.ss) which it creates. > >Implemented are several optimizations to reduce disk usage: > >- The files to backup are compressed (default bzip2) as discrete files > in the backup. Files with definable suffixes (like .gz, which is part > of the default value) will not be compressed. It is possible to > avoid compression in general. > >- If a file with the same contents exists in the previous backup, the > new backup will only be a hard link to the other one. (This > mechanism depends on the contents, not on a file name or path!) If you > rename a file or directory or move sub trees around, it will not cost > additional space in the backup. > >- You can also check older backups than the last one for files with > the same contents. But this is normally not worth the effort. You > can also check backups of *other* machines (or backup series) > for files with the same contents, which can be very efficient. > >- If a file with the same contents exists in the same tree to back up, > it will be hard linked to the other one (and naturally to the older > ones). > >As a result, only changes resulting in different files were stored >(compressed) and require disk space. Normally, the required disk space >for one backup a day for 30 days is less then the required space for the >original. But this depends on the number of changes. > > >There are several optimizations to improve performance. Normally, the >first backup is much slower than the followings, because all the data >has to be compressed and/or copied. storeBackup is able to take >advantage of multiprocessor machines. From the second run, it should not >be a problem to get more than 100 files/sec with a normal machine of >today. This does not depend on the file size. > >There is are special files in the root of the created backup called >.md5CheckSums.info and .md5CheckSums or .md5CheckSums.bz2 >(default). You must not delete these files. They contain all >information about the original files. You can use this information to >write you own tools beside the existing to restore or to analyze the >backups. > >When started, storeBackup will read .md5CheckSums and creates its own >databases (dbm file) in $TMPDIR or --tmpdir (default is /tmp). If you >back up a large number of files (some millions), the required space can >be several dozens of megabytes. If you do not have enough memory to >cache the dbm file, I recommend using a separate hard disk (if >available) for better performance. > > >LIMITATIONS >----------- > >- storeBackup can backup normal files, directories, symbolic links and > named pipes. Other file types are not supported up to now and will > generate a warning. > >- The permissions in the backup tree(s) are equal to the permissions > in the original directory. Under special rare conditions it is > possible, that a user cannot read one ore more of own his/her files > in the backup. With the restore tool - storeBackupRecover.pl - > everything is restored with the original permissions. > >- storeBackup uses hard links to save disk space. Linux with ext2 file > system supports up to 32000, reiserfs up to 64535 hard links. If > storeBackup needs more hard links, it will write a warning and store > a new (compressed) copy of the file. If you use ext2 for the backup, > you have to reserve enough (static) inodes! > > > >Hope this helps. As I wrote, it's not depending on the filesystem. It's >depending on files, while NetApp is depending on blocks. Making a >snapshot of a big database file with NetApp is a good idea. Making >backups of big database files with storebackup costs you a lot of space. > >On the other side, if you change a normal text file, the block related >algorithm of NetApp will not save much blocks. Compressing the whole >file is much more efficient. And duplicated files, with exist in a great >number in a "normal" file system containing the data of many users (I >also didn't believe this :-)) are saved only once in the backup. > >If you want to backup big DB files on linux, there's a better tool for >this use case which uses the algorithm of rsync and stores the original >file and a series of deltas. If I remember well it's called rdiff. >(Sorry, but I never used it.) > > > Flx, should we try this for our backups? -- Hans ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Snapshots a la NetApp? 2003-03-25 2:35 ` Hans Reiser @ 2003-03-26 14:15 ` Heinz-Josef Claes 2003-03-26 18:37 ` Hans Reiser 0 siblings, 1 reply; 33+ messages in thread From: Heinz-Josef Claes @ 2003-03-26 14:15 UTC (permalink / raw) To: Hans Reiser; +Cc: kend, reiserfs-list, Alexander Lyamin Am Die, 2003-03-25 um 03.35 schrieb Hans Reiser: > Heinz-Josef Claes wrote: > > >Am Mon, 2003-03-24 um 21.49 schrieb Hans Reiser: > > > > > >>Heinz-Josef Claes wrote: > >> > >> > >> > >>>Am Son, 2003-03-23 um 04.38 schrieb kend@xanoptix.com: > >>> > >>> > >>> > >>> > >>>>I'm just wondering if there's any development being done on NetApp-like > >>>>snapshots. (This differs from LVM snapshots in that it's file-by-file, > >>>>instead of volume based.) If not, has anyone given it any consideration? > >>>>It would be a huge win for the RAID-in-a-box folks, and, speaking as a > >>>>sysadmin, is something about which I dream frequently. > >>>> > >>>>Just curious... > >>>> > >>>>Ken D'Ambrosio > >>>>Sr. SysAdmin, > >>>>Xanoptix, Inc. > >>>> > >>>> > >>>> > >>>> > >>>Hi, > >>>you can look at the URL below. It's a kind of snapshot inspired by > >>>NetApp, but has nothing to do with LVM or the filesystem. It also has a > >>>different behaviour. > >>> > >>> > >>> > >>> > >>> > >>I didn't find the magic click that gave me the design doc for > >>storebackup, could you post it to the list? > >> > >> > > > >There is no design doc, because it's principal design is so simple :-). > >Here is an excerpt from the README file include in the tar file at > >sourceforge: > > > >GENERAL FUNCTIONALITY > >--------------------- > > > >storeBackup is a disk-to-disk backup tool for Linux. It should run on > >other Unix like machines. You can directly browse through the backuped > >files (locally, via NFS, via SAMBA or whatever). This gives the users > >the possibility to restore files absolutely easily and fast. He/She > >only has to copy (and possibly uncompress) the file. There is also a > >tool for easily restoring (sub) trees for the administrator. Every > >single backup of a specific time can be deleted without affecting the > >other existing backups. > > > > > >HOW DOES IT WORK? > >----------------- > > > >storeBackup makes a backup of a directory to another direct reachable > >location. It does not care about where this location is (same disk, > >another disc, via NFS over the network). You should use another disk > >or better another computer to store the backup. The target directory > >must be on a Unix virtual file system which supports hard links. So > >backing up to a SAMBA share is not possible. Naturally, you can also > >mount the source directory via NFS and backup in a local > >filesystem. In this case, it's good to have a fast network. The > >backup(s) can be seen in a directory in the form date_time > >(yyyy.mm.dd_hh.mm.ss) which it creates. > > > >Implemented are several optimizations to reduce disk usage: > > > >- The files to backup are compressed (default bzip2) as discrete files > > in the backup. Files with definable suffixes (like .gz, which is part > > of the default value) will not be compressed. It is possible to > > avoid compression in general. > > > >- If a file with the same contents exists in the previous backup, the > > new backup will only be a hard link to the other one. (This > > mechanism depends on the contents, not on a file name or path!) If you > > rename a file or directory or move sub trees around, it will not cost > > additional space in the backup. > > > >- You can also check older backups than the last one for files with > > the same contents. But this is normally not worth the effort. You > > can also check backups of *other* machines (or backup series) > > for files with the same contents, which can be very efficient. > > > >- If a file with the same contents exists in the same tree to back up, > > it will be hard linked to the other one (and naturally to the older > > ones). > > > >As a result, only changes resulting in different files were stored > >(compressed) and require disk space. Normally, the required disk space > >for one backup a day for 30 days is less then the required space for the > >original. But this depends on the number of changes. > > > > > >There are several optimizations to improve performance. Normally, the > >first backup is much slower than the followings, because all the data > >has to be compressed and/or copied. storeBackup is able to take > >advantage of multiprocessor machines. From the second run, it should not > >be a problem to get more than 100 files/sec with a normal machine of > >today. This does not depend on the file size. > > > >There is are special files in the root of the created backup called > >.md5CheckSums.info and .md5CheckSums or .md5CheckSums.bz2 > >(default). You must not delete these files. They contain all > >information about the original files. You can use this information to > >write you own tools beside the existing to restore or to analyze the > >backups. > > > >When started, storeBackup will read .md5CheckSums and creates its own > >databases (dbm file) in $TMPDIR or --tmpdir (default is /tmp). If you > >back up a large number of files (some millions), the required space can > >be several dozens of megabytes. If you do not have enough memory to > >cache the dbm file, I recommend using a separate hard disk (if > >available) for better performance. > > > > > >LIMITATIONS > >----------- > > > >- storeBackup can backup normal files, directories, symbolic links and > > named pipes. Other file types are not supported up to now and will > > generate a warning. > > > >- The permissions in the backup tree(s) are equal to the permissions > > in the original directory. Under special rare conditions it is > > possible, that a user cannot read one ore more of own his/her files > > in the backup. With the restore tool - storeBackupRecover.pl - > > everything is restored with the original permissions. > > > >- storeBackup uses hard links to save disk space. Linux with ext2 file > > system supports up to 32000, reiserfs up to 64535 hard links. If > > storeBackup needs more hard links, it will write a warning and store > > a new (compressed) copy of the file. If you use ext2 for the backup, > > you have to reserve enough (static) inodes! > > > > > > > >Hope this helps. As I wrote, it's not depending on the filesystem. It's > >depending on files, while NetApp is depending on blocks. Making a > >snapshot of a big database file with NetApp is a good idea. Making > >backups of big database files with storebackup costs you a lot of space. > > > >On the other side, if you change a normal text file, the block related > >algorithm of NetApp will not save much blocks. Compressing the whole > >file is much more efficient. And duplicated files, with exist in a great > >number in a "normal" file system containing the data of many users (I > >also didn't believe this :-)) are saved only once in the backup. > > > >If you want to backup big DB files on linux, there's a better tool for > >this use case which uses the algorithm of rsync and stores the original > >file and a series of deltas. If I remember well it's called rdiff. > >(Sorry, but I never used it.) > > > > > > > Flx, should we try this for our backups? If you test it, I would be glad to hear about your experiences, because I believe that you must be a stress tester by birth :-) btw: The only handycap of the used method is that all the links to a file can have only *one* set of permissions (chmod, chown). Will it be possible to change this with a loadable module in reiser4? best regards, HJC ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Snapshots a la NetApp? 2003-03-26 14:15 ` Heinz-Josef Claes @ 2003-03-26 18:37 ` Hans Reiser 2003-03-26 22:45 ` Valdis.Kletnieks 0 siblings, 1 reply; 33+ messages in thread From: Hans Reiser @ 2003-03-26 18:37 UTC (permalink / raw) To: Heinz-Josef Claes; +Cc: kend, reiserfs-list, Alexander Lyamin Heinz-Josef Claes wrote: > >btw: The only handycap of the used method is that all the links to a >file can have only *one* set of permissions (chmod, chown). Will it be >possible to change this with a loadable module in reiser4? > >best regards, >HJC > > > > > I am not sure if we will support that. If someone wanted to write the code to do it, then I could examine the problem more closely and it could be done..... -- Hans ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Snapshots a la NetApp? 2003-03-26 18:37 ` Hans Reiser @ 2003-03-26 22:45 ` Valdis.Kletnieks 0 siblings, 0 replies; 33+ messages in thread From: Valdis.Kletnieks @ 2003-03-26 22:45 UTC (permalink / raw) To: Hans Reiser; +Cc: reiserfs-list [-- Attachment #1: Type: text/plain, Size: 1107 bytes --] On Wed, 26 Mar 2003 21:37:10 +0300, Hans Reiser said: > Heinz-Josef Claes wrote: > >btw: The only handycap of the used method is that all the links to a > >file can have only *one* set of permissions (chmod, chown). Will it be > >possible to change this with a loadable module in reiser4? > I am not sure if we will support that. If someone wanted to write the > code to do it, then I could examine the problem more closely and it > could be done..... Please don't. There is just *WAY* too much Unix/Linux semantics that are dependent on the concept that an inode describes a file, with all it implies - the fact that all hard links have the same owner/perms (hint - how would you store 2 inodes that had different owner/perm/mtime/ctime/etc but the same block list, and still have it fsck correctly? If you have a linked list of owners/perms, how do you determine which one to use? etc etc..) I'm not even going into the aspects like programs that creat() and then unlink() in order to get a anonymous temp file that goes away at last close, etc etc.... You want different permissions, use ACLs. [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: unsubscribe 2003-03-24 20:49 ` Hans Reiser 2003-03-24 21:42 ` Heinz-Josef Claes @ 2003-03-25 6:15 ` Voicu Liviu 1 sibling, 0 replies; 33+ messages in thread From: Voicu Liviu @ 2003-03-25 6:15 UTC (permalink / raw) To: Hans Reiser, Heinz-Josef Claes; +Cc: kend, reiserfs-list unsubscribe ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Snapshots a la NetApp? 2003-03-23 3:38 ` Snapshots a la NetApp? kend 2003-03-23 9:16 ` Heinz-Josef Claes @ 2003-03-23 11:18 ` Lars Marowsky-Bree 2003-03-24 12:49 ` Ragnar Kjørstad 2003-03-23 12:44 ` Oleg Drokin 2 siblings, 1 reply; 33+ messages in thread From: Lars Marowsky-Bree @ 2003-03-23 11:18 UTC (permalink / raw) To: kend, reiserfs-list On 2003-03-22T22:38:01, kend@xanoptix.com said: > I'm just wondering if there's any development being done on NetApp-like > snapshots. (This differs from LVM snapshots in that it's file-by-file, > instead of volume based.) If not, has anyone given it any consideration? > It would be a huge win for the RAID-in-a-box folks, and, speaking as a > sysadmin, is something about which I dream frequently. Well, implementing it at the LVM level isn't entirely bad though. What kind of advantage would have implementing at the filesystem level have? It would need to compensate at least the disadvantage of being less general. Sincerely, Lars Marowsky-Brée <lmb@suse.de> -- SuSE Labs - Research & Development, SuSE Linux AG "If anything can go wrong, it will." "Chance favors the prepared (mind)." -- Capt. Edward A. Murphy -- Louis Pasteur ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Snapshots a la NetApp? 2003-03-23 11:18 ` Snapshots a la NetApp? Lars Marowsky-Bree @ 2003-03-24 12:49 ` Ragnar Kjørstad 0 siblings, 0 replies; 33+ messages in thread From: Ragnar Kjørstad @ 2003-03-24 12:49 UTC (permalink / raw) To: Lars Marowsky-Bree; +Cc: kend, reiserfs-list On Sun, Mar 23, 2003 at 12:18:31PM +0100, Lars Marowsky-Bree wrote: > > I'm just wondering if there's any development being done on NetApp-like > > snapshots. (This differs from LVM snapshots in that it's file-by-file, > > instead of volume based.) If not, has anyone given it any consideration? > > It would be a huge win for the RAID-in-a-box folks, and, speaking as a > > sysadmin, is something about which I dream frequently. > > Well, implementing it at the LVM level isn't entirely bad though. What kind of > advantage would have implementing at the filesystem level have? It would need > to compensate at least the disadvantage of being less general. I think the main advantage of having it in the filesystem is that it makes the snapshots available to the users. This makes it possible for regular users to undelete / restore files without having to contact the sysadmin. As for linux implementations of NetApp-like snapshots you should check out snapfs: http://www.mountainviewdata.com/us/product/product_snap_1.html http://www-mddsp.enel.ucalgary.ca/People/adilger/snapfs/ http://sourcefrog.net/projects/snapfs/ -- Ragnar Kjørstad Zet.no ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Snapshots a la NetApp? 2003-03-23 3:38 ` Snapshots a la NetApp? kend 2003-03-23 9:16 ` Heinz-Josef Claes 2003-03-23 11:18 ` Snapshots a la NetApp? Lars Marowsky-Bree @ 2003-03-23 12:44 ` Oleg Drokin 2 siblings, 0 replies; 33+ messages in thread From: Oleg Drokin @ 2003-03-23 12:44 UTC (permalink / raw) To: kend; +Cc: reiserfs-list Hello! On Sat, Mar 22, 2003 at 10:38:01PM -0500, kend@xanoptix.com wrote: > I'm just wondering if there's any development being done on NetApp-like > snapshots. (This differs from LVM snapshots in that it's file-by-file, > instead of volume based.) If not, has anyone given it any consideration? > It would be a huge win for the RAID-in-a-box folks, and, speaking as a > sysadmin, is something about which I dream frequently. That sounds like file versioning on filesystem, perhaps filesystem (binary only and stuff) from ClearCase suite (or what was the name) from Rational have this. Though I have not tried it myself. And their http://www.rational.com does not answers. May be related to the fact that they were bought by IBM recently ;) Bye, Oleg ^ permalink raw reply [flat|nested] 33+ messages in thread
* Encryption @ 2011-01-20 15:05 Carl Cook 2011-01-20 15:14 ` Encryption Hugo Mills 2011-01-20 16:10 ` Encryption Josef Bacik 0 siblings, 2 replies; 33+ messages in thread From: Carl Cook @ 2011-01-20 15:05 UTC (permalink / raw) To: linux-btrfs Does BTRFS have subvolume encryption built in? If not, why? ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2011-01-20 15:05 Encryption Carl Cook @ 2011-01-20 15:14 ` Hugo Mills 2011-01-20 16:10 ` Encryption Josef Bacik 1 sibling, 0 replies; 33+ messages in thread From: Hugo Mills @ 2011-01-20 15:14 UTC (permalink / raw) To: Carl Cook; +Cc: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1139 bytes --] On Thu, Jan 20, 2011 at 07:05:52AM -0800, Carl Cook wrote: > > Does BTRFS have subvolume encryption built in? If not, why? Not at the moment. My opinion on why: Getting crypto right is *hard*. There are far easier features that people are asking for that we can implement first. There may be technical issues that make it hard to implement within btrfs, although being able to do compression is harder from a FS structure point of view, so I suspect that the issues are more about ensuring correctness of the crypto implementation (not just the basic symmetric algorithm, because we've got those in the kernel, but all the key management and block chaining and probably a bunch of things I don't know about because I'm not a cryptographer -- all of which makes a big difference to the security of the final system). Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Once is happenstance; twice is coincidence; three times --- is enemy action. [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 190 bytes --] ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2011-01-20 15:05 Encryption Carl Cook 2011-01-20 15:14 ` Encryption Hugo Mills @ 2011-01-20 16:10 ` Josef Bacik 1 sibling, 0 replies; 33+ messages in thread From: Josef Bacik @ 2011-01-20 16:10 UTC (permalink / raw) To: Carl Cook; +Cc: linux-btrfs On Thu, Jan 20, 2011 at 07:05:52AM -0800, Carl Cook wrote: > > Does BTRFS have subvolume encryption built in? If not, why? > No, and because nobody has done it yet. Thanks, Josef ^ permalink raw reply [flat|nested] 33+ messages in thread
* (no subject) @ 2012-12-07 16:16 merc1984 2012-12-12 17:12 ` Encryption merc1984 0 siblings, 1 reply; 33+ messages in thread From: merc1984 @ 2012-12-07 16:16 UTC (permalink / raw) To: linux-btrfs We're using a backups server to back up all machines in a LAN. Four 2TB disks are assembled in a BTRFS RAID array and mounted as /media/backups. Under this are subvolumes droog, hex, etc, and snapshots droog_snap-{date1}, hex_snap-{date1}, etc. Goal is to encrypt backups, but the concern is with snapshots. Won't piping rsync through encryption with GPG or somesuch, play havoc with BTRFS snapshot accounting? Is there any way to encrypt an array so it is inaccesible while umounted? I've already asked on the ecryptfs listserv and it resulted in mass confusion. -- http://www.fastmail.fm - A fast, anti-spam email service. ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2012-12-07 16:16 merc1984 @ 2012-12-12 17:12 ` merc1984 2012-12-12 18:31 ` Encryption Mitch Harder 0 siblings, 1 reply; 33+ messages in thread From: merc1984 @ 2012-12-12 17:12 UTC (permalink / raw) To: linux-btrfs So there is no way to have filesystem encryption, while keeping snapshots? On Fri, Dec 7, 2012, at 8:16, [2]merc1984@f-m.fm wrote: > We're using a backups server to back up all machines in a LAN. Four 2TB disks are assembled in a BTRFS RAID array and mounted as /media/backups. Under this are subvolumes droog, hex, etc, and snapshots droog_snap-{date1}, hex_snap-{date1}, etc. > Goal is to encrypt backups, but the concern is with snapshots. Won't piping rsync through encryption with GPG or somesuch, play havoc with BTRFS snapshot accounting? > Is there any way to encrypt an array so it is inaccesible while umounted? > I've already asked on the ecryptfs listserv and it resulted in mass confusion. -- -- http://www.fastmail.fm - Or how I learned to stop worrying and love email again ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2012-12-12 17:12 ` Encryption merc1984 @ 2012-12-12 18:31 ` Mitch Harder 2012-12-12 18:38 ` Encryption merc1984 0 siblings, 1 reply; 33+ messages in thread From: Mitch Harder @ 2012-12-12 18:31 UTC (permalink / raw) To: merc1984; +Cc: linux-btrfs On Wed, Dec 12, 2012 at 11:12 AM, <merc1984@f-m.fm> wrote: > > So there is no way to have filesystem encryption, while keeping > snapshots? > > I run btrfs on top of LUKS encryption on my laptop. You should be able to do the same. You could then run rsync through ssh. However, rsync will have no knowledge of any blocks shared under subvolume snapshots. Btrfs does not yet have internal encryption. ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2012-12-12 18:31 ` Encryption Mitch Harder @ 2012-12-12 18:38 ` merc1984 2012-12-12 18:48 ` Encryption cwillu 0 siblings, 1 reply; 33+ messages in thread From: merc1984 @ 2012-12-12 18:38 UTC (permalink / raw) To: Mitch Harder; +Cc: linux-btrfs On Wed, Dec 12, 2012, at 10:31, Mitch Harder wrote: > I run btrfs on top of LUKS encryption on my laptop. You should be able to do the same. > > You could then run rsync through ssh. However, rsync will have no knowledge of any blocks shared under subvolume snapshots. > > Btrfs does not yet have internal encryption. The FAQ says specifically to NOT run BTRFS with any kind of volume encryption, so you're asking for trouble. And clearly encryption is not possible if you need snapshots. -- http://www.fastmail.fm - One of many happy users: http://www.fastmail.fm/help/overview_quotes.html ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2012-12-12 18:38 ` Encryption merc1984 @ 2012-12-12 18:48 ` cwillu 2012-12-12 20:06 ` Encryption merc1984 0 siblings, 1 reply; 33+ messages in thread From: cwillu @ 2012-12-12 18:48 UTC (permalink / raw) To: merc1984; +Cc: Mitch Harder, linux-btrfs On Wed, Dec 12, 2012 at 12:38 PM, <merc1984@f-m.fm> wrote: > > On Wed, Dec 12, 2012, at 10:31, Mitch Harder wrote: >> I run btrfs on top of LUKS encryption on my laptop. You should be able to do the same. >> >> You could then run rsync through ssh. However, rsync will have no knowledge of any blocks shared under subvolume snapshots. >> >> Btrfs does not yet have internal encryption. > The FAQ says specifically to NOT run BTRFS with any kind of volume > encryption, so you're asking for trouble. Sayeth the FAQ: Does Btrfs work on top of dm-crypt? This is deemed safe since 3.2 kernels. Corruption has been reported before that, so you want a recent kernel. The reason was improper passing of device barriers that are a requirement of the filesystem to guarantee consistency. > And clearly encryption is not possible if you need snapshots. Snapshots don't come into this at all: btrfs doesn't care where the block devices it's on come from. Things like dm-crypt show btrfs (or whatever filesystem you put on it) a decrypted view of the device. ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2012-12-12 18:48 ` Encryption cwillu @ 2012-12-12 20:06 ` merc1984 2012-12-12 20:22 ` Encryption cwillu 2012-12-13 9:17 ` Encryption Sander 0 siblings, 2 replies; 33+ messages in thread From: merc1984 @ 2012-12-12 20:06 UTC (permalink / raw) To: cwillu; +Cc: Mitch Harder, linux-btrfs On Wed, Dec 12, 2012, at 10:48, cwillu wrote: > Sayeth the FAQ: Oh pardon me, it's BTRFS RAID that's a no-go, which is just as critical to me as I have a 4 disk 8TB array. The FAQ goeth on to Say: ----------------------------------------------------------- This pretty much forbids you to use btrfs' cool RAID features if you need encryption. Using a RAID implementation on top of several encrypted disks is much slower than using encryption on top of a RAID device. So the RAID implementation must be on a lower layer than the encryption, which is not possible using btrfs' RAID support. ----------------------------------------------------------- You saw that I need RAID above. Were you just trying to criticize my memory of the FAQ cwillu? -- http://www.fastmail.fm - Accessible with your email software or over the web ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2012-12-12 20:06 ` Encryption merc1984 @ 2012-12-12 20:22 ` cwillu 2012-12-13 9:17 ` Encryption Sander 1 sibling, 0 replies; 33+ messages in thread From: cwillu @ 2012-12-12 20:22 UTC (permalink / raw) To: merc1984; +Cc: Mitch Harder, linux-btrfs On Wed, Dec 12, 2012 at 2:06 PM, <merc1984@f-m.fm> wrote: > On Wed, Dec 12, 2012, at 10:48, cwillu wrote: >> Sayeth the FAQ: > > Oh pardon me, it's BTRFS RAID that's a no-go, which is just as critical > to me as I have a 4 disk 8TB array. > The FAQ goeth on to Say: > ----------------------------------------------------------- > This pretty much forbids you to use btrfs' cool RAID features if you > need encryption. Using a RAID implementation on top of several encrypted > disks is much slower than using encryption on top of a RAID device. So > the RAID implementation must be on a lower layer than the encryption, > which is not possible using btrfs' RAID support. > ----------------------------------------------------------- > > You saw that I need RAID above. Were you just trying to criticize my > memory of the FAQ cwillu? It's not asking for trouble, it's just asking for poor performance, and I suspect even that will depend greatly on the workload. Snapshots still have nothing to do with it: you could have btrfs (with snapshots) on dm-crypt on mdraid. Btrfs would just lose the ability to try alternate mirrors and similar; snapshots would still work just fine. ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2012-12-12 20:06 ` Encryption merc1984 2012-12-12 20:22 ` Encryption cwillu @ 2012-12-13 9:17 ` Sander 2012-12-13 17:23 ` Encryption merc1984 1 sibling, 1 reply; 33+ messages in thread From: Sander @ 2012-12-13 9:17 UTC (permalink / raw) To: merc1984; +Cc: cwillu, Mitch Harder, linux-btrfs merc1984@f-m.fm wrote (ao): > Oh pardon me, it's BTRFS RAID that's a no-go, which is just as critical > to me as I have a 4 disk 8TB array. > The FAQ goeth on to Say: > ----------------------------------------------------------- > This pretty much forbids you to use btrfs' cool RAID features if you > need encryption. Forbids? That is just plain wrong. I have one btrfs filesystem on top of two encrypted devices. Works just fine. Sander ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2012-12-13 9:17 ` Encryption Sander @ 2012-12-13 17:23 ` merc1984 2012-12-13 22:39 ` Encryption Hugo Mills 0 siblings, 1 reply; 33+ messages in thread From: merc1984 @ 2012-12-13 17:23 UTC (permalink / raw) To: Sander; +Cc: cwillu, Mitch Harder, linux-btrfs On Thu, Dec 13, 2012, at 1:17, Sander wrote: Forbids? That is just plain wrong. I have one btrfs filesystem on top of two encrypted devices. Works just fine. That's dynamite Sander. But I am not going to contravene the instructions, then have problems, only to come back here and have fingers wagged in my face telling me this is all EXPERIMENTAL! -- http://www.fastmail.fm - Send your email first class ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: Encryption 2012-12-13 17:23 ` Encryption merc1984 @ 2012-12-13 22:39 ` Hugo Mills 0 siblings, 0 replies; 33+ messages in thread From: Hugo Mills @ 2012-12-13 22:39 UTC (permalink / raw) To: merc1984; +Cc: Sander, cwillu, Mitch Harder, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1105 bytes --] On Thu, Dec 13, 2012 at 09:23:05AM -0800, merc1984@f-m.fm wrote: > > On Thu, Dec 13, 2012, at 1:17, Sander wrote: > Forbids? That is just plain wrong. > I have one btrfs filesystem on top of two encrypted devices. Works just > fine. > > That's dynamite Sander. > > But I am not going to contravene the instructions, then have problems, > only to come back here and have fingers wagged in my face telling me > this is all EXPERIMENTAL! Well, I'm afraid that applies to the information on the wiki, too -- that's also experimental, to a degree. The notes on the wiki about behaviour of encryption layers weren't added by any of the core developers. Nobody's published concrete tests *either* way yet, and those comments are one person's opinion, as far as I'm aware (and note that they don't actually quote sources, results, or even personal experience). YMMV. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Great oxymorons of the world, no. 2: Common Sense --- [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 828 bytes --] ^ permalink raw reply [flat|nested] 33+ messages in thread
* encryption @ 2015-02-16 17:19 Henry Noack 2015-02-18 11:03 ` encryption Stefan Hajnoczi 0 siblings, 1 reply; 33+ messages in thread From: Henry Noack @ 2015-02-16 17:19 UTC (permalink / raw) To: kvm Hello you guys, it is possible to decrypt a kvm volume only by using the command line after starting it? Best regards Henry ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: encryption 2015-02-16 17:19 encryption Henry Noack @ 2015-02-18 11:03 ` Stefan Hajnoczi 2015-02-18 11:58 ` encryption Markus Armbruster 0 siblings, 1 reply; 33+ messages in thread From: Stefan Hajnoczi @ 2015-02-18 11:03 UTC (permalink / raw) To: Henry Noack; +Cc: kvm [-- Attachment #1: Type: text/plain, Size: 750 bytes --] On Mon, Feb 16, 2015 at 06:19:04PM +0100, Henry Noack wrote: > it is possible to decrypt a kvm volume only by using the command line after > starting it? Encryption can be done at 3 levels: 1. Inside the guest. Just like you do on a physical machine with LUKS (dm-crypt), ecryptfs, TrueCrypt, etc. 2. In QEMU with qcow2, although this feature is not widely used and not up to modern disk encryption standards. 3. On the host using LUKS (dm-crypt), ecryptfs, TrueCrypt, etc or on the storage appliance. It depends what you are trying to achieve. Keep in mind that encrypting the disk image does not stop the host from seeing inside the guest. The host is always trusted, today's virtualization technology has this limitation. Stefan [-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: encryption 2015-02-18 11:03 ` encryption Stefan Hajnoczi @ 2015-02-18 11:58 ` Markus Armbruster 0 siblings, 0 replies; 33+ messages in thread From: Markus Armbruster @ 2015-02-18 11:58 UTC (permalink / raw) To: Stefan Hajnoczi; +Cc: Henry Noack, kvm Stefan Hajnoczi <stefanha@gmail.com> writes: > On Mon, Feb 16, 2015 at 06:19:04PM +0100, Henry Noack wrote: >> it is possible to decrypt a kvm volume only by using the command line after >> starting it? > > Encryption can be done at 3 levels: [...] > 2. In QEMU with qcow2, although this feature is not widely used and not > up to modern disk encryption standards. Quoting the fine manual: The use of encryption in qcow and qcow2 images is considered to be flawed by modern cryptography standards, suffering from a number of design problems: − The AES-CBC cipher is used with predictable initialization vectors based on the sector number. This makes it vulnerable to chosen plaintext attacks which can reveal the existence of encrypted data. − The user passphrase is directly used as the encryption key. A poorly chosen or short passphrase will compromise the security of the encryption. − In the event of the passphrase being compromised there is no way to change the passphrase to protect data in any qcow images. The files must be cloned, using a different encryption passphrase in the new file. The original file must then be securely erased using a program like shred, though even this is ineffective with many modern storage technologies. Use of qcow / qcow2 encryption is thus strongly discouraged. Users are recommended to use an alternative encryption technology such as the Linux dm-crypt / LUKS system. [...] ^ permalink raw reply [flat|nested] 33+ messages in thread
end of thread, other threads:[~2015-02-18 11:58 UTC | newest] Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2003-03-22 23:22 Encryption Pierre Abbat 2003-03-22 23:29 ` Encryption Yury Umanets 2003-03-24 20:37 ` Encryption Hans Reiser 2003-03-25 19:18 ` Encryption Edward Shushkin 2003-03-23 3:38 ` Snapshots a la NetApp? kend 2003-03-23 9:16 ` Heinz-Josef Claes 2003-03-24 20:49 ` Hans Reiser 2003-03-24 21:42 ` Heinz-Josef Claes 2003-03-25 0:03 ` Tom Vier 2003-03-26 21:11 ` Heinz-Josef Claes 2003-03-25 2:35 ` Hans Reiser 2003-03-26 14:15 ` Heinz-Josef Claes 2003-03-26 18:37 ` Hans Reiser 2003-03-26 22:45 ` Valdis.Kletnieks 2003-03-25 6:15 ` unsubscribe Voicu Liviu 2003-03-23 11:18 ` Snapshots a la NetApp? Lars Marowsky-Bree 2003-03-24 12:49 ` Ragnar Kjørstad 2003-03-23 12:44 ` Oleg Drokin 2011-01-20 15:05 Encryption Carl Cook 2011-01-20 15:14 ` Encryption Hugo Mills 2011-01-20 16:10 ` Encryption Josef Bacik 2012-12-07 16:16 merc1984 2012-12-12 17:12 ` Encryption merc1984 2012-12-12 18:31 ` Encryption Mitch Harder 2012-12-12 18:38 ` Encryption merc1984 2012-12-12 18:48 ` Encryption cwillu 2012-12-12 20:06 ` Encryption merc1984 2012-12-12 20:22 ` Encryption cwillu 2012-12-13 9:17 ` Encryption Sander 2012-12-13 17:23 ` Encryption merc1984 2012-12-13 22:39 ` Encryption Hugo Mills 2015-02-16 17:19 encryption Henry Noack 2015-02-18 11:03 ` encryption Stefan Hajnoczi 2015-02-18 11:58 ` encryption Markus Armbruster
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.