I give up

I give up

[Gene Heskett]

Greetings Guys & Dolls;

I give up, I'm whupped.

When I installed FC6, I let it use LVM2 to handle the disks because I was 
tired of fighting with the most broken, nasty, evil disk partitioner ever 
dreamed up, disk druid.  I personally think that abomination was written 
by an M$ plant, specifically to poison users against linux.

Now the 64k$ question: While running with LVM2 managed disks, is it 
possible to run without dm_mod, the device-mapper?  If so, please tell me 
how to achieve this.

I rebooted last night, from 2.6.21-rc6 to 2.6.21-rc6 and for some reason 
device-mapper went from #252 (cat /proc/devices) to #254, and has also 
been seen at #253 in addition to the temporary address of #238 where it 
was before that patch was reverted for 2.6.21-rc6

Anytime this changes, the device numbers in a stat report change, and tar 
thinks its all new & restarts everything on a 45GB system from a level 0.
My vtape setup only has room for 8.5GB per vtape, so the catchup time is 
many days.  I can speed that up by temporarily making the vtape bigger in 
my amanda.conf, but now the disk itself that the vtapes are on is at 96% 
and will not tolerate another such catchup run.

So I'm pleading for either a stable, known address for device-mapper, or a 
workaround that will immunize tar against its apparently uncontrolled 
wanderings.

Please, pretty please, with sugar and cream in it even, please, give some 
stability to this situation....

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
George Washington was first in war, first in peace -- and the first to
have his birthday juggled to make a long weekend.
		-- Ashley Cooper

Re: I give up

[Jeff Garzik]

Gene Heskett wrote:
> Now the 64k$ question: While running with LVM2 managed disks, is it 
> possible to run without dm_mod, the device-mapper?  If so, please tell me 
> how to achieve this.

No; device mapper is the kernel portion of LVM2.

	Jeff, who actively avoids LVM on home computers

Re: I give up

[Gene Heskett]

On Monday 09 April 2007, Jeff Garzik wrote:
>Gene Heskett wrote:
>> Now the 64k$ question: While running with LVM2 managed disks, is it
>> possible to run without dm_mod, the device-mapper?  If so, please tell
>> me how to achieve this.
>
>No; device mapper is the kernel portion of LVM2.
>
>	Jeff, who actively avoids LVM on home computers

Ya shoulda warned me.  :-)

It should have lots bigger warning labels, in bright red self-illuminating 
signs, than it does.  It in fact seems to work well, but it is a major 
breakage for some common apps, like doing backups...

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Gates' Law: Every 18 months, the speed of software halves. 

   -- From a Slashdot.org post

Re: I give up

[Jan Engelhardt]

On Apr 9 2007 11:37, Gene Heskett wrote:
>On Monday 09 April 2007, Jeff Garzik wrote:
>>Gene Heskett wrote:
>>
>>No; device mapper is the kernel portion of LVM2.
>>
>>	Jeff, who actively avoids LVM on home computers
>
>Ya shoulda warned me.  :-)
>
>It should have lots bigger warning labels, in bright red self-illuminating 
>signs, than it does.

Actually, you have to blame some FC developer for making an LVM-based
setup the _default_ for home installs (aka "FC", enterprise is RHEL/¢OS).

>It in fact seems to work well, but it is a major 
>breakage for some common apps, like doing backups...

dm is on 254 for me.. in opensuse with a 2.6.20 that is. I wonder why it
even moves around. However, even then, those who use udev and device names
rather than (major,minor) tuples should not have any problem.


Jan
-- 

Re: I give up

[H. Peter Anvin]

Jan Engelhardt wrote:
> 
> dm is on 254 for me.. in opensuse with a 2.6.20 that is. I wonder why it
> even moves around. However, even then, those who use udev and device names
> rather than (major,minor) tuples should not have any problem.
> 

It moves around because someone at some point thought it was a great 
idea to assign dynamic majors to core functionality.

It stays put at 254 for you since that's the highest (and thus 
first-used) dynamic major.

	-hpa

Re: I give up

[Gene Heskett]

On Monday 09 April 2007, H. Peter Anvin wrote:
>Jan Engelhardt wrote:
>> dm is on 254 for me.. in opensuse with a 2.6.20 that is. I wonder why
>> it even moves around. However, even then, those who use udev and
>> device names rather than (major,minor) tuples should not have any
>> problem.
>
>It moves around because someone at some point thought it was a great
>idea to assign dynamic majors to core functionality.
>
>It stays put at 254 for you since that's the highest (and thus
>first-used) dynamic major.
>
>	-hpa

Until I also build pkdcdvd or md, each of which shove it down the list.

Now, as an update here at the coyote.den, fixing the typu in 
my /etc/modprobe.conf (Daves original message used dm_mod in the sample 
and it should have been dm-mod) and another rebuild/install cycle, and it 
is now at the LANANA assigned address that the patch which was reverted 
put it at, 238=EE and EE00h=60928.  Now all we have to do is write a 
script to fix the reference files tar is given for incremental detection 
so that the device number is corrected to the current device as reported 
by a 'stat /' command.

However, there seems to be a  wholesale conviction that the files are 
encrypted or otherwise smunched, they are not, but absolutely plain text:
[amanda@coyote gnutar-lists]$ cat coyote_var_5
1175495769
60928 19169380 ./run/cups
60928 19300353 ./cache/gstreamer-0.8
60928 19169448 ./lib/scrollkeeper/az
60928 19595604 ./tmp/RealPlayer/share/locale/ko
60928 19300370 ./cache/yum/updates-source
60928 19169358 ./cache/man/local
60928 19235048 ./tmp/kdecache-root/background
60928 19988737 ./tmp/kdecache-root/kio_help/root/.kde/share/doc
60928 19791895 ./spool/cups-pdf

So writing a script that compares that first number to one of the FE, FD, 
FC variations and corrects it to the EE value as converted above should 
be relatively trivial and I'll try to do that yet today, but there are a 
couple of honeydo's to take care of first.



-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Don't talk to me about naval tradition.  It's nothing but rum, sodomy and
the lash.
	-- Winston Churchill

Re: I give up

[Gene Heskett]

On Monday 09 April 2007, H. Peter Anvin wrote:
>Jan Engelhardt wrote:
>> dm is on 254 for me.. in opensuse with a 2.6.20 that is. I wonder why
>> it even moves around. However, even then, those who use udev and
>> device names rather than (major,minor) tuples should not have any
>> problem.
>
>It moves around because someone at some point thought it was a great
>idea to assign dynamic majors to core functionality.
>
What were they smoking, I want some of that!

>It stays put at 254 for you since that's the highest (and thus
>first-used) dynamic major.
>
>	-hpa



-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
<Culus> OH MY GOD NOT A RANDOM QUOTE GENERATOR
<netgod> surely you didnt think that was static? how lame would that 
be? :-)

Re: I give up

[Jan Engelhardt]

On Apr 9 2007 15:38, Gene Heskett wrote:
>On Monday 09 April 2007, H. Peter Anvin wrote:
>>Jan Engelhardt wrote:
>>> dm is on 254 for me.. in opensuse with a 2.6.20 that is. I wonder why
>>> it even moves around. However, even then, those who use udev and
>>> device names rather than (major,minor) tuples should not have any
>>> problem.
>>
>>It moves around because someone at some point thought it was a great
>>idea to assign dynamic majors to core functionality.
>>
>What were they smoking, I want some of that!

Do you actually use udev?



Jan
-- 

Re: I give up

[Dave Dillow]

On Mon, 2007-04-09 at 23:35 +0200, Jan Engelhardt wrote:
> On Apr 9 2007 15:38, Gene Heskett wrote:
> >On Monday 09 April 2007, H. Peter Anvin wrote:
> >>Jan Engelhardt wrote:
> >>> dm is on 254 for me.. in opensuse with a 2.6.20 that is. I wonder why
> >>> it even moves around. However, even then, those who use udev and
> >>> device names rather than (major,minor) tuples should not have any
> >>> problem.
> >>
> >>It moves around because someone at some point thought it was a great
> >>idea to assign dynamic majors to core functionality.
> >>
> >What were they smoking, I want some of that!
> 
> Do you actually use udev?

udev doesn't help the problem he is having (and he is using it, since he
is using Fedora).

His problem is that GNU tar stores the device number of the file in a
database that it uses to perform incremental updates, and considers the
file new (and re-dumps it) when the device number changes.

So, when he changes the kernel config, adding any block device that uses
a dynamic major (and is loaded before device-mapper), or goes between
kernels that have the reserve-LANANA numbers patch, his incremental
dumps become full ones because device-mapper gets a different dynamic
major.
-- 
Dave Dillow <dave@thedillows.org>

Re: I give up

[Dave Jones]

On Mon, Apr 09, 2007 at 06:22:10PM -0400, Dave Dillow wrote:
 > On Mon, 2007-04-09 at 23:35 +0200, Jan Engelhardt wrote:
 > > On Apr 9 2007 15:38, Gene Heskett wrote:
 > > >On Monday 09 April 2007, H. Peter Anvin wrote:
 > > >>Jan Engelhardt wrote:
 > > >>> dm is on 254 for me.. in opensuse with a 2.6.20 that is. I wonder why
 > > >>> it even moves around. However, even then, those who use udev and
 > > >>> device names rather than (major,minor) tuples should not have any
 > > >>> problem.
 > > >>
 > > >>It moves around because someone at some point thought it was a great
 > > >>idea to assign dynamic majors to core functionality.
 > > >>
 > > >What were they smoking, I want some of that!
 > > 
 > > Do you actually use udev?
 > 
 > udev doesn't help the problem he is having (and he is using it, since he
 > is using Fedora).

However, it also doesn't explain what the point is of backing up /dev
when it's dynamically created.

	Dave

-- 
http://www.codemonkey.org.uk

Re: I give up

[Dave Dillow]

On Mon, 2007-04-09 at 21:27 -0400, Dave Jones wrote:
> On Mon, Apr 09, 2007 at 06:22:10PM -0400, Dave Dillow wrote:
>  > On Mon, 2007-04-09 at 23:35 +0200, Jan Engelhardt wrote:
>  > > On Apr 9 2007 15:38, Gene Heskett wrote:
>  > > >On Monday 09 April 2007, H. Peter Anvin wrote:
>  > > >>Jan Engelhardt wrote:
>  > > >>> dm is on 254 for me.. in opensuse with a 2.6.20 that is. I wonder why
>  > > >>> it even moves around. However, even then, those who use udev and
>  > > >>> device names rather than (major,minor) tuples should not have any
>  > > >>> problem.
>  > > >>
>  > > >>It moves around because someone at some point thought it was a great
>  > > >>idea to assign dynamic majors to core functionality.
>  > > >>
>  > > >What were they smoking, I want some of that!
>  > > 
>  > > Do you actually use udev?
>  > 
>  > udev doesn't help the problem he is having (and he is using it, since he
>  > is using Fedora).
> 
> However, it also doesn't explain what the point is of backing up /dev
> when it's dynamically created.

It's not /dev he's backing up -- its /home, /usr, and others. GNU tar
saves the device and inode numbers from the {,l}stat() call on each file
and decides it is a new file if either number changes from run to run.

You are correct about /dev -- who'd back it up if it is dynamically
created? But that is also not the problem Gene is having.

Re: I give up

[Dave Jones]

On Mon, Apr 09, 2007 at 09:40:46PM -0400, Dave Dillow wrote:
 
 > > However, it also doesn't explain what the point is of backing up /dev
 > > when it's dynamically created.
 > 
 > It's not /dev he's backing up -- its /home, /usr, and others. GNU tar
 > saves the device and inode numbers from the {,l}stat() call on each file
 > and decides it is a new file if either number changes from run to run.

Ah apologies, I jumped into the thread halfway, and misunderstood the problem.

	Dave

-- 
http://www.codemonkey.org.uk

Re: I give up

[Olaf Hering]

On Mon, Apr 09, Dave Dillow wrote:

> It's not /dev he's backing up -- its /home, /usr, and others. GNU tar
> saves the device and inode numbers from the {,l}stat() call on each file
> and decides it is a new file if either number changes from run to run.

So fix tar to not do silly things.
Kernel major:minor numbers are not stable.

Re: I give up

[Gene Heskett]

On Tuesday 10 April 2007, Olaf Hering wrote:
>On Mon, Apr 09, Dave Dillow wrote:
>> It's not /dev he's backing up -- its /home, /usr, and others. GNU tar
>> saves the device and inode numbers from the {,l}stat() call on each
>> file and decides it is a new file if either number changes from run to
>> run.
>
>So fix tar to not do silly things.
>Kernel major:minor numbers are not stable.

YOU Tell that to the tar/star people, they are flabbergasted that its not 
stable.  It apparently is for every other OS tar can be run on.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
All your people must learn before you can reach for the stars.
		-- Kirk, "The Gamesters of Triskelion", stardate 3259.2

Re: I give up

[Jeff Garzik]

Gene Heskett wrote:
> On Tuesday 10 April 2007, Olaf Hering wrote:
>> On Mon, Apr 09, Dave Dillow wrote:
>>> It's not /dev he's backing up -- its /home, /usr, and others. GNU tar
>>> saves the device and inode numbers from the {,l}stat() call on each
>>> file and decides it is a new file if either number changes from run to
>>> run.
>> So fix tar to not do silly things.
>> Kernel major:minor numbers are not stable.
> 
> YOU Tell that to the tar/star people, they are flabbergasted that its not 
> stable.  It apparently is for every other OS tar can be run on.

It's just the new world that we live in.

I don't see us ever going back to 100% static major/minors.

Though as a point of history, Linux has /always/ supported dynamic 
major/minor numbers, even back in 0.99 days.  We avoided the problem 
then because the dynamic major/minors were only used by drivers that had 
not yet had static numbers assigned, or their author chose to avoid 
LANANA for some other reason.

	Jeff

Re: I give up

[Olaf Hering]

On Tue, Apr 10, Gene Heskett wrote:

> On Tuesday 10 April 2007, Olaf Hering wrote:
> >On Mon, Apr 09, Dave Dillow wrote:
> >> It's not /dev he's backing up -- its /home, /usr, and others. GNU tar
> >> saves the device and inode numbers from the {,l}stat() call on each
> >> file and decides it is a new file if either number changes from run to
> >> run.
> >
> >So fix tar to not do silly things.
> >Kernel major:minor numbers are not stable.

I forgot to add: '.. not stable across reboots.'

> YOU Tell that to the tar/star people, they are flabbergasted that its not 
> stable.  It apparently is for every other OS tar can be run on.

They probably have a point with the st_dev usage.
You better find out why your major:minor pairs keep jumping around.
Simply because the 'not stable across reboot' statement holds only for
added/removed disks and maybe if the detection order changes.
If your setup relies on a certain order, make sure the drivers get
loaded in a fixed order. Its not clear from your other mails what
exactly caused it. If its only due to a temporary change in
testkernels, noone can do anything about it.

Re: I give up

[David Lang]

On Tue, 10 Apr 2007, Olaf Hering wrote:

> On Tue, Apr 10, Gene Heskett wrote:
>
>> On Tuesday 10 April 2007, Olaf Hering wrote:
>>> On Mon, Apr 09, Dave Dillow wrote:
>>>> It's not /dev he's backing up -- its /home, /usr, and others. GNU tar
>>>> saves the device and inode numbers from the {,l}stat() call on each
>>>> file and decides it is a new file if either number changes from run to
>>>> run.
>>>
>>> So fix tar to not do silly things.
>>> Kernel major:minor numbers are not stable.
>
> I forgot to add: '.. not stable across reboots.'
>
>> YOU Tell that to the tar/star people, they are flabbergasted that its not
>> stable.  It apparently is for every other OS tar can be run on.
>
> They probably have a point with the st_dev usage.
> You better find out why your major:minor pairs keep jumping around.
> Simply because the 'not stable across reboot' statement holds only for
> added/removed disks and maybe if the detection order changes.
> If your setup relies on a certain order, make sure the drivers get
> loaded in a fixed order. Its not clear from your other mails what
> exactly caused it. If its only due to a temporary change in
> testkernels, noone can do anything about it.

he's running into problems when he adds additional drivers to the kernel (I 
don't remember which drivers, he's mentioned them earlier in this thread)

David Lang

Re: I give up

[Jan Engelhardt]

On Apr 10 2007 03:51, Gene Heskett wrote:
>On Tuesday 10 April 2007, Olaf Hering wrote:
>>On Mon, Apr 09, Dave Dillow wrote:
>>> It's not /dev he's backing up -- its /home, /usr, and others. GNU tar
>>> saves the device and inode numbers from the {,l}stat() call on each
>>> file and decides it is a new file if either number changes from run to
>>> run.
>>
>>So fix tar to not do silly things.
>>Kernel major:minor numbers are not stable.
>
>YOU Tell that to the tar/star people, they are flabbergasted that its not 
>stable.  It apparently is for every other OS tar can be run on.

FreeBSD also seems to be quite "dynamic".
/dev/da0 is (0,92) for the 'fixit shell' -- how about you?


Jan
-- 

Re: I give up

[Gene Heskett]

On Tuesday 10 April 2007, Jan Engelhardt wrote:
>On Apr 10 2007 03:51, Gene Heskett wrote:
>>On Tuesday 10 April 2007, Olaf Hering wrote:
>>>On Mon, Apr 09, Dave Dillow wrote:
>>>> It's not /dev he's backing up -- its /home, /usr, and others. GNU
>>>> tar saves the device and inode numbers from the {,l}stat() call on
>>>> each file and decides it is a new file if either number changes from
>>>> run to run.
>>>
>>>So fix tar to not do silly things.
>>>Kernel major:minor numbers are not stable.
>>
>>YOU Tell that to the tar/star people, they are flabbergasted that its
>> not stable.  It apparently is for every other OS tar can be run on.
>
>FreeBSD also seems to be quite "dynamic".
>/dev/da0 is (0,92) for the 'fixit shell' -- how about you?
>
I don't have such a beast here.  Whats that supposed to do?
>
>Jan



-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Life sucks, but death doesn't put out at all.
		-- Thomas J. Kopp

Re: I give up

[Jan Engelhardt]

On Apr 10 2007 23:54, Gene Heskett wrote:
>>>>
>>>>So fix tar to not do silly things.
>>>>Kernel major:minor numbers are not stable.
>>>
>>>YOU Tell that to the tar/star people, they are flabbergasted that its
>>> not stable.  It apparently is for every other OS tar can be run on.
>>
>>FreeBSD also seems to be quite "dynamic".
>>/dev/da0 is (0,92) for the 'fixit shell' -- how about you?
>>
>I don't have such a beast here.  Whats that supposed to do?

Well I was just pointing out that Linux is not the only one to have device
numbers (for promiment block-backed storage) that can move across reboots.


Jan
-- 

Re: I give up

[John Stoffel]

>>>>> "Jan" == Jan Engelhardt <jengelh@linux01.gwdg.de> writes:

Jan> On Apr 10 2007 23:54, Gene Heskett wrote:
>>>>> 
>>>>> So fix tar to not do silly things.
>>>>> Kernel major:minor numbers are not stable.
>>>> 
>>>> YOU Tell that to the tar/star people, they are flabbergasted that its
>>>> not stable.  It apparently is for every other OS tar can be run on.
>>> 
>>> FreeBSD also seems to be quite "dynamic".
>>> /dev/da0 is (0,92) for the 'fixit shell' -- how about you?
>>> 
>> I don't have such a beast here.  Whats that supposed to do?

Jan> Well I was just pointing out that Linux is not the only one to
Jan> have device numbers (for promiment block-backed storage) that can
Jan> move across reboots.

I guess one work around would be to export your local filesystems via
NFS and then have amanda back them up that way, which means gnutar
should then ignore the devmajor and devminor numbers.  

Or you could just use a hack of the perl script 'tarcust' to hack the
minor/major numbers stored in the index files, and make them match the
current major/minor pair for each filesystem just before you do a
backup.   

A total hack, but probably what I would do, since gtar should (IMNSHO)
just ignore the devmajor/devminor and just go by the name.  It's not
like the sysadmin can't shoot himself in the foot anyway.

Or, again, get away from Amanda and tar and instead move to Bacula.
Smarter software.

John

Re: I give up

[Gene Heskett]

On Wednesday 11 April 2007, John Stoffel wrote:
>>>>>> "Jan" == Jan Engelhardt <jengelh@linux01.gwdg.de> writes:
>
>Jan> On Apr 10 2007 23:54, Gene Heskett wrote:
>>>>>> So fix tar to not do silly things.
>>>>>> Kernel major:minor numbers are not stable.
>>>>>
>>>>> YOU Tell that to the tar/star people, they are flabbergasted that
>>>>> its not stable.  It apparently is for every other OS tar can be run
>>>>> on.
>>>>
>>>> FreeBSD also seems to be quite "dynamic".
>>>> /dev/da0 is (0,92) for the 'fixit shell' -- how about you?
>>>
>>> I don't have such a beast here.  Whats that supposed to do?
>
>Jan> Well I was just pointing out that Linux is not the only one to
>Jan> have device numbers (for promiment block-backed storage) that can
>Jan> move across reboots.
>
>I guess one work around would be to export your local filesystems via
>NFS and then have amanda back them up that way, which means gnutar
>should then ignore the devmajor and devminor numbers.
>
>Or you could just use a hack of the perl script 'tarcust' to hack the
>minor/major numbers stored in the index files, and make them match the
>current major/minor pair for each filesystem just before you do a
>backup.
>
>A total hack, but probably what I would do, since gtar should (IMNSHO)
>just ignore the devmajor/devminor and just go by the name.  It's not
>like the sysadmin can't shoot himself in the foot anyway.
>
>Or, again, get away from Amanda and tar and instead move to Bacula.
>Smarter software.
>
>John

Does Bacula not use tar for its dirty work?


-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Nobody ever died from oven crude poisoning.

Re: I give up

[John Stoffel]

>>>>> "Gene" == Gene Heskett <gene.heskett@gmail.com> writes:

Gene> Does Bacula not use tar for its dirty work?

Nope, it uses it's own filesystem walking code.  

John

Re: I give up

[Gene Heskett]

On Wednesday 11 April 2007, John Stoffel wrote:
>>>>>> "Gene" == Gene Heskett <gene.heskett@gmail.com> writes:
>
>Gene> Does Bacula not use tar for its dirty work?
>
>Nope, it uses it's own filesystem walking code.
>
>John

Humm, interesting John.  Neither yumex, or smart, is offering it to me as 
an installable rpm for FC6.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Everything should be built top-down, except this time.

Re: I give up

[John Stoffel]

Gene> Humm, interesting John.  Neither yumex, or smart, is offering it
Gene> to me as an installable rpm for FC6.

Go and grab it from http://www.bacula.org, I'm pretty sure there are
RPMs available on there.  You might also need to install seperate RPMs
for the follwoing packages:

	bacula-common
	bacula-director
	bacula-sd
	bacula-fd
	bacula-console
	bacula-mysql

Oh yeah, bacula really wants to use an SQL DB for it's file index.  So
I'd strongly suggest you go and read the manual,which is quite
complete, before you do anything else.

Feel free to contact me off-list for help, or join the bacula-users
list.  We're a bit off topic for LKML now...

John

Re: I give up

[Gene Heskett]

On Wednesday 11 April 2007, Jan Engelhardt wrote:
>On Apr 10 2007 23:54, Gene Heskett wrote:
>>>>>So fix tar to not do silly things.
>>>>>Kernel major:minor numbers are not stable.
>>>>
>>>>YOU Tell that to the tar/star people, they are flabbergasted that its
>>>> not stable.  It apparently is for every other OS tar can be run on.
>>>
>>>FreeBSD also seems to be quite "dynamic".
>>>/dev/da0 is (0,92) for the 'fixit shell' -- how about you?
>>
>>I don't have such a beast here.  Whats that supposed to do?
>
>Well I was just pointing out that Linux is not the only one to have
> device numbers (for promiment block-backed storage) that can move
> across reboots.
>
Which is one more argument in favor of fixing tar, Jorg not withstanding.  
Yeah, that Jorg...  AFAIC, time & technology move on, and I believe I see 
a future where the actual device number might not be the same across a 
reboot, for reasons a hell of a lot less clear than what triggered this 
latest 3 week long battle with something so simple as making a backup.

But, its a battle we'll never win with Jorg stirring the pot, he is the 
single, most vociferous opponent of anything labeled progress that I've 
ever had the displeasure to trying to follow in his rants, and that IS 
what they are, on this and on the scsi list when I was subscribed there.

I do think that a reasoned argument could be made to Paul Eggert, for at 
least a command line option to be made available that would cause tar to 
ignore it, process it yes, but ignore it for --listed-incremental in 
particular.  Maybe even make it part of the --listed-incremental as a 
permanent feature.  For us, that would be the ideal.  It is not an issue 
for level 0's of course.

But my single voice is not going to be able to effect a campaign, its 
going to take a general consensus of all the movers and shakers of linux, 
TPTB if you will, all ganging up on Paul.  Or someone doing a patch to 
that effect, then making distro packages out of it, such as debs, rpm's, 
etc, etc.  A fork I think its called.

There is another also listed in the tar ChangeLog, but I haven't rx'd a 
response from that person although he was Cc:'d at the time.

Dave Dillow, you mentioned another routine in tar that might be an 
educational file to walk through, besides the compare.c I tried to play 
with and apparently broke, can you repeat that phrase again?  I might 
take a stick and poke around in that one too.  Maybe I can un-break 
it? :)

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Earn cash in your spare time -- blackmail your friends.

Re: I give up

[Gene Heskett]

On Tuesday 10 April 2007, Olaf Hering wrote:
>On Mon, Apr 09, Dave Dillow wrote:
>> It's not /dev he's backing up -- its /home, /usr, and others. GNU tar
>> saves the device and inode numbers from the {,l}stat() call on each
>> file and decides it is a new file if either number changes from run to
>> run.
>
>So fix tar to not do silly things.
>Kernel major:minor numbers are not stable.

YOU Tell the tar people, they are flabbergasted that linux is apparently 
the only unstable OS that tar can be run on.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Stay the curse.

Re: I give up

[John Anthony Kazos Jr.]

> >So fix tar to not do silly things.
> >Kernel major:minor numbers are not stable.
> 
> YOU Tell the tar people, they are flabbergasted that linux is apparently 
> the only unstable OS that tar can be run on.

Linux is not unstable. It's developmental! Linux is like one giant 
international research project marching toward the future of computing. 
MacOS (oh, how I miss System 7.5.5), Windows (shoot me, please), BSD 
(wannabe GNU/Linux), and everything else are there just for companies to 
spend money on and idiots to drool over until Linux is ready to stand up 
and say "Yo. It's just bugfixes and improvements from here. You can put 
the Legos down now."

Re: I give up

[Gene Heskett]

On Tuesday 10 April 2007, John Anthony Kazos Jr. wrote:
>> >So fix tar to not do silly things.
>> >Kernel major:minor numbers are not stable.
>>
>> YOU Tell the tar people, they are flabbergasted that linux is
>> apparently the only unstable OS that tar can be run on.
>
>Linux is not unstable. It's developmental! Linux is like one giant
>international research project marching toward the future of computing.
>MacOS (oh, how I miss System 7.5.5), Windows (shoot me, please), BSD
>(wannabe GNU/Linux), and everything else are there just for companies to
>spend money on and idiots to drool over until Linux is ready to stand up
>and say "Yo. It's just bugfixes and improvements from here. You can put
>the Legos down now."

I don't know who you are, but that's not germane to this discussion, what 
is germane is that once the experimental label has been removed from 
something as capable of screwing up the system as the LMV stuff is, then 
there is no excuse not to give it a LANANA compatible address and quit 
screwing up peoples backup systems just because they turned on pktcdvd or 
md or some other similar option in a make xconfig.  FWIW, md, for raid 
users, should get the same treatment since raids often can have 
terrabytes of data to be backed up.

The patch to do that caught tar users totally off-guard, but once we(I) 
understood why it was done, it was obviously a good patch.  But rather 
than explaining WTF it was doing so we/I understood the reason for the 
pain, the *&%^ patch was reverted, starting the backup related screwups 
all over again from scratch.

Now this thread has a life of its own, and the noise it continues to make 
will prevent the re-application of that patch which WILL stabilize 
things, probably until hell freezes over and pigs can use it for a 
takeoff runway.

I have a fix (thanks Dave D.) worked out for MY system, and you guys can 
do as you damned well please, until you break my fix anyway.  In the 
meantime, I have backup.sh, my amdump wrapper, running every 3 hours 
around the clock without a verify stage until it has managed to update 
all 118 (about 20 megs worth) of its reference files.  I expect that to 
take 3 or 4 days.  Hopefully my drives are up to the torture, if not, 
well they aren't that precious & I CAN drop the card for more & might 
even do that yet tonight unless Circuit City or Staples think that a 
500Gb is still worth its weight in gold bar.  Yes, I spend my money 
locally where possible.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
"Don't discount flying pigs before you have good air defense."
-- jvh@clinet.FI

Re: I give up

[Phillip Susi]

Gene Heskett wrote:
> YOU Tell the tar people, they are flabbergasted that linux is apparently 
> the only unstable OS that tar can be run on.

How about cygwin/windows?  It has no concept of static device numbers. 
And what about external usb disks on any other unix os?  Surely they 
don't have static minor device numbers in the face of hot plugging?

The bug is in tar and that is what needs fixed.  It should not attach 
any meaning to the device number, but rather should assume that the 
admin knows what he is doing when he said to backup a given path using a 
given incremental backup log, and that he isn't yanking tar's chain and 
pointing it to a different path between incremental backups.

Re: I give up

[Gene Heskett]

On Tuesday 10 April 2007, Phillip Susi wrote:
>Gene Heskett wrote:
>> YOU Tell the tar people, they are flabbergasted that linux is
>> apparently the only unstable OS that tar can be run on.
>
>How about cygwin/windows?  It has no concept of static device numbers.
>And what about external usb disks on any other unix os?  Surely they
>don't have static minor device numbers in the face of hot plugging?
>
>The bug is in tar and that is what needs fixed.  It should not attach
>any meaning to the device number, but rather should assume that the
>admin knows what he is doing when he said to backup a given path using a
>given incremental backup log, and that he isn't yanking tar's chain and
>pointing it to a different path between incremental backups.

I violently agree on that point, but the limited conversations I've had 
with the tar maintainer so far indicates that AFAHIC, its linux that's 
broken.  A new device number is a new disk and must be treated as such.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
QOTD:
	"I'll listen to reason when it comes out on CD."

Re: I give up

[Phillip Susi]

Gene Heskett wrote:
> I violently agree on that point, but the limited conversations I've had 
> with the tar maintainer so far indicates that AFAHIC, its linux that's 
> broken.  A new device number is a new disk and must be treated as such.

Sumbit a patch adding an option to ignore the device number, and slap 
him around a bit with a large trout ;)

Re: I give up

[Gene Heskett]

On Wednesday 11 April 2007, Phillip Susi wrote:
>Gene Heskett wrote:
>> I violently agree on that point, but the limited conversations I've
>> had with the tar maintainer so far indicates that AFAHIC, its linux
>> that's broken.  A new device number is a new disk and must be treated
>> as such.
>
>Sumbit a patch adding an option to ignore the device number, and slap
>him around a bit with a large trout ;)

Well, I made one pass at that, and blew it up with a 2 pound block of C4.  
I know just enough about C to be dangerous, and my real C knowledge is 
pretty much K&R.

As for the large trout, them are definitely on the hard to find list here 
in WV.  Now if I still lived in Rapid City S.D., this time of the year is 
a great time to pick up the odd old hook nosed Brown Trout that might go 
as high as 8 or 9 pounds and take up 30+ inches laying on your yardstick.  
I had 2 such on my stringer about 40 years ago one late April day but 
they got to fighting it, broke it and got away.  I thought my wife, in 
the other end of the boat, was going to throw me overboard to retrieve 
them, but I don't swim well, don't even float, and the nearest solid 
earth was 170 feet away, straight down...  Fortunately for me she didn't.  
Unfortunately, she had a stroke and died a year later, at age 34.  I 
still miss her, she was what you would call a good woman.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Politics is not the art of the possible.  It consists in choosing
between the disastrous and the unpalatable.
		-- John Kenneth Galbraith

Re: I give up

[John Anthony Kazos Jr.]

> > I violently agree on that point, but the limited conversations I've had with
> > the tar maintainer so far indicates that AFAHIC, its linux that's broken.  A
> > new device number is a new disk and must be treated as such.
> 
> Sumbit a patch adding an option to ignore the device number, and slap him
> around a bit with a large trout ;)

I think that's especially true. If a user begins with a single full disk 
for their entire filesystem, uses tar to backup, and then later adds a 
second disk, copies everything from /usr and /home onto partitions there 
(making sure to preserve all interesting bits like ctime/mtime), and 
mounts them over the original directories, tar should not decide that 
every file in /usr and /home was deleted and recreated.

Re: I give up

[Krzysztof Halasa]

"John Anthony Kazos Jr." <jakj@j-a-k-j.com> writes:

> I think that's especially true. If a user begins with a single full disk 
> for their entire filesystem, uses tar to backup, and then later adds a 
> second disk, copies everything from /usr and /home onto partitions there 
> (making sure to preserve all interesting bits like ctime/mtime), and 
> mounts them over the original directories, tar should not decide that 
> every file in /usr and /home was deleted and recreated.

That wouldn't work, even if you managed to copy ctime, the target
inode number will be different than the original and backup has
to assume it's a different file.

For filesystems with no stable inode numbers my backup scheme
assumes that the same file = the same name and times but it's
really fragile.

An option to ignore block device number does make sense, of course.
Especially when backing up only one filesystem (-xdev etc).
-- 
Krzysztof Halasa

Re: I give up

[Jan Engelhardt]

On Apr 11 2007 18:43, Krzysztof Halasa wrote:
>"John Anthony Kazos Jr." <jakj@j-a-k-j.com> writes:
>
>> I think that's especially true. If a user begins with a single full disk 
>> for their entire filesystem, uses tar to backup, and then later adds a 
>> second disk, copies everything from /usr and /home onto partitions there 
>> (making sure to preserve all interesting bits like ctime/mtime), and 
>> mounts them over the original directories, tar should not decide that 
>> every file in /usr and /home was deleted and recreated.
>
>That wouldn't work, even if you managed to copy ctime, the target
>inode number will be different than the original and backup has
>to assume it's a different file.

Time for rsync and/or rsync -c. And some target filesystem that is
fused to a [can-be-]non-solid archive, like 7z.


Jan
-- 

Re: I give up

[Gene Heskett]

On Monday 09 April 2007, Dave Jones wrote:
>On Mon, Apr 09, 2007 at 06:22:10PM -0400, Dave Dillow wrote:
> > On Mon, 2007-04-09 at 23:35 +0200, Jan Engelhardt wrote:
> > > On Apr 9 2007 15:38, Gene Heskett wrote:
> > > >On Monday 09 April 2007, H. Peter Anvin wrote:
> > > >>Jan Engelhardt wrote:
> > > >>> dm is on 254 for me.. in opensuse with a 2.6.20 that is. I
> > > >>> wonder why it even moves around. However, even then, those who
> > > >>> use udev and device names rather than (major,minor) tuples
> > > >>> should not have any problem.
> > > >>
> > > >>It moves around because someone at some point thought it was a
> > > >> great idea to assign dynamic majors to core functionality.
> > > >
> > > >What were they smoking, I want some of that!
> > >
> > > Do you actually use udev?
> >
> > udev doesn't help the problem he is having (and he is using it, since
> > he is using Fedora).
>
>However, it also doesn't explain what the point is of backing up /dev
>when it's dynamically created.
>
>	Dave

I'm glad you mentioned that, I was, leftovers from RH7.3 probably. :(

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Windows Tip of the Day:
Add DEVICE=FNGRCROS.SYS to your CONFIG.SYS file.

Re: I give up

[Gene Heskett]

On Monday 09 April 2007, Jan Engelhardt wrote:
>On Apr 9 2007 15:38, Gene Heskett wrote:
>>On Monday 09 April 2007, H. Peter Anvin wrote:
>>>Jan Engelhardt wrote:
>>>> dm is on 254 for me.. in opensuse with a 2.6.20 that is. I wonder
>>>> why it even moves around. However, even then, those who use udev and
>>>> device names rather than (major,minor) tuples should not have any
>>>> problem.
>>>
>>>It moves around because someone at some point thought it was a great
>>>idea to assign dynamic majors to core functionality.
>>
>>What were they smoking, I want some of that!
>
>Do you actually use udev?
>
Yes, and it works just fine, for non-LVM filesystems.
>
>
>Jan



-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
"The greatest warriors are the ones who fight for peace."
-- Holly Near

Re: I give up

[John Stoffel]

>>>>> "Gene" == Gene Heskett <gene.heskett@gmail.com> writes:

Gene> On Monday 09 April 2007, Jeff Garzik wrote:
>> Gene Heskett wrote:
>>> Now the 64k$ question: While running with LVM2 managed disks, is it
>>> possible to run without dm_mod, the device-mapper?  If so, please tell
>>> me how to achieve this.
>> 
>> No; device mapper is the kernel portion of LVM2.
>> 
>> Jeff, who actively avoids LVM on home computers

Gene> Ya shoulda warned me.  :-)

Gene> It should have lots bigger warning labels, in bright red
Gene> self-illuminating signs, than it does.  It in fact seems to work
Gene> well, but it is a major breakage for some common apps, like
Gene> doing backups...

>From the GNU info doc on the tar program:

    Metadata stored in snapshot files include device numbers, which,
    obviously is supposed to be a non-volatile value. However, it turns
    out that NFS devices have undependable values when an automounter gets
    in the picture. This can lead to a great deal of spurious redumping in
    incremental dumps, so it is somewhat useless to compare two NFS
    devices numbers over time. The solution implemented currently is to
    considers all NFS devices as being equal when it comes to comparing
    directories; this is fairly gross, but there does not seem to be a
    better way to go.

This to me, seems to be another borkage of the tar program.  Can you
tell Amanda to use 'dump/restore' instead of tar instead?  Or heck,
just move to Bacula, it's smarter than amanda/tar and it supports
writing to DVDs, etc.  http://www.bacula.org

This tar borkness is pretty annoying, since Amanda (from the docs
page) only keeps indexes in terms of host/filesystem/path/date,
nothing about the major/minor (device) numbers.

It just re-inforces my desire to never use Amanda again.

John
john@stoffel.org

Re: I give up

[Gene Heskett]

On Monday 09 April 2007, John Stoffel wrote:
>>>>>> "Gene" == Gene Heskett <gene.heskett@gmail.com> writes:
>
>Gene> On Monday 09 April 2007, Jeff Garzik wrote:
>>> Gene Heskett wrote:
>>>> Now the 64k$ question: While running with LVM2 managed disks, is it
>>>> possible to run without dm_mod, the device-mapper?  If so, please
>>>> tell me how to achieve this.
>>>
>>> No; device mapper is the kernel portion of LVM2.
>>>
>>> Jeff, who actively avoids LVM on home computers
>
>Gene> Ya shoulda warned me.  :-)
>
>Gene> It should have lots bigger warning labels, in bright red
>Gene> self-illuminating signs, than it does.  It in fact seems to work
>Gene> well, but it is a major breakage for some common apps, like
>Gene> doing backups...
>
>>From the GNU info doc on the tar program:
>
>    Metadata stored in snapshot files include device numbers, which,
>    obviously is supposed to be a non-volatile value. However, it turns
>    out that NFS devices have undependable values when an automounter
> gets in the picture. This can lead to a great deal of spurious
> redumping in incremental dumps, so it is somewhat useless to compare
> two NFS devices numbers over time. The solution implemented currently
> is to considers all NFS devices as being equal when it comes to
> comparing directories; this is fairly gross, but there does not seem to
> be a better way to go.
>
>This to me, seems to be another borkage of the tar program.  Can you
>tell Amanda to use 'dump/restore' instead of tar instead?  Or heck,
>just move to Bacula, it's smarter than amanda/tar and it supports
>writing to DVDs, etc.  http://www.bacula.org
>
I wouldn't touch dump/restore with a 50 foot pole, particularly since I 
have serious doubts about it viability in the LVM environment.

For those of you with big tapes that can hold a complete dump of every 
partition (and partitions is the only way dump works in case some have 
forgotten), go ahead and use dump/restore.  Tar quite simply, allows one 
to break his backup files down into small enough pieces that a tape drive 
that's only 20% of the system drives size is totally usable.  I ran dds2 
tapes for a long time, and it wasn't at all unusual to have amanda fill 
those to the 95% or better mark every night for a week running, without 
ever hitting EOT.

>This tar borkness is pretty annoying, since Amanda (from the docs
>page) only keeps indexes in terms of host/filesystem/path/date,
>nothing about the major/minor (device) numbers.
>
>It just re-inforces my desire to never use Amanda again.

Your call, that's what this linux thing is all about.  But do I not recall 
your name from the amanda lists at some point in the last 8+ years?
>
>John
>john@stoffel.org

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
<Culus> OH MY GOD NOT A RANDOM QUOTE GENERATOR
<netgod> surely you didnt think that was static? how lame would that 
be? :-)

Re: I give up

[John Stoffel]

>>>>> "Gene" == Gene Heskett <gene.heskett@gmail.com> writes:

Gene> I wouldn't touch dump/restore with a 50 foot pole, particularly
Gene> since I have serious doubts about it viability in the LVM
Gene> environment.

Why?  It's can't be any worse than Tar with it's silly assumption
about the static device numbers, which are shown in their own docs to
be an invalid assumption over NFS, so why enforce them locally?  

On Linux, dump/restore are open source, so it's not like you can't get
your data back, you know the format it's written in.  Any backup
software has problems with filesystems which change underneath them as
they run, esp for incremental backups.  

The only sure strategy is to do your dumps in single user mode, with
only one process touching the filesystem during the backup window.
This obviously won't fly in a 24x7 environment, so people take their
calculated chances and run backups with the system live.  

Some people are working to be able to take snapshots of live
filesystems, then backups the snapshots, but that's still a work in
progress right now.  I'd personally love to do that, but not quite
yet. 

Gene> For those of you with big tapes that can hold a complete dump of
Gene> every partition (and partitions is the only way dump works in
Gene> case some have forgotten), go ahead and use dump/restore.  Tar
Gene> quite simply, allows one to break his backup files down into
Gene> small enough pieces that a tape drive that's only 20% of the
Gene> system drives size is totally usable. 

So does Bacula.  *grin*  It will happily span backups across multiple
volumes (read tapes).  

The problem with Amanda is that you have to *manually* break down your
filesystem into chunks small enough to fit on a single volume.  I
don't have to think about it with Bacula.

Gene> I ran dds2 tapes for a long time, and it wasn't at all unusual
Gene> to have amanda fill those to the 95% or better mark every night
Gene> for a week running, without ever hitting EOT.

The problem with Amanda for the longest time was that if you had a
full backup which spanned more than one tape, you were screwed.  

>> This tar borkness is pretty annoying, since Amanda (from the docs
>> page) only keeps indexes in terms of host/filesystem/path/date,
>> nothing about the major/minor (device) numbers.
>> 
>> It just re-inforces my desire to never use Amanda again.

Gene> Your call, that's what this linux thing is all about.  But do I
Gene> not recall your name from the amanda lists at some point in the
Gene> last 8+ years?

Probably, but it's been a long time since I was active in Amanda,
probably back in my WPI days (john@wpi.edu) or my Fluent days
(jfs@fluent.com), but I honestly don't remember.

As I said, the big problem with Amanda for the longest time (and which
has now been worked around I understand) was a single backup which had
to span multiple volumes.  

Bacula does this all for you, and works better than Amanda IMHO.  It
also allows you to stage to disk and then to DVD or other types of
volumes.  

And it doesn't have amanda's inistence of writing to a new piece of
media each day either.  Which I admit is a tradeoff, but for a home
system, not having to swap tapes daily is a big help, and saves me
money.  DLTIV tapes aren't cheap, even on ebay...

Heck, if you don't trust Dump, find tar to be broken, what about cpio?
*grin*

John

Re: I give up

[Jeff Garzik]

John Stoffel wrote:
> The only sure strategy is to do your dumps in single user mode, with
> only one process touching the filesystem during the backup window.
> This obviously won't fly in a 24x7 environment, so people take their
> calculated chances and run backups with the system live.  

That's what snapshots are for.

No need for single user mode or taking chances.

A snapshot gives you a consistent filesystem that you can dump or tar.

	Jeff

Re: I give up

[Jeff Garzik]

Gene Heskett wrote:
> For those of you with big tapes that can hold a complete dump of every 
> partition (and partitions is the only way dump works in case some have 
> forgotten), go ahead and use dump/restore.  Tar quite simply, allows one 
> to break his backup files down into small enough pieces that a tape drive 
> that's only 20% of the system drives size is totally usable.  I ran dds2 
> tapes for a long time, and it wasn't at all unusual to have amanda fill 
> those to the 95% or better mark every night for a week running, without 
> ever hitting EOT.


Wow, people still use tapes for backup?

With current hard drive prices (200GB @ US$55, 500GB @ US$120) you can 
just keep buying hard drives :)

Surely tape price/GB is higher than hard drive price/GB...

	Jeff

Re: I give up

[Ralf Baechle]

On Mon, Apr 09, 2007 at 04:08:34PM -0400, Jeff Garzik wrote:

> Wow, people still use tapes for backup?
> 
> With current hard drive prices (200GB @ US$55, 500GB @ US$120) you can 
> just keep buying hard drives :)
> 
> Surely tape price/GB is higher than hard drive price/GB...

Tapes can be archived for long time, drives can't.  Drives need to be
mounted into fairly bulky subsystems, tapes can be easily handled by
robots.

  Ralf

Re: I give up

[Jeff Garzik]

Ralf Baechle wrote:
> Tapes can be archived for long time, drives can't.  Drives need to be

True.


> mounted into fairly bulky subsystems, tapes can be easily handled by
> robots.

Actually there are HD robots now :)

	Jeff

Re: I give up

[H. Peter Anvin]

Jeff Garzik wrote:
> 
>> mounted into fairly bulky subsystems, tapes can be easily handled by
>> robots.
> 
> Actually there are HD robots now :)
> 

The best part is you can just electrify each slot, no need for 
mechanical arms.

	-hpa

Re: I give up

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1063 bytes --]

On Mon, 09 Apr 2007 16:08:34 EDT, Jeff Garzik said:
> With current hard drive prices (200GB @ US$55, 500GB @ US$120) you can 
> just keep buying hard drives :)
> 
> Surely tape price/GB is higher than hard drive price/GB...

Erm. No.  We're in the middle of installing an StorageTek SL8500 for backups,
simply because an LTO3 tape holds 300G at a price point of $100 or so.
And there's no sane way to build a half-petabyte disk farm at that price.
(And yes, our tape backup service is well into the half-petabyte range, and
will probably shoot over that as soon as the SL8500 is in production and we
deal with all the backlogged requests for backup service).

Don't forget to factor in the cost of disk shelves, power, cooling, and all
that when you're building something to hold 2,500 200G drives.  Oh. and
controllers.  And machines to put the controllers in... and all the rest of it.

And remember - you *dont* want to be backing up critical data on the sort of
hard drives that cost $55 for 200G - you'll want multiple copies, probably
some RAID, etc etc.  

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: I give up

[Jeff Garzik]

Valdis.Kletnieks@vt.edu wrote:
> On Mon, 09 Apr 2007 16:08:34 EDT, Jeff Garzik said:
>> With current hard drive prices (200GB @ US$55, 500GB @ US$120) you can 
>> just keep buying hard drives :)
>>
>> Surely tape price/GB is higher than hard drive price/GB...
> 
> Erm. No.  We're in the middle of installing an StorageTek SL8500 for backups,
> simply because an LTO3 tape holds 300G at a price point of $100 or so.
> And there's no sane way to build a half-petabyte disk farm at that price.
> (And yes, our tape backup service is well into the half-petabyte range, and
> will probably shoot over that as soon as the SL8500 is in production and we
> deal with all the backlogged requests for backup service).
> 
> Don't forget to factor in the cost of disk shelves, power, cooling, and all
> that when you're building something to hold 2,500 200G drives.  Oh. and
> controllers.  And machines to put the controllers in... and all the rest of it.

Who says you have to keep the hard drives plugged into a machine at all 
times?


> And remember - you *dont* want to be backing up critical data on the sort of
> hard drives that cost $55 for 200G - you'll want multiple copies, probably
> some RAID, etc etc.  

The same can be said for tape.  Anyone who only has a single tape backup 
of critical data is an idiot.  Plenty of war stories where such 
scenarios go awry...

	Jeff

Re: I give up

[Gene Heskett]

On Monday 09 April 2007, Jeff Garzik wrote:
>Gene Heskett wrote:
>> For those of you with big tapes that can hold a complete dump of every
>> partition (and partitions is the only way dump works in case some have
>> forgotten), go ahead and use dump/restore.  Tar quite simply, allows
>> one to break his backup files down into small enough pieces that a
>> tape drive that's only 20% of the system drives size is totally
>> usable.  I ran dds2 tapes for a long time, and it wasn't at all
>> unusual to have amanda fill those to the 95% or better mark every
>> night for a week running, without ever hitting EOT.
>
>Wow, people still use tapes for backup?
>
>With current hard drive prices (200GB @ US$55, 500GB @ US$120) you can
>just keep buying hard drives :)
>
>Surely tape price/GB is higher than hard drive price/GB...
>
>	Jeff
I haven't seen any 200GB for $55 yet, more like $129 & maybe a rebate at 
Circuit City.  We don't have a Fry's around here.


-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
You will be called upon to help a friend in trouble.

Re: I give up

[John Stoffel]

Gene> I haven't seen any 200GB for $55 yet, more like $129 & maybe a
Gene> rebate at Circuit City.  We don't have a Fry's around here.

Newegg.com, 320Gb for $85 ea, plus shipping, plus a SATA controller
board, just under $200.  I'm happy.  And thanks for the SATA
controller work Jeff!

John

Re: I give up

[Jeff Garzik]

Gene Heskett wrote:
> I haven't seen any 200GB for $55 yet, more like $129 & maybe a rebate at 
> Circuit City.  We don't have a Fry's around here.


pricewatch.com is your friend :)

	Jeff

Re: I give up

[CaT]

On Mon, Apr 09, 2007 at 11:34:44PM -0400, Gene Heskett wrote:
> I haven't seen any 200GB for $55 yet, more like $129 & maybe a rebate at 
> Circuit City.  We don't have a Fry's around here.

Wow. 200GB HDs can be had for AUD91 here. I think you need to shop
around. The internet can be your friend. :)

-- 
    "To the extent that we overreact, we proffer the terrorists the
    greatest tribute."
    	- High Court Judge Michael Kirby

Re: I give up

[H. Peter Anvin]

Gene Heskett wrote:
> I haven't seen any 200GB for $55 yet, more like $129 & maybe a rebate at 
> Circuit City.  We don't have a Fry's around here.

Fry's:
http:/http://shop1.outpost.com/product/4697788?site=sr:SEARCH:MAIN_RSLT_PG

500 GB SATA for $119.  Pricewatch shows several other vendors having the 
same pricing.

	-hpa

Re: I give up

[Gene Heskett]

On Tuesday 10 April 2007, H. Peter Anvin wrote:
>Gene Heskett wrote:
>> I haven't seen any 200GB for $55 yet, more like $129 & maybe a rebate
>> at Circuit City.  We don't have a Fry's around here.
>
>Fry's:
>http:/http://shop1.outpost.com/product/4697788?site=sr:SEARCH:MAIN_RSLT_
>PG
>
>500 GB SATA for $119.  Pricewatch shows several other vendors having the
>same pricing.
>
>	-hpa

I got a 320GB Maxtor, 3 year warranty, at Staples tonight for $90 & 
change.  It could have been worse locally.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
QOTD:
	"I'll listen to reason when it comes out on CD."