BK->CVS, kernel.bkbits.net

BK->CVS, kernel.bkbits.net

[Larry McVoy]

It's back up, and the CVS server up to date with the 2.4 2.5 kernels as
of a few minutes ago.  The CVS server is at

:pserver:anonymous@kernel.bkbits.net:/home/cvs 

There are linux-2.4/ and linux-2.5/ subdirectories there (should this go in
a FAQ someplace or does nobody except Andrea care?).
-- 
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm

Re: BK->CVS, kernel.bkbits.net

[H. Peter Anvin]

Followup to:  <20030417162723.GA29380@work.bitmover.com>
By author:    Larry McVoy <lm@bitmover.com>
In newsgroup: linux.dev.kernel
>
> It's back up, and the CVS server up to date with the 2.4 2.5 kernels as
> of a few minutes ago.  The CVS server is at
> 
> :pserver:anonymous@kernel.bkbits.net:/home/cvs 
> 
> There are linux-2.4/ and linux-2.5/ subdirectories there (should this go in
> a FAQ someplace or does nobody except Andrea care?).
> 

It definitely should.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64

Re: BK->CVS, kernel.bkbits.net

[Larry McVoy]

On Thu, Apr 17, 2003 at 01:52:30PM -0700, H. Peter Anvin wrote:
> Followup to:  <20030417162723.GA29380@work.bitmover.com>
> By author:    Larry McVoy <lm@bitmover.com>
> In newsgroup: linux.dev.kernel
> >
> > It's back up, and the CVS server up to date with the 2.4 2.5 kernels as
> > of a few minutes ago.  The CVS server is at
> > 
> > :pserver:anonymous@kernel.bkbits.net:/home/cvs 
> > 
> > There are linux-2.4/ and linux-2.5/ subdirectories there (should this go in
> > a FAQ someplace or does nobody except Andrea care?).
> 
> It definitely should.

OK, so how about this?  I assume you manage DNS for kernel.org, right?
How about a DNS entry for cvs.kernel.org -> 64.241.2.13?  If you ever
find a machine to host this then you already own cvs.kernel.org and you
can just reset the address.  By the way, I think the bandwidth is pretty
darn low, after all that fuss almost nobody seems to use this, it just
gives them warm fuzzies to know that the history has been captured in
an open format which is worth it if it means no more BK flame wars, eh?

Then whoever maintains the kernel FAQ these days could add something like
this:

SCM access to the kernel trees:
-------------------------------

Linus started using an SCM (source code management) tool called BitKeeper
in February of 2002.  Since BitKeeper isn't free software, he does not
require that anyone else use BitKeeper, he continues to accept patches
just like he always did.  The only difference is that information about
who did what, and maybe why they did it, is recorded and is useful for
learning the source base, tracking down bugs, etc.  Many, but not all,
of the core developers have switched to using BitKeeper because it makes
their life easier in various ways.

Some people haven't switched because BitKeeper isn't free software and
they feel uncomfortable using non-free software as part of working on
the kernel.  That's fine, it's an explicit goal of both Linus and the
BitKeeper developers that nobody is required to use BitKeeper to work
on the kernel.  Some senior developers have decided they'd rather
not use BitKeeper, Alan Cox being a good example.  That's not a problem,
the BitKeeper developers worked with Linus to streamline the importing
of traditional patches so that anyone can work in any way they see fit.

If you want to use BitKeeper (http://www.bitkeeper.com) then the official
trees are maintained on linux.bkbits.net - to get a particular release
try this:

	bk clone bk://linux.bkbits.net/linux-2.4

There was a fair amount of fuss amongst the free software purists,
over the fact that a lot of information that was available in BitKeeper
was lost when Linus provided the traditional tarball releases and patch
updates.  Flame wars happened and when the dust settled, the BitKeeper
folks built a BitKeeper to CVS gateway which captures the bulk of the
information (as of this writing on April 19th 2003 there are 9,311
snapshots captured).  If you would prefer to get your source with 100%
God fearing, politically correct, open source, fully buzzword enabled
software, then you can do this:

	cvs -d:pserver:anonymous@cvs.kernel.org:/home/cvs co linux-2.4

As releases progress, the release numbers will change so some day you might
say

	bk clone bk://linux.bkbits.net/linux-4.2
or
	cvs -d:pserver:anonymous@cvs.kernel.org:/home/cvs co linux-4.2

-- 
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm

Re: BK->CVS, kernel.bkbits.net

[Ben Collins]

On Thu, Apr 17, 2003 at 09:27:23AM -0700, Larry McVoy wrote:
> It's back up, and the CVS server up to date with the 2.4 2.5 kernels as
> of a few minutes ago.  The CVS server is at
> 
> :pserver:anonymous@kernel.bkbits.net:/home/cvs 
> 
> There are linux-2.4/ and linux-2.5/ subdirectories there (should this go in
> a FAQ someplace or does nobody except Andrea care?).

I hate asking this on top of the work you already provide, but would it
be possible to allow rsync access to the repo itself? I have atleast 6
computers on my LAN where I keep source trees (2.4 and 2.5), and it
would be much less b/w on my metered T1 and on your link aswell if I
could rsync one main "mirror" of the cvs repo and then point all my
machines at it.

I do a lot of diff'ing and log reading, so it would help out there too
if I didn't have to connect back to bkbits to perform those frequent
operations.

No biggy if not.

-- 
Debian     - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
Subversion - http://subversion.tigris.org/
Deqo       - http://www.deqo.com/

Re: BK->CVS, kernel.bkbits.net

[Larry McVoy]

> I hate asking this on top of the work you already provide, but would it
> be possible to allow rsync access to the repo itself? 

If HPA wants to provide that, that's cool.  I think he might already.
If not, ping me again, no problem, we'll set something up.
-- 
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm

Re: BK->CVS, kernel.bkbits.net

[Shachar Shemesh]

Ben Collins wrote:

>I hate asking this on top of the work you already provide, but would it
>be possible to allow rsync access to the repo itself? I have atleast 6
>computers on my LAN where I keep source trees (2.4 and 2.5), and it
>would be much less b/w on my metered T1 and on your link aswell if I
>could rsync one main "mirror" of the cvs repo and then point all my
>machines at it.
>
There is a better tool (for this particular task), called "cvsup". It 
does a wonderful job of keeping cvs repositories in synch. I realize I 
just asked for a THIRD tool, so it should only go in if the admins are 
willing to take care of it.

The idea is that it uses the full duplexity of the channel to get client 
side information about the repository on that end while downloading 
changes, thus increasing the effective bandwidth. It only falls back to 
rsynch if CVS repository specific updates are not possible. I use it on 
the Wine repository, and it does, indeed, work very efficiently.

On the negative side - as far as I could tell, neither RedHat nor 
Mandrake carry it as a standard package (Debian does, at least in unstable).

             Shachar

-- 
Shachar Shemesh
Open Source integration consultant
Home page & resume - http://www.shemesh.biz/

Re: BK->CVS, kernel.bkbits.net

[Geert Uytterhoeven]

On Sun, 20 Apr 2003, Shachar Shemesh wrote:
> Ben Collins wrote:
> >I hate asking this on top of the work you already provide, but would it
> >be possible to allow rsync access to the repo itself? I have atleast 6
> >computers on my LAN where I keep source trees (2.4 and 2.5), and it
> >would be much less b/w on my metered T1 and on your link aswell if I
> >could rsync one main "mirror" of the cvs repo and then point all my
> >machines at it.
> >
> There is a better tool (for this particular task), called "cvsup". It 
> does a wonderful job of keeping cvs repositories in synch. I realize I 
> just asked for a THIRD tool, so it should only go in if the admins are 
> willing to take care of it.
> 
> The idea is that it uses the full duplexity of the channel to get client 
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> side information about the repository on that end while downloading 
> changes, thus increasing the effective bandwidth. It only falls back to 

What does this mean for asymmetric links (ADSL or cable)?

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: BK->CVS, kernel.bkbits.net

[Ben Collins]

On Sun, Apr 20, 2003 at 10:32:08AM +0300, Shachar Shemesh wrote:
> Ben Collins wrote:
> 
> >I hate asking this on top of the work you already provide, but would it
> >be possible to allow rsync access to the repo itself? I have atleast 6
> >computers on my LAN where I keep source trees (2.4 and 2.5), and it
> >would be much less b/w on my metered T1 and on your link aswell if I
> >could rsync one main "mirror" of the cvs repo and then point all my
> >machines at it.
> >
> There is a better tool (for this particular task), called "cvsup". It 
> does a wonderful job of keeping cvs repositories in synch. I realize I 
> just asked for a THIRD tool, so it should only go in if the admins are 
> willing to take care of it.

How does cvsup help when I have 6 copies of two different repositories
on my side and I only want to hit the other side one time to update all
6 copies?

-- 
Debian     - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
Subversion - http://subversion.tigris.org/
Deqo       - http://www.deqo.com/

Re: BK->CVS, kernel.bkbits.net

[Roman Zippel]

Hi,

On Sat, 19 Apr 2003, Larry McVoy wrote:

> Some people haven't switched because BitKeeper isn't free software and
> they feel uncomfortable using non-free software as part of working on
> the kernel.

You forgot to mention that some people are not allowed to use bk (without 
paying) and some people also don't like the invasion of privacy (unless 
they pay).

> There was a fair amount of fuss amongst the free software purists,
> over the fact that a lot of information that was available in BitKeeper
> was lost when Linus provided the traditional tarball releases and patch
> updates.

This has never been an issue, the complete version history is only 
available to bk users. This makes it difficult for other SCM developers to 
assist users in easy exchange of information with bk users or converting 
their information from bk into other formats.

>  Flame wars happened and when the dust settled, the BitKeeper
> folks built a BitKeeper to CVS gateway which captures the bulk of the
> information

, because only the bk folks have the tools and the license to produce such 
a product.

bye, Roman

Re: BK->CVS, kernel.bkbits.net

[Shachar Shemesh]

Ben Collins wrote:

>>>I hate asking this on top of the work you already provide, but would it
>>>be possible to allow rsync access to the repo itself? I have atleast 6
>>>computers on my LAN where I keep source trees (2.4 and 2.5), and it
>>>would be much less b/w on my metered T1 and on your link aswell if I
>>>could rsync one main "mirror" of the cvs repo and then point all my
>>>machines at it.
>>>      
>>>
>How does cvsup help when I have 6 copies of two different repositories
>on my side and I only want to hit the other side one time to update all
>6 copies?
>  
>
"cvsup" is for synching repositories (I was not talking about "cvs up" - 
the command line). It achives the exact same end effect as rsync, except 
it is much more bandwidth efficient when used to sync CVS repositories. 
Homepage at http://www.cvsup.org/.

As Adam Richter said in private, however, the tool is a bitch to 
compile. It is written in Modula-3, and most people don't have the 
development environment to build it. Add to that the fact that most 
distros don't carry it as a package (a while back I tried, 
unsuccessfully, to locate an RPM for it, anywhere), and you get 
something that should be deployed with care.

On the other hand, both Wine (where I got to know it) and KDE seem to 
offer cvsup for getting the repository, so it can't be THAT difficult. 
As also noted above, Debian does carry it in easy to deploy .deb, as 
part of the main distro's archive (confirmed available on stable).

          Sh.

-- 
Shachar Shemesh
Open Source integration consultant
Home page & resume - http://www.shemesh.biz/

Re: BK->CVS, kernel.bkbits.net

[viro]

On Sun, Apr 20, 2003 at 04:37:09PM +0300, Shachar Shemesh wrote:
 
> On the other hand, both Wine (where I got to know it) and KDE seem to 
> offer cvsup for getting the repository, so it can't be THAT difficult. 
> As also noted above, Debian does carry it in easy to deploy .deb, as 
> part of the main distro's archive (confirmed available on stable).

Debian carries it for i386.  Modula 3 _is_ a bitch to bootstrap and most
of architectures simply do not bother.

Re: BK->CVS, kernel.bkbits.net

[Shachar Shemesh]

Geert Uytterhoeven wrote:

>On Sun, 20 Apr 2003, Shachar Shemesh wrote:
>  
>
>>The idea is that it uses the full duplexity of the channel to get client 
>>    
>>
>                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>  
>
>>side information about the repository on that end while downloading 
>>changes, thus increasing the effective bandwidth. It only falls back to 
>>    
>>
>
>What does this mean for asymmetric links (ADSL or cable)?
>
>Gr{oetje,eeting}s,
>
>						Geert
>  
>
ADSL is still full duplex, just not symetrical.

If I understand cvsup's operation enough, it uses the fact it 
understands what a CVS repository is to send to the server the revisions 
available for a given file. The lets the server know which parts of the 
file it needs to send back. The uplink side receives a very low 
utilization compared to the downlink side. In practice, I'm using cvsup 
for the Wine repository over an ADSL (1.5M down, I don't remeber whether 
it's 64 or 128K up), and am very pleased from it. Admitebly, I was not a 
very enthusiastic rsync convert, so I can't tell you how much faster 
cvsup is.

If you want an official benchmark, you'll have to wait a few days for my 
Wine rep. to fall out of synch. I should note the cvsup is useless if 
all your'e going to do is get the initial version. If I recall 
correctly, it actually use rsync to transfer files it cannot parse as 
CVS files, which means that initial repository retrieval should be 
equally fast with both.

-- 
Shachar Shemesh
Open Source integration consultant
Home page & resume - http://www.shemesh.biz/

Re: BK->CVS, kernel.bkbits.net

[Ben Collins]

> On the other hand, both Wine (where I got to know it) and KDE seem to 
> offer cvsup for getting the repository, so it can't be THAT difficult. 
> As also noted above, Debian does carry it in easy to deploy .deb, as 
> part of the main distro's archive (confirmed available on stable).

If only i386 was my only development environment. Add sparc64 as my
primary and hppa, ia64, mips, i386, arm and powerpc as secondaries.

CVSup is only available on i386 because of the compiler from what I can
see.

-- 
Debian     - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
Subversion - http://subversion.tigris.org/
Deqo       - http://www.deqo.com/

Re: BK->CVS, kernel.bkbits.net

[Shachar Shemesh]

Ben Collins wrote:

>If only i386 was my only development environment. Add sparc64 as my
>primary and hppa, ia64, mips, i386, arm and powerpc as secondaries.
>  
>
I don't see why that matters to you. You only need one cvsup able 
machine to sync the local repository. Once it's local, you can check out 
any tree from it using CVS as usual. There is no difference between 
cvsup and rsync in that respect. I'm not sure where and how the sparc 
tree is held, as I understand it's not in the main tree (but is it in 
the main repository?).

In any case, I'm not even sure that cvsup CAN run without an underling 
rsync server. I simply don't know. I do know that cvsup uses rsync for 
files it cannot optimize using its own algorithm.

-- 
Shachar Shemesh
Open Source integration consultant
Home page & resume - http://www.shemesh.biz/

Re: BK->CVS, kernel.bkbits.net

[Shachar Shemesh]

Ben Collins wrote:

>CVSup is only available on i386 because of the compiler from what I can
>see.
>  
>
The site offers binary images for download for FreeBSD and Digital Unix 
(Alpha), and Solaris Sparc. It is therefor unlikely that this is a 
problem with lack of development tools. More probably - the maintainers 
did not have these platforms available to them.

-- 
Shachar Shemesh
Open Source integration consultant
Home page & resume - http://www.shemesh.biz/

Re: BK->CVS, kernel.bkbits.net

[Wichert Akkerman]

Previously Shachar Shemesh wrote:
> The site offers binary images for download for FreeBSD and Digital Unix 
> (Alpha), and Solaris Sparc. It is therefor unlikely that this is a 
> problem with lack of development tools. More probably - the maintainers 
> did not have these platforms available to them.

Are you offering to bootstrap Modula-3 on other Linux architectures?

Wichert.

-- 
Wichert Akkerman <wichert@wiggy.net>           http://www.wiggy.net/
A random hacker

Re: BK->CVS, kernel.bkbits.net

[Shachar Shemesh]

Wichert Akkerman wrote:

>Previously Shachar Shemesh wrote:
>  
>
>>The site offers binary images for download for FreeBSD and Digital Unix 
>>(Alpha), and Solaris Sparc. It is therefor unlikely that this is a 
>>problem with lack of development tools. More probably - the maintainers 
>>did not have these platforms available to them.
>>    
>>
>
>Are you offering to bootstrap Modula-3 on other Linux architectures?
>
>Wichert.
>
>  
>
Unfortunetly, I am as swamped as many other here. (I don't have any 
non-Intel platform available to me even if I did have the time, except 
perhaps a 68000 Amiga 500, which may or may not still be in working 
order, but defenitely has no networking). I am saying that cvsup IN 
ADDITION to the rsync will be nice. If the site administrators can't 
maintain it, rsync is preferable for the obvious portability reasons..

Not trying to cause anyone any extra work. Sorry for the noise.

-- 
Shachar Shemesh
Open Source integration consultant
Home page & resume - http://www.shemesh.biz/

Re: BK->CVS, kernel.bkbits.net

[Larry McVoy]

On Sun, Apr 20, 2003 at 03:16:16PM +0200, Roman Zippel wrote:
[trolled, looking for another BK war]

Don't feed the trolls, please, we're all tired of it.
-- 
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm

Re: BK->CVS, kernel.bkbits.net

[Roman Zippel]

Hi,

On Sun, 20 Apr 2003, Larry McVoy wrote:

> [trolled, looking for another BK war]
> 
> Don't feed the trolls, please, we're all tired of it.

Huh? I'm the last one who is interested in a flame war, I just provided 
the missing information you omitted. If I said something wrong, it should 
be no problem to correct me, but it seems you can't, so you try to 
discredit me as troll instead.

bye, Roman

Re: BK->CVS, kernel.bkbits.net

[viro]

On Sun, Apr 20, 2003 at 05:42:03PM +0300, Shachar Shemesh wrote:
> Ben Collins wrote:
> 
> >CVSup is only available on i386 because of the compiler from what I can
> >see.
> > 
> >
> The site offers binary images for download for FreeBSD and Digital Unix 
> (Alpha), and Solaris Sparc. It is therefor unlikely that this is a 
> problem with lack of development tools. More probably - the maintainers 
> did not have these platforms available to them.

Care to take a look at the bootstrap procedures for Modula 3 compiler?

Re: BK->CVS, kernel.bkbits.net

[Neil Brown]

On Saturday April 19, lm@bitmover.com wrote:
>                              By the way, I think the bandwidth is pretty
> darn low, after all that fuss almost nobody seems to use this, it just
> gives them warm fuzzies to know that the history has been captured in
> an open format which is worth it if it means no more BK flame wars, eh?


Well, I just became a big fan:

% time bk pull
....
444.95user 42.29system 49:09.46elapsed 16%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (326737major+196385minor)pagefaults 0swaps


% time cvs update
.....
2.78user 1.94system 4:12.36elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (333major+7240minor)pagefaults 0swaps


That is an order of magnitude difference in wall-clock time!  This is
on my humble notebook with "only" 128Meg of RAM.  The delay is mostly 
in the consistency checking.  Sure there is a way to turn that off.

NeilBrown

(I only used bk to
   "bk tag LATEST  ; bk pull; bk export -tpatch -rLATEST, > file"
 and cvs will allow the same end result)

Re: BK->CVS, kernel.bkbits.net

[Jeff Garzik]

Neil Brown wrote:
> That is an order of magnitude difference in wall-clock time!  This is
> on my humble notebook with "only" 128Meg of RAM.  The delay is mostly 
> in the consistency checking.  Sure there is a way to turn that off.


Yeah, my laptop w/ 128M is the same.  I think you need at least 256M for 
vaguely usable caching, and preferably 512M or more :)

	Jeff

Re: BK->CVS, kernel.bkbits.net

[Jamie Lokier]

Larry McVoy wrote:
> Some people haven't switched because BitKeeper isn't free software
> and they feel uncomfortable using non-free software as part of
> working on the kernel.

That's misleading.  I would be comfortable using it, but was politely
requested not to.

To avoid misleading, you should add this after the above quoted
sentence, or something similar:

	Some other people are unable to use the gratis version of
	BitKeeper due to licensing restrictions, and are unwilling to
	pay for the commercial edition.

> If you would prefer to get your source with 100% God fearing,
                                                   ~~~~~~~~~~~
> politically correct, open source, fully buzzword enabled software,
> then you can do this:

Please note that St.Ignucius of the Church of Emacs is officially an Atheist :)

-- Jamie

Re: BK->CVS, kernel.bkbits.net

[H. Peter Anvin]

Followup to:  <20030420014930.GA13699@work.bitmover.com>
By author:    Larry McVoy <lm@bitmover.com>
In newsgroup: linux.dev.kernel
>
> > I hate asking this on top of the work you already provide, but would it
> > be possible to allow rsync access to the repo itself? 
> 
> If HPA wants to provide that, that's cool.  I think he might already.
> If not, ping me again, no problem, we'll set something up.
> 

rsync://rsync.kernel.org/pub/scm/linux/kernel/bkcvs/linux-2.[45]/

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64

Re: BK->CVS, kernel.bkbits.net

[Ben Collins]

On Mon, Apr 21, 2003 at 11:58:18AM -0700, H. Peter Anvin wrote:
> Followup to:  <20030420014930.GA13699@work.bitmover.com>
> By author:    Larry McVoy <lm@bitmover.com>
> In newsgroup: linux.dev.kernel
> >
> > > I hate asking this on top of the work you already provide, but would it
> > > be possible to allow rsync access to the repo itself? 
> > 
> > If HPA wants to provide that, that's cool.  I think he might already.
> > If not, ping me again, no problem, we'll set something up.
> > 
> 
> rsync://rsync.kernel.org/pub/scm/linux/kernel/bkcvs/linux-2.[45]/

Thanks, in a big way.

-- 
Debian     - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
Subversion - http://subversion.tigris.org/
Deqo       - http://www.deqo.com/

Re: BK->CVS, kernel.bkbits.net

[Gerd Knorr]

Shachar Shemesh <lkml@shemesh.biz> writes:

> development environment to build it. Add to that the fact that most
> distros don't carry it as a package (a while back I tried,
> unsuccessfully, to locate an RPM for it, anywhere), and you get
> something that should be deployed with care.

There are rpms for suse 8.1 on ftp.suse.com (i386 only for obvious
reasons).

BTW: With glibc 2.3 the modula-3 compiler bootstrap broke (due to
thread code changes as far I can see, I'm no m3 guru ...), so with
very recent linux distributions you likely have problems to build the
thing even on i386 (unless someone has fixed that in the meantime).
Old cvsup binaries continue to work through.

> On the other hand, both Wine (where I got to know it) and KDE seem to
> offer cvsup for getting the repository, so it can't be THAT
> difficult.

xfree86 + freebsd use it too.

  Gerd

-- 
Michael Moore for president!

Re: BK->CVS, kernel.bkbits.net

[Carl-Daniel Hailfinger]

Neil Brown wrote:
> % time bk pull
> ....
> 444.95user 42.29system 49:09.46elapsed 16%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (326737major+196385minor)pagefaults 0swaps
> 
> 
> % time cvs update
> .....
> 2.78user 1.94system 4:12.36elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (333major+7240minor)pagefaults 0swaps
> 
> 
> That is an order of magnitude difference in wall-clock time!  This is
> on my humble notebook with "only" 128Meg of RAM.  The delay is mostly 
> in the consistency checking.  Sure there is a way to turn that off.

Just add this line to your /etc/BitKeeper/etc/config:
[]partial_check:yes!

and you should notice a big speedup.

HTH,
Carl-Daniel

P.S. If anyone knows other speedup tricks for a kernel tree in bk,
please tell me.
-- 
http://www.hailfinger.org/

Re: BK->CVS, kernel.bkbits.net

[Larry McVoy]

On Wed, Apr 23, 2003 at 05:49:47PM +0200, Carl-Daniel Hailfinger wrote:
> > % time bk pull
> > ....
> > 444.95user 42.29system 49:09.46elapsed 16%CPU (0avgtext+0avgdata 0maxresident)k
> > 0inputs+0outputs (326737major+196385minor)pagefaults 0swaps
> > 
> > 
> > % time cvs update
> > .....
> > 2.78user 1.94system 4:12.36elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k
> > 0inputs+0outputs (333major+7240minor)pagefaults 0swaps

Fast or safe, pick one.  CVS has no integrity check and you will never know
if you have bad data or not.  And the BK checks find el cheapo memory dimms
and all sorts of other problems all the time.  It even found a cache aliasing
bug in SPARC/Linux...

The BK integrity check will tell you right away if any of your data is bad.
*Everyone* hates the check until it saves their butt and then they decide
it's not such a bad idea.  It's a lot like a seatbelt - you don't like it
until something goes wrong.

BK != CVS.  You want fast and loose, by all means, use CVS, that's not our
intended market and we don't care about fast where fast means bad data.

> > That is an order of magnitude difference in wall-clock time!  This is
> > on my humble notebook with "only" 128Meg of RAM.  The delay is mostly 
> > in the consistency checking.  Sure there is a way to turn that off.
> 
> Just add this line to your /etc/BitKeeper/etc/config:
> []partial_check:yes!
> 
> and you should notice a big speedup.
> 
> P.S. If anyone knows other speedup tricks for a kernel tree in bk,
> please tell me.

Mount the file system with noatime.

Buy enough memory to fit the kernel in memory and on a 1Ghz processor that
pull will take about 20 seconds.  Even laptop memory is pretty cheap these
days.  Pricewatch says $80 for .5GB for laptops, that's cheap.
-- 
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm

Re: BK->CVS, kernel.bkbits.net

[Christoph Hellwig]

On Wed, Apr 23, 2003 at 07:45:49PM -0700, Larry McVoy wrote:
> Fast or safe, pick one.  CVS has no integrity check and you will never know
> if you have bad data or not.  And the BK checks find el cheapo memory dimms
> and all sorts of other problems all the time.  It even found a cache aliasing
> bug in SPARC/Linux...
> 
> The BK integrity check will tell you right away if any of your data is bad.
> *Everyone* hates the check until it saves their butt and then they decide
> it's not such a bad idea.  It's a lot like a seatbelt - you don't like it
> until something goes wrong.
> 
> BK != CVS.  You want fast and loose, by all means, use CVS, that's not our
> intended market and we don't care about fast where fast means bad data.

Well, 90% of the BK repositories are in read-only mode for me, i.e. just
mirros of some public repository.  I couldn't care less whether a
corruption sneaked in, I'll just reclone as soon as the mainers complains
my patches don't apply anymore :)  So putting this get faster hints
somewhere where they could be found easily (or even a go fast option
for bk clone that applies this..) would be really nice.

Re: BK->CVS, kernel.bkbits.net

[Carl-Daniel Hailfinger]

Larry McVoy wrote:
> On Wed, Apr 23, 2003 at 05:49:47PM +0200, Carl-Daniel Hailfinger wrote:
> 
> Fast or safe, pick one.  CVS has no integrity check and you will never know

I picked bitkeeper because it is convenient, easy to use, has a
distributed structure and keeps revision history locally.

> [description of feats achieved by integrity checking snipped]
> 
> The BK integrity check will tell you right away if any of your data is bad.

Every once in a while (cron job at night), I do a
bk -r check -ac
According to the man page, that should cover me nicely. Files which
change during a bk pull are checked anyway, so the only possible
corruption remaining undetected for a few hours is corruption in files
which were not modified at all. I just don't see the BIG benefit in
knowing a few hours earlier that an unused file is corrupted.

If corruption does not affect your working set, it is nice to know about
(to get rid of the offending hardware), but it will not harm you
directly. Checking readonly files is a job for cron because it is highly
unlikely that corruption of your on-disk data will happen on read. I'ts
either already corrupt on the platter or not (cabling/RAM etc. issues
aside), READING a sector from disk should NEVER change the contents of
this sector. If it does, you have a much bigger problem.

> *Everyone* hates the check until it saves their butt and then they decide
> it's not such a bad idea.  It's a lot like a seatbelt - you don't like it
> until something goes wrong.

You should introduce periodic checks, then. Data corruption is not a
function of the read count of a specific sector, but of the write count
and/or time. Feel free to point me to contrary evidence, I'm always
willing to learn.

> BK != CVS.  You want fast and loose, by all means, use CVS, that's not our
> intended market and we don't care about fast where fast means bad data.
> 
> 
>>P.S. If anyone knows other speedup tricks for a kernel tree in bk,
>>please tell me.
> 
> 
> Mount the file system with noatime.

Did that already. But thanks for the suggestion anyway.

> Buy enough memory to fit the kernel in memory and on a 1Ghz processor that
> pull will take about 20 seconds.  Even laptop memory is pretty cheap these
> days.  Pricewatch says $80 for .5GB for laptops, that's cheap.

With a upper maximum of 192MB in my laptop (the chipset won't accept
more), I have no choice but to buy a new one or enable partial checks. I
even use ext2 instead of reiser/ext3 on my notebook to save RAM. The
only thing running during a bk pull is XFree86 and fvwm2 as windowmanager.

Unfortunately, I was unable to find a good laptop in the 80$ range;
heck, I would even consider a good used laptop in the 300$ range. And
before you ask: as an university student, I think twice before shelling
out money for a new machine. Maybe that will change if I have gobs of
money available.

Don't understand me wrong, bitkeeper is a tool which has saved me hours
of merging and debugging mismerges, but I'd perefer not to use xiafs
instead of ext2 just to save a couple more kB of RAM to speed up bk.

Regards,
Carl-Daniel

-- 
Carl-Daniel Hailfinger
Xiafs maintainer
More about Xiafs soon at
http://www.hailfinger.org/