2.6.0 stability and the BK scsi trees

2.6.0 stability and the BK scsi trees

[James Bottomley]

In the light of the recently announced "bug fixes only" edict for the
2.6 kernel, I'll be splitting our current SCSI BK tree:

bk://linux-scsi.bkbits.net/scsi-misc-2.5

Into two trees.  One:

bk://linux-scsi.bkbits.net/scsi-bugfixes-2.6

for upstream inclusion, and the other

bk://linux-scsi.bkbits.net/scsi-misc-2.7

To hold all the other stuff for eventual inclusion in 2.7.

I'll be on Holiday until 22 October, but I'll try to do this shortly
after I get back home.

If you send any patches to the SCSI list in the mean time, could you
make sure you label them "bug fix" (and make sure they really are) if
they are for immediate inclusion.

Thanks,

James

Re: 2.6.0 stability and the BK scsi trees

[Douglas Gilbert]

James Bottomley wrote:
> In the light of the recently announced "bug fixes only" edict for the
> 2.6 kernel, I'll be splitting our current SCSI BK tree:

Ok, I'll bite:-)

What was the point of putting 32 dev_t's into the
kernel? Many people who were advocating it used
the increased number of scsi disks (> 256) and
partitions (from 15 to 63 [to match the ide subsystem])
as a major reason.

The sd driver is still littered with hacks to distribute
its 256 (max) disks over 8 majors. Shouldn't this be
fixed?

Comments?

Doug Gilbert

Re: 2.6.0 stability and the BK scsi trees

[Matthew Wilcox]

On Thu, Oct 16, 2003 at 09:56:54PM +1000, Douglas Gilbert wrote:
> What was the point of putting 32 dev_t's into the
> kernel? Many people who were advocating it used
> the increased number of scsi disks (> 256) and
> partitions (from 15 to 63 [to match the ide subsystem])
> as a major reason.
> 
> The sd driver is still littered with hacks to distribute
> its 256 (max) disks over 8 majors. Shouldn't this be
> fixed?

Well, let's see some fixes and then decide whether it's worth merging
before 2.6.0 or after 2.6.0.  We can be sure that vendors will integrate
this patch even if it's not in mainline kernel.org ... and i'd rather
see one variant of the patch which everybody uses than a different one
in RH, SuSE and Linux 2.7.

Here's some things that need to be done ...

 * Maintain backward compatibility for NFS's purposes and on-disc
   representations of /dev.  ie we still need to keep the major numbers
   the same, no question.
 * Come up with a sensible naming scheme.  Is everybody happy with
   /dev/sdabc for discs after #702?  And /dev/sdabcd for discs after #18954?
 * Decide whether we want to sacrifice two bits for increasing the number
   of partitions beyond 15.  I've never personally seen more than 11
   partitions on a single device, and that was a very strange partitioning
   scheme involving mixed BSD disclabels and FAT.
 * If we're seriously considering allocating thousands of scsi discs, I
   guess we'll want a custom slab for them rather than just using kmalloc.
 * Need to get rid of that index bitmap.

We have 16 majors for SCSI currently (and, as discussed above, they need
to be kept the same).  We have 20 bits of minor per major, so in total
we have 16 million minor numbers available to use for discs & their
partitions.  Staying at 15 partitions gives us up to 1 million scsi discs.
At 8 Watts/drive, that's 8 megawatts which would practically require your
own power plant to operate.  So if there's need, I suppose we can grow the
number of partitions which would limit us to a quarter-million discs
(and a mere 2MW of power ;-).

    major     p2    disc2   disc  p1
|............|..|..........|....|....| <- dev_t
 31        20    17       8 7  4 3  0

This layout makes sense to me, let me explain it ;-)

partitions 0-15 fit in p1.  Overflow into p2 (bits 18 & 19) for partitions
16-63.

discs are allocated as:

0-15 in sd_major(0), 16-31 in sd_major(1), ... , 240-255 in sd_major(15)
256-16639 in sd_major(0), 16640-33023 in sd_major(1), ...

Stop explaining and give us the code, I hear you cry.  I guess it's ...

unsigned int dev_to_sd_nr(unsigned int dev) {
	return	((dev >> 4) & 15) | (inv_sd_major(dev >> 20) << 4) |
		(dev & 0x3ff00);
}

unsigned int dev_to_sd_part(unsigned int dev) {
	return	(dev & 15) | ((dev >> 14) & 0x30);
}

unsigned int make_sd_dev(unsigned int sd_nr, unsigned int part) {
	return	(part & 0xf) | ((part & 0x30) << 14) | ((dev & 0xf) << 4) |
		(sd_major(dev & 0xf0) << 20) | (dev & 0x3ff00);
}

Everybody happy with this?  Didn't think so.

-- 
"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk

Re: 2.6.0 stability and the BK scsi trees

[Christoph Hellwig]

On Thu, Oct 16, 2003 at 09:56:54PM +1000, Douglas Gilbert wrote:
> What was the point of putting 32 dev_t's into the
> kernel? Many people who were advocating it used
> the increased number of scsi disks (> 256) and
> partitions (from 15 to 63 [to match the ide subsystem])
> as a major reason.
> 
> The sd driver is still littered with hacks to distribute
> its 256 (max) disks over 8 majors. Shouldn't this be
> fixed?
> 
> Comments?

I agree with you that there's some late features that need
to be addresses, and the larger dev_t probably is number
one on that list.  But with Linus' policy having two trees
is the best thing James can do.

Re: 2.6.0 stability and the BK scsi trees

[Patrick Mansfield]

On Thu, Oct 16, 2003 at 02:28:04PM +0100, Matthew Wilcox wrote:
> On Thu, Oct 16, 2003 at 09:56:54PM +1000, Douglas Gilbert wrote:
> > What was the point of putting 32 dev_t's into the
> > kernel? Many people who were advocating it used
> > the increased number of scsi disks (> 256) and
> > partitions (from 15 to 63 [to match the ide subsystem])
> > as a major reason.
> > 
> > The sd driver is still littered with hacks to distribute
> > its 256 (max) disks over 8 majors. Shouldn't this be
> > fixed?
> 
> Well, let's see some fixes and then decide whether it's worth merging
> before 2.6.0 or after 2.6.0.  We can be sure that vendors will integrate
> this patch even if it's not in mainline kernel.org ... and i'd rather
> see one variant of the patch which everybody uses than a different one
> in RH, SuSE and Linux 2.7.

If you have not seen it, Badari's sd patch is in the mm tree:

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.0-test7/2.6.0-test7-mm1/broken-out/support-zillions-of-scsi-disks.patch

-- Patrick Mansfield

Re: 2.6.0 stability and the BK scsi trees

[Matthew Wilcox]

On Sat, Oct 18, 2003 at 08:24:50AM -0700, Patrick Mansfield wrote:
> If you have not seen it, Badari's sd patch is in the mm tree:
> 
> ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.0-test7/2.6.0-test7-mm1/broken-out/support-zillions-of-scsi-disks.patch

I hadn't ... it doesn't address expanding the number of partitions or
the problem that the sd_index_bits array will grow to a huge size.

-- 
"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk

Re: 2.6.0 stability and the BK scsi trees

[Badari Pulavarty]

On Saturday 18 October 2003 09:38 am, Matthew Wilcox wrote:
> On Sat, Oct 18, 2003 at 08:24:50AM -0700, Patrick Mansfield wrote:
> > If you have not seen it, Badari's sd patch is in the mm tree:
> >
> > ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.0-test7
> >/2.6.0-test7-mm1/broken-out/support-zillions-of-scsi-disks.patch
>
> I hadn't ... it doesn't address expanding the number of partitions or
> the problem that the sd_index_bits array will grow to a huge size.

Yes. I did not want to address expanding the number of partitions issue -
since there is no resolution on it. (to be frank, I am not really interested 
in it - so never looked at).

I address sd_index_bits array problem in a crude way. I made the max number
of disks to support as configurable.

I wanted to make the patch really simple - so that we can make few 
decisions before making too many changes.

All I wanted to do was following:

- support > 256 scsi disks
- for now, maintain backward compatibility with current /dev major/minor
  assignments
- test out any kernel (lowmem bloating) issues.

Thanks,
Badari 

Re: 2.6.0 stability and the BK scsi trees

[James Bottomley]

OK, I now have the scsi-misc-2.5 tree split up correctly into
scsi-bugfixes-2.6 and scsi-misc-2.7

Since BK isn't very good at keeping two parallel trees like this
(especially if I have to move patches from one to the other), this is
probably the end of usable SCSI BK trees.


On Thu, 2003-10-16 at 06:56, Douglas Gilbert wrote:
> What was the point of putting 32 dev_t's into the
> kernel? Many people who were advocating it used
> the increased number of scsi disks (> 256) and
> partitions (from 15 to 63 [to match the ide subsystem])
> as a major reason.
> 
> The sd driver is still littered with hacks to distribute
> its 256 (max) disks over 8 majors. Shouldn't this be
> fixed?

Well, my opinion is that this bugfix only edict is to get 2.6.0 stable. 
I anticipate the rules for driver updates will be relaxed again for
2.6.1+

I'm also going to argue that our current refcounting problems are a bug
(especially since they induce oopses), so fixes for refcounting problems
will go into the bugfix tree.

I'm not sure about 32 bit kdev_t.  Initially, I think it counts as an
enhancement, but will revise this if any other driver gets 32 bit kdev_t
patches in under the bugfixes label.

As far as driver bug fixes go, using the old error handler is still a
bug.  However, I'll only place it into the bugfix tree if you've tested
it out on the actual hardware (however obvious the change looks).

I'll cc the list on tree submissions, so you'll be able to read any
objections I get from Andrew or Linus over what constitutes a bug.

James