Re: [linux-usb-devel] Re: USB storage problems on OHCI..

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Pat LaVarre]

> From: Linus Torvalds <torvalds () osdl ! org>
> Date: 2003-09-22 19:55:30
> ...
> How about just trusting the size ...
> and capping it at 20?

Besides dramatic overestimates of mode sense additional length, I've
also seen the off-by-one slight overestimate and the high-byte-zeroed
dramatic underestimate.

> General MODE SENSE stuff is in:
> http://www.t10.org/ftp/t10/drafts/spc3/spc3r15.pdf

Yes, but t10 mmc often quietly contradicts & overrides t10 spc for pdt
x05 dvd/cd and for atapi pdt x00 hdd/rmb.  For t10 newbies wanting to
fetch the bleeding edge, latest final draft, oldest still available,
etc. I sometimes recommend the index:

http://www.t10.org/scsi-3.htm

> > A scsi device declares its level of scsi compliance.
> ...
> From: Matthew Dharm <mdharm-scsi () one-eyed-alien ! net>
> Date: 2003-09-22 20:51:59
> ...
> reporting of 0, 1, 2, or something random ...
> appears ... unrelated to the capabilities of the device ...

The pernicious idea of zeroing the scsi revision in byte 2 of op x12
inquiry forms part of how mmc quietly redefines spc.  Misapplying the
sff precursor of the pdt x05 mmc spec to pdt x00 devices is part of how
msft whql qualified atapi pdt x00 devices for sale thru oem's in the
late 1990's.

> > > > From: Alan Stern <stern () rowland ! harvard ! edu>
> > > > Date: 2003-09-22 14:25:28
> > > > ...
> > > > many, many USB storage devices.  
> > > > They just don't handle MODE-SENSE page 8 correctly.
> > >
> > > From: Linus Torvalds <torvalds () osdl ! org>
> > > Date: 2003-09-22 16:09:47
> > > ...
> > > How about just making the sd.c layer more robust? 
> >
> > From: Alan Stern <stern () rowland ! harvard ! edu>
> > Date: 2003-09-22 16:42:52
> > ...
> > variety of ways ... fail ... truly amazing.
>
> From:  Christoph Hellwig <hch () infradead ! org>
> Date: 2003-09-22 17:21:42
> ...
> for those usb-storage devices where ...
> nothing that isn't excercised by the windows drivers
> has the slightest chance of working..

In the sd world, usb hdd/rmb (pdt x00) may handle specifically the mode
sense of page 8 that win sends.

For usb bInterfaceSubClass = x06 TransparentScsi that cdb might be:
x 1A 00:08:00 18 00

For usb bInterfaceSubClass != x06 TransparentScsi that cdb might be:
x 5A 00 08:00:00:00 00 00:1C 00

usb dvd/cd (pdt x05) sell into the sr/ide-cd world instead, of course.

Tell me we care to hear specifically what some win sends to some of my
usb hdd/rmb, and I'll go collect a few sample usb bus traces.  I do NOT
know that the above cdb's are correct.  Those are just my guesses,
summing up little clues from years of messing around near here.  I hear
in particular that win 9x/me differs from win 2k/xp near hear.

> Subject:  Re: [linux-usb-devel] Re: USB storage problems on OHCI..
> From: Linus Torvalds <torvalds () osdl ! org>
> Date: 2003-09-22 17:41:47
> ...
> is it saner to try to read just the bytes we need
> (3 bytes: page code, page size and the cache bits),
> or the full 20 bytes?

I thought "everyone knew" sanest was to always fetch header + block
descriptors + whole pages?

Trouble is, that's a circular definition in that the host can't know
whether the device will volunteer an optional block descriptor or not. 
mmc says shall not, spc says maybe not but should.

We only have to care if we have to know how many Data bytes to expect to
copy In.

> From: Andries.Brouwer () cwi ! nl
> Date: 2003-09-22 18:55:22
> ...
> Similarly, USB storage devices tend to ...
> only systematically work
> if one does precisely what Windows does.

Yes, what ain't tested don't work.

> USB is hot pluggable,
> so it should not be necessary
> to send a flush cache command at shutdown.

Who has tested this rule?  I know I've seen ata hdd lose data in
response to a pin 1 reset issued without op x35 sync cache.  I've heard
same of dvd.  And loss of bus power can kill any write cache in a drive,
of course.

> From: James Bottomley <James.Bottomley () steeleye ! com>
> Date: 2003-09-22 17:55:09 ...
> ...
> I think we could try 4 bytes

We know that isn't talk like windows.  msdn.microsoft.com tells us the
win 2k kernel crashes if a root-privileged app asks for just the 4 byte
header from devices for which Win translates to x5A MODE_SENSE_10 from
x1A MODE_SENSE  Because the win 2k kernel crashes so reliably, we can
confidently guess no working win 2k apps talk that way.

> From: James Bottomley <James.Bottomley () steeleye ! com>
> Date: 2003-09-22 19:28:56
> ...
> sd? ... the most conservative ..., we could probably
> dump spin up, read write protect, and read cache type ...

Aye, although in sr/ide-cd we have a default of not writable to overcome
somehow.

Pat LaVarre

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Pat LaVarre]

> From: Linus Torvalds <torvalds () osdl ! org>
> Date: 2003-09-23 15:46:21
>
> Normal users consider hot-pluggable
> to be "I can rip the thing out by hand".

Yes.

Meanwhile, much (all except flash?) actual usb-storage now commercially
available is Not hot-pluggable in this sense.  Instead, "ata hdd lose
data in response to a pin 1 reset issued without op x35
"SYNCHRONIZE_CACHE". I've heard same of" writeable "dvd.  And loss of
bus power can kill any write cache in a drive, of course."

Pat LaVarre

2 KiB/block loopback found where

[Pat LaVarre]

I want a 2 KiB/block loopback device to help debug udf.ko.

I get 0.5 KiB/block.  Did I fail to find an option or an appropriate
kernel patch, I do I have to write my own patch?  I'm guessing I'll find
2 KiB/block easy to fake, since 0.5 KiB divides evenly into 2 KiB.

Specifically, I want to see 2048 where now I see 512 in response to my
query `blockdev --getss`.  A more complete example console log follows:

Pat LaVarre

$
$ dd of=dd.bin bs=1M count=1 if=/dev/zero
1+0 records in
1+0 records out
$
$ ls -l dd.bin
-rw-rw-r--    1 pat      pat       1048576 Sep 29 09:43 dd.bin
$
$ sudo /sbin/losetup /dev/loop0 dd.bin
$
$ sudo blockdev --getss /dev/loop0
512
$

Re: 2 KiB/block loopback found where

[Jens Axboe]

On Mon, Sep 29 2003, Pat LaVarre wrote:
> 
> I want a 2 KiB/block loopback device to help debug udf.ko.
> 
> I get 0.5 KiB/block.  Did I fail to find an option or an appropriate
> kernel patch, I do I have to write my own patch?  I'm guessing I'll find
> 2 KiB/block easy to fake, since 0.5 KiB divides evenly into 2 KiB.
> 
> Specifically, I want to see 2048 where now I see 512 in response to my
> query `blockdev --getss`.  A more complete example console log follows:
> 
> Pat LaVarre
> 
> $
> $ dd of=dd.bin bs=1M count=1 if=/dev/zero
> 1+0 records in
> 1+0 records out
> $
> $ ls -l dd.bin
> -rw-rw-r--    1 pat      pat       1048576 Sep 29 09:43 dd.bin
> $
> $ sudo /sbin/losetup /dev/loop0 dd.bin
> $
> $ sudo blockdev --getss /dev/loop0
> 512
> $

Add a quick hack to loop to set 2kb block size, grep for set_blocksize.

-- 
Jens Axboe

Re: 2 KiB/block loopback found where

[Pat LaVarre]

> > I want a 2 KiB/block loopback device ...
> > I get 0.5 KiB/block ...
> > 0.5 KiB divides evenly into 2 KiB.
>
> Add a quick hack to loop
> to set 2kb block size,
> grep for set_blocksize.

Great hint forward, thanks.  My first guess at what we mean follows,
I'll report back to say how this goes.

Pat LaVarre

diff -Nur linux-2.6.0-test6/drivers/block/loop.c linux/drivers/block/loop.c 

--- linux-2.6.0-test6/drivers/block/loop.c	2003-09-27 18:50:29.000000000 -0600
+++ linux/drivers/block/loop.c	2003-09-29 11:06:50.057826432 -0600
@@ -732,7 +732,7 @@
 	mapping_set_gfp_mask(inode->i_mapping,
 			     lo->old_gfp_mask & ~(__GFP_IO|__GFP_FS));
 
-	set_blocksize(bdev, lo_blocksize);
+	set_blocksize(bdev, 0x800); // not lo_blocksize);
 
 	lo->lo_bio = lo->lo_biotail = NULL; 

aligned /dev/scd$n reads less rare how

[Pat LaVarre]

Anything easy I can do to make aligned reads more common?

I ask because often thru /dev/scd$n I see seemingly pointless
misalignment, for example, if I read a 1 GiB disk:

$ sudo blockdev --flushbufs /dev/scd0
$ time sudo dd if=/dev/scd0 bs=1M
...

usb-storage:  x 43 00 00 00 00 00 00 00 0C 00
usb-storage:  x 43 02 00 00 00 00 01 00 0C 00
usb-storage:  x 1E 00 00 00 01 00
usb-storage:  x 00 00 00 00 00 00
usb-storage:  x 28 00 00 00 00 00 00 00 20 00
usb-storage:  x 28 00 00 00 00 20 00 00 24 00
usb-storage:  x 28 00 00 00 00 44 00 00 40 00
usb-storage:  x 28 00 00 00 00 84 00 00 40 00
usb-storage:  x 28 00 00 00 00 C4 00 00 40 00
...
usb-storage:  x 28 00 00 07 FE C4 00 00 40 00
usb-storage:  x 28 00 00 07 FF 04 00 00 06 00
usb-storage:  x 28 00 00 07 FF 0A 00 00 3A 00
usb-storage:  x 28 00 00 07 FF 44 00 00 40 00
usb-storage:  x 28 00 00 07 FF 84 00 00 0A 00
usb-storage:  x 28 00 00 07 FF 8E 00 00 36 00
usb-storage:  x 28 00 00 07 FF C4 00 00 3C 00
...

1024+0 records in
1024+0 records out
 
real    0m54.809s
user    0m0.015s
sys     0m9.039s
$

Pat LaVarre

P.S.

I know I can read aligned thru /dev/sg0 as follows.

$
$ time sudo sg_dd if=/dev/sg0 bs=2k bpt=512 count=524288
of=/mnt/hda11/z.bin
...
 
Attached scsi generic sg0 at scsi1, channel 0, id 0, lun 0,  type 5
usb-storage:  x 28 00 00 00 00 00 00 02 00 00
usb-storage:  x 28 00 00 00 02 00 00 02 00 00
usb-storage:  x 28 00 00 00 04 00 00 02 00 00
usb-storage:  x 28 00 00 00 06 00 00 02 00 00
...
usb-storage:  x 28 00 00 07 FA 00 00 02 00 00
usb-storage:  x 28 00 00 07 FC 00 00 02 00 00
usb-storage:  x 28 00 00 07 FE 00 00 02 00 00

524288+0 records in
524288+0 records out
 
real    0m56.567s
user    0m0.027s
sys     0m7.236s
$

I see this approach to aligning reads slowed me down a second or two.

zip of GiB cross-platform

[Pat LaVarre]

Is linux-scsi a place where folk hang out who often mail around GiB from
Linux?

I see Linux `zip -r $folder $folder` produces .zip files that Mac OS X
and Win XP can read ...

... only when the folder is small.  When the folder contains a `dd`
image of a mostly zero 1GiB disk (specifically an initially zero +
`mkudffs`) disk, then Mac OS X & Win XP choke.

`unzip -vt $folder` is happy, `unzip` produces a file that `diff` and
`cmp` say matches the original.

Did I commit some brain dead newbie error?  Reproduce a known bug? 
Discover a new bug?

Pat LaVarre

P.S. For example, Mac OS X reports:

An error has occurred while expanding the file ...zlxudf.zip"
(Unspecified StuffIt Engine internal error).

Error #17999

Re: zip of GiB cross-platform

[Matthew Wilcox]

I suggest you file a bug with your distribution (if you're using one)
or talk to the zip maintainers (if you're not).  This kind of question
does not belong on this list.

-- 
"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk

Re: zip of GiB cross-platform

[Pat LaVarre]

> I suggest ... zip maintainers (if you're not).
> This kind of question
> does not belong on this list.

Thanks for clueing me in, sorry to take your time.

Pat LaVarre

Re: 2 KiB/block loopback found where

[Pat LaVarre]

> > > Subject: 2 KiB/block loopback found where
> > ...
> > Subject: udf folders pointing to self and above
> ...
> Subject: Re: zip of GiB cross-platform
> This kind of question
> does not belong on this list.

I wonder if this "2 KiB/block loopback found where" thread belongs on
linux-kernel, rather than on linux-scsi, since the code involved seems
to be drivers/block/loop.c

Pat LaVarre

Re: max GiB written per boot

[Pat LaVarre]

I wish in the Subject line had mentioned so far I've tried writing GiB
only via usb where it dies, not elsewhere.  Today I returned after
leaving a drive idle for some days.  Nothing had appeared in the `dmesg`
meanwhile, but trying to talk to the drive provoked:

scsi: Device offlined - not ready after error recovery: host 0 channel 0
id 0 lun 0

Time to reboot I guess.  Pat LaVarre

writable mmc profiles actually are writable

[Pat LaVarre]

> Newsgroups: mlist.linux.scsi
> Date: 2003-09-10 16:00:27 PST
> Subject: Re: [PATCH] mount -w of dvd+rw etc. in vanilla 2.6
> From: Patrick Mansfield (patmans@us.ibm.com)
>
> Should sd.c and sr.c be calling set_device_ro?
> (after adding a read only block device, or
> changing back to read/write)?

That good question stands open, I think.

The following inline plain text, now revved for -test6 & posted here,
allows modprobe to overcome such uncooperative attitudes as:

$ sudo dd if=/dev/scd2 bs=2k count=1 | wc
1+0 records in
1+0 records out
      0       0    2048
$
$ sudo dd of=/dev/scd2 bs=2k count=1 if=/dev/zero
dd: opening `/dev/scd2': Read-only file system
$

Allowing dd, mkudffs, etc. are the good consequences.  The evil
consequences I do not yet know.  I do notice we have achieved
writability without extending the enumeration that `cat
/proc/sys/dev/cdrom/info` dumps:

Can write CD-R:         0
Can write CD-RW:        0
Can read DVD:           0
Can write DVD-R:        0
Can write DVD-RAM:      0

Pat LaVarre

diff -ur linux-2.6.0-test6/drivers/cdrom/cdrom.c linux/drivers/cdrom/cdrom.c
--- linux-2.6.0-test6/drivers/cdrom/cdrom.c	2003-09-27 18:50:10.000000000 -0600
+++ linux/drivers/cdrom/cdrom.c	2003-10-06 11:53:41.826299728 -0600
@@ -426,8 +426,8 @@
 	if ((fp->f_flags & O_NONBLOCK) && (cdi->options & CDO_USE_FFLAGS))
 		ret = cdi->ops->open(cdi, 1);
 	else {
-		if ((fp->f_mode & FMODE_WRITE) && !CDROM_CAN(CDC_DVD_RAM))
-			return -EROFS;
+//		if ((fp->f_mode & FMODE_WRITE) && !CDROM_CAN(CDC_DVD_RAM))
+//			return -EROFS;
 
 		ret = open_for_data(cdi);
 	}
diff -ur linux-2.6.0-test6/drivers/scsi/sr.c linux/drivers/scsi/sr.c
--- linux-2.6.0-test6/drivers/scsi/sr.c	2003-09-27 18:50:13.000000000 -0600
+++ linux/drivers/scsi/sr.c	2003-10-06 11:51:59.655832000 -0600
@@ -327,8 +327,8 @@
 	}
 
 	if (rq_data_dir(SCpnt->request) == WRITE) {
-		if (!cd->device->writeable)
-			return 0;
+//		if (!cd->device->writeable)
+//			return 0;
 		SCpnt->cmnd[0] = WRITE_10;
 		SCpnt->sc_data_direction = SCSI_DATA_WRITE;
 	} else if (rq_data_dir(SCpnt->request) == READ) {

Re: writable mmc profiles actually are writable

[Jens Axboe]

On Mon, Oct 06 2003, Pat LaVarre wrote:
> > Newsgroups: mlist.linux.scsi
> > Date: 2003-09-10 16:00:27 PST
> > Subject: Re: [PATCH] mount -w of dvd+rw etc. in vanilla 2.6
> > From: Patrick Mansfield (patmans@us.ibm.com)
> >
> > Should sd.c and sr.c be calling set_device_ro?
> > (after adding a read only block device, or
> > changing back to read/write)?
> 
> That good question stands open, I think.
> 
> The following inline plain text, now revved for -test6 & posted here,
> allows modprobe to overcome such uncooperative attitudes as:
> 
> $ sudo dd if=/dev/scd2 bs=2k count=1 | wc
> 1+0 records in
> 1+0 records out
>       0       0    2048
> $
> $ sudo dd of=/dev/scd2 bs=2k count=1 if=/dev/zero
> dd: opening `/dev/scd2': Read-only file system
> $
> 
> Allowing dd, mkudffs, etc. are the good consequences.  The evil
> consequences I do not yet know.  I do notice we have achieved
> writability without extending the enumeration that `cat
> /proc/sys/dev/cdrom/info` dumps:
> 
> Can write CD-R:         0
> Can write CD-RW:        0
> Can read DVD:           0
> Can write DVD-R:        0
> Can write DVD-RAM:      0

And the device isn't writable, so...?

> diff -ur linux-2.6.0-test6/drivers/cdrom/cdrom.c linux/drivers/cdrom/cdrom.c
> --- linux-2.6.0-test6/drivers/cdrom/cdrom.c	2003-09-27 18:50:10.000000000 -0600
> +++ linux/drivers/cdrom/cdrom.c	2003-10-06 11:53:41.826299728 -0600
> @@ -426,8 +426,8 @@
>  	if ((fp->f_flags & O_NONBLOCK) && (cdi->options & CDO_USE_FFLAGS))
>  		ret = cdi->ops->open(cdi, 1);
>  	else {
> -		if ((fp->f_mode & FMODE_WRITE) && !CDROM_CAN(CDC_DVD_RAM))
> -			return -EROFS;
> +//		if ((fp->f_mode & FMODE_WRITE) && !CDROM_CAN(CDC_DVD_RAM))
> +//			return -EROFS;
>  
>  		ret = open_for_data(cdi);
>  	}
> diff -ur linux-2.6.0-test6/drivers/scsi/sr.c linux/drivers/scsi/sr.c
> --- linux-2.6.0-test6/drivers/scsi/sr.c	2003-09-27 18:50:13.000000000 -0600
> +++ linux/drivers/scsi/sr.c	2003-10-06 11:51:59.655832000 -0600
> @@ -327,8 +327,8 @@
>  	}
>  
>  	if (rq_data_dir(SCpnt->request) == WRITE) {
> -		if (!cd->device->writeable)
> -			return 0;
> +//		if (!cd->device->writeable)
> +//			return 0;
>  		SCpnt->cmnd[0] = WRITE_10;
>  		SCpnt->sc_data_direction = SCSI_DATA_WRITE;
>  	} else if (rq_data_dir(SCpnt->request) == READ) {

This is obviously wrong. What are you trying to do? The uniform layer
uses CDC_DVD_RAM as meaning randomly writable media, the only thing the
kernel supports out of the box. So that is what the test is for.

Honestly, I have no idea what your are trying to pull. Calling read-only
media failing to be opened read-write as "uncooperative" is confusing,
borderline amusing :). Explain yourself in clear text, thanks.

-- 
Jens Axboe

Re: writable mmc profiles actually are writable

[Jens Axboe]

On Mon, Oct 06 2003, Jens Axboe wrote:
> Honestly, I have no idea what your are trying to pull. Calling read-only
> media failing to be opened read-write as "uncooperative" is confusing,
> borderline amusing :). Explain yourself in clear text, thanks.

Ok, first clue: don't start a new post by replying to your old message,
it screws threading of decent mailers. I didn't see your subject because
of that, for instance.

Second, assuming the subject and body of the email are connected (no
clear clues there), I'm assuming this has to do with misdetected
hardware? If yes, then you should probably be fixing that instead.

-- 
Jens Axboe

Re: writable mmc profiles actually are writable

[Pat LaVarre]

> assuming this has to do with misdetected hardware?

Seemingly so, aye.

> hardware?

I'm plugging in a pre-production sample of a new device.

Seemingly I cannot mkudffs its disks except by patching the kernel.

That's normal for yet another new kind of dvd/cd, yes?

Via sg I see 35 GB of 64 KiB/block writes, 2 KiB/block reads, with
writable profile x0002 removable_disk of the 1999 mmc 2 standard.  Yes
pdt x05 dvd/cd, yes feature x0020 random_writable, no not the cd_r,
cd_rw, dvd, dvd_r, and dvd_ram enumerated in the obsolete legacy 1997
mmc 1 mode page x2A Capabilities.

Previously near here we discussed some as-yet-ineffective alternatives
to my patch: the Andy Polyakov patch for mount -w of dvd+rw, the Jens
Axboe patch for mount -w of cd-rw, and blockdev --setrw.

> The uniform layer uses CDC_DVD_RAM as meaning
> randomly writable media,

When I try setting CDC_DVD_RAM for a device, I see
/proc/sys/dev/cdrom/info erroneously reports:

Can write DVD-RAM:      1

I conclude we wish to express the discovery of feature x0020
random_writable differently.  Wrong?

In weak confirmation, earlier I did see the usb-storage folk reject a
patch to tweak mode page x2A to appear to have mode page x2A byte 3 mask
x20 dvd_ram_write set.  Should I submit a patch like that for sr_mod.ko
and ide-cd.ko?

> Calling read-only media failing to be opened
> read-write as "uncooperative" is confusing,

Clear now?  I mean to say, sg_dd shows the real disk of the real device
is randomly rewritable.  But cdrom.ko and sr_mod.ko refuse to pass
writes thru.

> screws threading of decent mailers

Sorry.  I'm pleased to mention people have kindly volunteered offline to
help me avoid repeating that.

Pat LaVarre

Re: writable mmc profiles actually are writable

[Pat LaVarre]

> > +//   if ((fp->f_mode & FMODE_WRITE) && !CDROM_CAN(CDC_DVD_RAM))
> > +//           return -EROFS;
> > +//   if (!cd->device->writeable)
> > +//           return 0;
>
> ... obviously wrong

May I ask why this patch is obviously wrong?

What actually breaks?

How do we benefit from having cdrom.ko and sr_mod.ko redundantly refuse
writes?

I figure I should try mount of a cd-rom with this patch.  Maybe I get
mount -w when I wanted mount -r and then life goes downhill from there. 
Or maybe iso fs doesn't try to write.  Or ...

Pat LaVarre

Re: writable mmc profiles actually are writable

[Pat LaVarre]

> CDC_DVD_RAM ...
> the only thing the kernel supports out of the box.

Do we understand this choice?  I'm new to linux, new to t10 mmc, old to
unix, old to t10 scsi.  I think I see:

1) In mmc, seven of the standard profiles imply feature x0020
random_writable, specifically profiles x 0001 0002 0003 0005 0012 001A
0022 i.e. non_removable_disk, removable_disk, magneto_optical_erasable,
as_mo, dvd_ram, dvd+rw, ddcd_rw.

2) In sr_mod.ko and ide-cd.ko, we conservatively fetch only mode page
x2A Capabilities.  Specifically we do not yet try the op x46 Get
Configuration of 1999 mmc 2, not even after we see mode page x2A.

3) Omitting op x46 by definition precludes linux from distinguishing
from legacy cd_rom all the new writable kinds of pdt x05 dvd/cd,
excepting only the 1997 mmc 1 choices that /proc/sys/dev/cdrom already
offers i.e. cd_r, cd_rw, dvd, dvd_r, and dvd_ram.

How wrong am I?

Pat LaVarre

Re: writable mmc profiles actually are writable

[Jens Axboe]

On Mon, Oct 06 2003, Pat LaVarre wrote:
> > > +//   if ((fp->f_mode & FMODE_WRITE) && !CDROM_CAN(CDC_DVD_RAM))
> > > +//           return -EROFS;
> > > +//   if (!cd->device->writeable)
> > > +//           return 0;
> >
> > ... obviously wrong
> 
> May I ask why this patch is obviously wrong?

Allowing write open of a read-only device.

> What actually breaks?

Lots of stuff. As I mentioned, only DVD-RAM "type" media will work with
sending random writes, things like cd-rw require special treatment.

> How do we benefit from having cdrom.ko and sr_mod.ko redundantly refuse
> writes?
> 
> I figure I should try mount of a cd-rom with this patch.  Maybe I get
> mount -w when I wanted mount -r and then life goes downhill from there. 

> Or maybe iso fs doesn't try to write.  Or ...

iso fs doesn't support writing.

-- 
Jens Axboe

Re: writable mmc profiles actually are writable

[Jens Axboe]

On Mon, Oct 06 2003, Pat LaVarre wrote:
> > CDC_DVD_RAM ...
> > the only thing the kernel supports out of the box.
> 
> Do we understand this choice?  I'm new to linux, new to t10 mmc, old to
> unix, old to t10 scsi.  I think I see:

You conviniently snipped the rest of the message. I said that media that
is _like_ DVD-RAM. I'll grant you that it isn't quite obvious, but that
is the way it was done. That covers stuff like MO drives, too.

> 1) In mmc, seven of the standard profiles imply feature x0020
> random_writable, specifically profiles x 0001 0002 0003 0005 0012 001A
> 0022 i.e. non_removable_disk, removable_disk, magneto_optical_erasable,
> as_mo, dvd_ram, dvd+rw, ddcd_rw.

the disk profiles aren't driven by ide-cd or sr. mo and dvd-ram have
been supported for a long time and should be set with CDC_DVD_RAM, I
haven't yet had to deal with dvd+rw or ddcd_rw (is this like mtr?). Feel
free to extend that to cover those, too. I know of patches for dvd+rw
floating around, they have never been sent for kernel inclusion though.
Which either means that they don't work too well (they probably do), or
that people just don't care enough for that feature.

> 2) In sr_mod.ko and ide-cd.ko, we conservatively fetch only mode page
> x2A Capabilities.  Specifically we do not yet try the op x46 Get
> Configuration of 1999 mmc 2, not even after we see mode page x2A.
>
> 3) Omitting op x46 by definition precludes linux from distinguishing
> from legacy cd_rom all the new writable kinds of pdt x05 dvd/cd,
> excepting only the 1997 mmc 1 choices that /proc/sys/dev/cdrom already
> offers i.e. cd_r, cd_rw, dvd, dvd_r, and dvd_ram.
> 
> How wrong am I?

You are not wrong, and I'd be happy to review patches to make the
detection more modern. And this is what you should have done from the
beginning, instead of just hacking around the write open. This is what
my objection was for, not supporting other devices.

-- 
Jens Axboe

Re: writable mmc profiles actually are writable

[Jens Axboe]

On Mon, Oct 06 2003, Pat LaVarre wrote:
> > assuming this has to do with misdetected hardware?
> 
> Seemingly so, aye.
> 
> > hardware?
> 
> I'm plugging in a pre-production sample of a new device.
> 
> Seemingly I cannot mkudffs its disks except by patching the kernel.
> 
> That's normal for yet another new kind of dvd/cd, yes?

Completely.

> Via sg I see 35 GB of 64 KiB/block writes, 2 KiB/block reads, with
> writable profile x0002 removable_disk of the 1999 mmc 2 standard.  Yes
> pdt x05 dvd/cd, yes feature x0020 random_writable, no not the cd_r,
> cd_rw, dvd, dvd_r, and dvd_ram enumerated in the obsolete legacy 1997
> mmc 1 mode page x2A Capabilities.
> 
> Previously near here we discussed some as-yet-ineffective alternatives
> to my patch: the Andy Polyakov patch for mount -w of dvd+rw, the Jens
> Axboe patch for mount -w of cd-rw, and blockdev --setrw.

blockdev --setrw is the same bypass again. dvd+rw patch and cd-rw patch
are two different beasts. The latter should probably be allowed to die
given the hideousness of that hardware, I haven't even looked at the
dvd+rw patch to pass any judgement on that (but I presume dvd+rw design
is saner and doesn't include software supported read-modify-write cycles
and defect management).

> > The uniform layer uses CDC_DVD_RAM as meaning
> > randomly writable media,
> 
> When I try setting CDC_DVD_RAM for a device, I see
> /proc/sys/dev/cdrom/info erroneously reports:
> 
> Can write DVD-RAM:      1
> 
> I conclude we wish to express the discovery of feature x0020
> random_writable differently.  Wrong?

Correct.

> In weak confirmation, earlier I did see the usb-storage folk reject a
> patch to tweak mode page x2A to appear to have mode page x2A byte 3 mask
> x20 dvd_ram_write set.  Should I submit a patch like that for sr_mod.ko
> and ide-cd.ko?

I'd propose just adding better detection of new devices, and adding a
specific "randomly writable" flag instead of abusing CDC_DVD_RAM.

> > Calling read-only media failing to be opened
> > read-write as "uncooperative" is confusing,
> 
> Clear now?  I mean to say, sg_dd shows the real disk of the real device
> is randomly rewritable.  But cdrom.ko and sr_mod.ko refuse to pass
> writes thru.

Well yes, now that you explain what the issue actually is rather than
show up with an obscure "patch" :)

-- 
Jens Axboe

Re: writable mmc profiles actually are writable

[Pat LaVarre]

> patches to make the detection more modern ...

Below appears a copy of an old patch to try op x46 get_configuration any
time mode page x2a doesn't already say the pdt x05 dvd/cd device is
writable.

> I'd propose just adding better detection of
> new devices, and adding a specific "randomly
> writable" flag instead of abusing CDC_DVD_RAM.

Ah, did that, see below.  With this remark to motivate me, I will rev
this -test5 patch to fit with -test6 and then repost.

> I haven't yet had to deal with dvd+rw or
> ddcd_rw (is this like mtr?

Sorry I do not know mtr.

> mo and dvd-ram have been supported for a long
> time and should be set with CDC_DVD_RAM,

Sorry I do not understand.  I remember noticing CDC_MO_DRIVE appears
only drivers/ide/ not in drivers/scsi/.  Weakly I conclude mo doesn't
work the same in atapi and usb-storage.

> the disk profiles aren't driven by ide-cd or
> sr.

Sorry I do not understand.

In drivers/scsi/sr.c I find:
cd->cdi.mask |= CDC_DVD_RAM;

In drivers/ide/ide-cd.c I find:
devinfo->mask |= CDC_DVD_RAM;

I conclude sr and ide-cd tell cdrom whether the device "profile" is
writable or not.  Is that wrong?

> you should have done

Thanks again for finding the time to clue me in.

Pat LaVarre

diff -Nur linux-2.6.0-test5/include/scsi/scsi.h linux/include/scsi/scsi.h
diff -Nur linux-2.6.0-test5/include/linux/cdrom.h linux/include/linux/cdrom.h
diff -Nur linux-2.6.0-test5/drivers/cdrom/cdrom.c linux/drivers/cdrom/cdrom.c
diff -Nur linux-2.6.0-test5/drivers/scsi/sr.c linux/drivers/scsi/sr.c
--- linux-2.6.0-test5/include/scsi/scsi.h	2003-09-08 13:50:41.000000000 -0600
+++ linux/include/scsi/scsi.h	2003-09-10 17:05:37.000000000 -0600
@@ -81,6 +81,7 @@
 #define CHANGE_DEFINITION     0x40
 #define WRITE_SAME            0x41
 #define READ_TOC              0x43
+#define GET_CONFIGURATION     0x46
 #define LOG_SELECT            0x4c
 #define LOG_SENSE             0x4d
 #define MODE_SELECT_10        0x55
--- linux-2.6.0-test5/include/linux/cdrom.h	2003-09-08 13:49:51.000000000 -0600
+++ linux/include/linux/cdrom.h	2003-09-10 17:05:37.000000000 -0600
@@ -388,6 +388,7 @@
 #define CDC_DVD_R		0x10000	/* drive can write DVD-R */
 #define CDC_DVD_RAM		0x20000	/* drive can write DVD-RAM */
 #define CDC_MO_DRIVE		0x40000 /* drive is an MO device */
+#define CDC_MMC_RW		0x80000 /* drive is other mmc "writable" "profile" */
 
 /* drive status possibilities returned by CDROM_DRIVE_STATUS ioctl */
 #define CDS_NO_INFO		0	/* if not implemented */
--- linux-2.6.0-test5/drivers/cdrom/cdrom.c	2003-09-08 13:49:53.000000000 -0600
+++ linux/drivers/cdrom/cdrom.c	2003-09-10 17:05:37.000000000 -0600
@@ -426,7 +426,7 @@
 	if ((fp->f_flags & O_NONBLOCK) && (cdi->options & CDO_USE_FFLAGS))
 		ret = cdi->ops->open(cdi, 1);
 	else {
-		if ((fp->f_mode & FMODE_WRITE) && !CDROM_CAN(CDC_DVD_RAM))
+		if ((fp->f_mode & FMODE_WRITE) && !CDROM_CAN(CDC_DVD_RAM|CDC_MMC_RW))
 			return -EROFS;
 
 		ret = open_for_data(cdi);
@@ -2456,6 +2456,10 @@
 	for (cdi=topCdromPtr;cdi!=NULL;cdi=cdi->next)
 	    pos += sprintf(info+pos, "\t%d", CDROM_CAN(CDC_DVD_RAM) != 0);
 
+	pos += sprintf(info+pos, "\nCan write other MMC-RW:");
+	for (cdi=topCdromPtr;cdi!=NULL;cdi=cdi->next)
+	    pos += sprintf(info+pos, "\t%d", CDROM_CAN(CDC_MMC_RW) != 0);
+
 	strcpy(info+pos,"\n\n");
 		
         return proc_dostring(ctl, write, filp, buffer, lenp);
--- linux-2.6.0-test5/drivers/scsi/sr.c	2003-09-08 13:49:58.000000000 -0600
+++ linux/drivers/scsi/sr.c	2003-09-10 17:13:08.000000000 -0600
@@ -67,7 +67,8 @@
 	(CDC_CLOSE_TRAY|CDC_OPEN_TRAY|CDC_LOCK|CDC_SELECT_SPEED| \
 	 CDC_SELECT_DISC|CDC_MULTI_SESSION|CDC_MCN|CDC_MEDIA_CHANGED| \
 	 CDC_PLAY_AUDIO|CDC_RESET|CDC_IOCTLS|CDC_DRIVE_STATUS| \
-	 CDC_CD_R|CDC_CD_RW|CDC_DVD|CDC_DVD_R|CDC_DVD_RAM|CDC_GENERIC_PACKET)
+	 CDC_CD_R|CDC_CD_RW|CDC_DVD|CDC_DVD_R|CDC_DVD_RAM|CDC_GENERIC_PACKET| \
+	 CDC_MMC_RW)
 
 static int sr_probe(struct device *);
 static int sr_remove(struct device *);
@@ -92,6 +93,10 @@
 static void get_sectorsize(struct scsi_cd *);
 static void get_capabilities(struct scsi_cd *);
 
+static void get_configuration(struct scsi_cd *);
+static int scsi_get_configuration(struct scsi_device *sdev,
+	unsigned char *buffer, int len, int timeout, int retries);
+
 static int sr_media_change(struct cdrom_device_info *, int);
 static int sr_packet(struct cdrom_device_info *, struct cdrom_generic_command *);
 
@@ -755,6 +760,7 @@
 	rc = scsi_mode_sense(cd->device, 0, 0x2a, buffer, 128,
 			     SR_TIMEOUT, 3, &data);
 
+	cd->cdi.mask |= CDC_MMC_RW;
 	if (!scsi_status_is_good(rc)) {
 		/* failed, drive doesn't have capabilities mode page */
 		cd->cdi.speed = 1;
@@ -817,6 +823,83 @@
 
 	scsi_release_request(SRpnt);
 	kfree(buffer);
+
+	if (!cd->device->writeable) {
+		printk("%s: scsi3-mmc maybe not writeable\n",
+		       cd->cdi.name);
+		get_configuration(cd);
+	}
+}
+
+static void get_configuration(struct scsi_cd *cd)
+{
+	unsigned char *buffer;
+	int rc, mmc_profile;
+
+	buffer = kmalloc(512, GFP_KERNEL | GFP_DMA);
+	if (!buffer) {
+		printk(KERN_ERR "sr: out of memory.\n");
+		return;
+	}
+
+	rc = scsi_get_configuration(cd->device, buffer, 8,
+			     SR_TIMEOUT, 1);
+	if (!scsi_status_is_good(rc)) {
+		kfree(buffer);
+		return;
+	}
+
+	mmc_profile = ((buffer[6] << 8) + buffer[7]);
+	switch (mmc_profile) {
+	case 0x0001: /* Non-Removable Disk */
+	case 0x0002: /* Removable Disk */
+	case 0x0003: /* Magneto-Optical Erasable */
+	case 0x0005: /* AS-MO */
+	case 0x0012: /* DVD-RAM */
+	case 0x001A: /* DVD+RW */
+	case 0x0022: /* DDCD-RW */
+		printk("%s: scsi3-mmc writable profile: 0x%04x\n",
+		       cd->cdi.name, mmc_profile);
+		cd->device->writeable = 1;
+		if (mmc_profile != 0x001A) { /* != DVD+RW */
+			cd->cdi.mask &= ~CDC_MMC_RW;
+		}
+		break;
+	default:
+		break;
+	}
+
+	kfree(buffer);
+}
+
+/* intended to be block-copy-edit of scsi.lib.c scsi_mode_sense */
+static int
+scsi_get_configuration(struct scsi_device *sdev,
+		  unsigned char *buffer, int len, int timeout, int retries)
+{
+	struct scsi_request *sreq = scsi_allocate_request(sdev, GFP_KERNEL);
+	unsigned char cmd[MAX_COMMAND_SIZE];
+	int ret;
+
+	if (!sreq)
+		return -1;
+
+	memset(&cmd[0], 0, MAX_COMMAND_SIZE);
+	cmd[0] = GET_CONFIGURATION;
+	cmd[8] = len;
+
+	sreq->sr_cmd_len = 0;
+	sreq->sr_sense_buffer[0] = 0;
+	sreq->sr_sense_buffer[2] = 0;
+	sreq->sr_data_direction = DMA_FROM_DEVICE;
+
+	memset(buffer, 0, len);
+
+	scsi_wait_req(sreq, cmd, buffer, len, timeout, retries);
+
+	ret = sreq->sr_result;
+	scsi_release_request(sreq);
+	return ret;
 }
 
 /*

Re: max GiB written per boot

[Pat LaVarre]

Now commonly in -test6 I see such seemingly spontaneous events as:

scsi: Device offlined - not ready after error recovery: host 2 channel 0 id 0 lun 0
sr2: CDROM (ioctl) error, command: 0x00 00 00 00 00 00
sr?: sense =  0  0
Non-extended sense class 0 code 0x0
Raw sense data:0x00 0x00 0x00 0x00
cdrom: open failed.
SCSI error: host 2 id 0 lun 0 return code = 6070000
        Sense class 0, sense error 0, extended sense 0

Did I change some other significant variable?  Could be ...

I will rebuild my usb trace infrastructure and report again.

Pat LaVarre

Re: writable mmc profiles actually are writable

[Pat LaVarre]

> diff -Nur linux-2.6.0-test5/include/scsi/scsi.h linux/include/scsi/scsi.h
> diff -Nur linux-2.6.0-test5/include/linux/cdrom.h linux/include/linux/cdrom.h
> diff -Nur linux-2.6.0-test5/drivers/cdrom/cdrom.c linux/drivers/cdrom/cdrom.c
> diff -Nur linux-2.6.0-test5/drivers/scsi/sr.c linux/drivers/scsi/sr.c

Trying that -test5 patch in -test6 I get:

patching file include/scsi/scsi.h
Hunk #1 succeeded at 74 (offset -7 lines).
patching file include/linux/cdrom.h
patching file drivers/cdrom/cdrom.c
Hunk #2 succeeded at 2465 (offset 9 lines).
patching file drivers/scsi/sr.c
Hunk #3 succeeded at 759 (offset -1 lines).

I will study this result and report back.

Pat LaVarre

Re: max GiB written per boot

[Pat LaVarre]

With usb2 in linux 2.6.0-test6 just now, ...

I find what works well for me is /etc/grub.conf:

title Red Hat Linux (2.6.0-test6)
        root (hd0,8)
        kernel /vmlinuz-2.6.0-test6 ro root=LABEL=/
        initrd /initrd-2.6.0-test6.img

What chokes often is:

title Red Hat Linux (2.6.0-test6)
        root (hd0,8)
        kernel /vmlinuz-2.6.0-test6 ro root=LABEL=/ hdc=ide-scsi
        initrd /initrd-2.6.0-test6.img

Having hdc=ide-scsi matter I think is a clue.

I don't yet know if this apparent correlation will hold up or not.

Pat LaVarre

Re: writable mmc profiles actually are writable

[Pat LaVarre]

Yep, still happy, same in -test6 as in -test5 e.g.

$ dmesg | grep ^sr
sr0: scsi3-mmc drive: 125x/125x caddy
sr0: scsi3-mmc maybe not writeable
sr0: scsi3-mmc writable profile: 0x0002
$
$ grep MMC /proc/sys/dev/cdrom/info
Can write other MMC-RW: 1 ...
$

Pat LaVarre

diff -ur linux-2.6.0-test6/include/scsi/scsi.h linux/include/scsi/scsi.h
--- linux-2.6.0-test6/include/scsi/scsi.h	2003-09-27 18:51:13.000000000 -0600
+++ linux/include/scsi/scsi.h	2003-10-06 16:11:52.000000000 -0600
@@ -74,6 +74,7 @@
 #define CHANGE_DEFINITION     0x40
 #define WRITE_SAME            0x41
 #define READ_TOC              0x43
+#define GET_CONFIGURATION     0x46
 #define LOG_SELECT            0x4c
 #define LOG_SENSE             0x4d
 #define MODE_SELECT_10        0x55
diff -ur linux-2.6.0-test6/include/linux/cdrom.h linux/include/linux/cdrom.h
--- linux-2.6.0-test6/include/linux/cdrom.h	2003-09-27 18:50:06.000000000 -0600
+++ linux/include/linux/cdrom.h	2003-10-06 16:11:52.000000000 -0600
@@ -388,6 +388,7 @@
 #define CDC_DVD_R		0x10000	/* drive can write DVD-R */
 #define CDC_DVD_RAM		0x20000	/* drive can write DVD-RAM */
 #define CDC_MO_DRIVE		0x40000 /* drive is an MO device */
+#define CDC_MMC_RW		0x80000 /* drive is other mmc "writable" "profile" */
 
 /* drive status possibilities returned by CDROM_DRIVE_STATUS ioctl */
 #define CDS_NO_INFO		0	/* if not implemented */
diff -ur linux-2.6.0-test6/drivers/cdrom/cdrom.c linux/drivers/cdrom/cdrom.c
--- linux-2.6.0-test6/drivers/cdrom/cdrom.c	2003-09-27 18:50:10.000000000 -0600
+++ linux/drivers/cdrom/cdrom.c	2003-10-06 17:50:21.934425152 -0600
@@ -426,7 +426,7 @@
 	if ((fp->f_flags & O_NONBLOCK) && (cdi->options & CDO_USE_FFLAGS))
 		ret = cdi->ops->open(cdi, 1);
 	else {
-		if ((fp->f_mode & FMODE_WRITE) && !CDROM_CAN(CDC_DVD_RAM))
+		if ((fp->f_mode & FMODE_WRITE) && !CDROM_CAN(CDC_DVD_RAM|CDC_MMC_RW))
 			return -EROFS;
 
 		ret = open_for_data(cdi);
@@ -2465,6 +2465,10 @@
 	for (cdi=topCdromPtr;cdi!=NULL;cdi=cdi->next)
 	    pos += sprintf(info+pos, "\t%d", CDROM_CAN(CDC_DVD_RAM) != 0);
 
+	pos += sprintf(info+pos, "\nCan write other MMC-RW:");
+	for (cdi=topCdromPtr;cdi!=NULL;cdi=cdi->next)
+	    pos += sprintf(info+pos, "\t%d", CDROM_CAN(CDC_MMC_RW) != 0);
+
 	strcpy(info+pos,"\n\n");
 		
         return proc_dostring(ctl, write, filp, buffer, lenp);
diff -ur linux-2.6.0-test6/drivers/scsi/sr.c linux/drivers/scsi/sr.c
--- linux-2.6.0-test6/drivers/scsi/sr.c	2003-09-27 18:50:13.000000000 -0600
+++ linux/drivers/scsi/sr.c	2003-10-06 17:50:15.781360560 -0600
@@ -67,7 +67,8 @@
 	(CDC_CLOSE_TRAY|CDC_OPEN_TRAY|CDC_LOCK|CDC_SELECT_SPEED| \
 	 CDC_SELECT_DISC|CDC_MULTI_SESSION|CDC_MCN|CDC_MEDIA_CHANGED| \
 	 CDC_PLAY_AUDIO|CDC_RESET|CDC_IOCTLS|CDC_DRIVE_STATUS| \
-	 CDC_CD_R|CDC_CD_RW|CDC_DVD|CDC_DVD_R|CDC_DVD_RAM|CDC_GENERIC_PACKET)
+	 CDC_CD_R|CDC_CD_RW|CDC_DVD|CDC_DVD_R|CDC_DVD_RAM|CDC_GENERIC_PACKET| \
+	 CDC_MMC_RW)
 
 static int sr_probe(struct device *);
 static int sr_remove(struct device *);
@@ -92,6 +93,10 @@
 static void get_sectorsize(struct scsi_cd *);
 static void get_capabilities(struct scsi_cd *);
 
+static void get_configuration(struct scsi_cd *);
+static int scsi_get_configuration(struct scsi_device *sdev,
+	unsigned char *buffer, int len, int timeout, int retries);
+
 static int sr_media_change(struct cdrom_device_info *, int);
 static int sr_packet(struct cdrom_device_info *, struct cdrom_generic_command *);
 
@@ -754,6 +759,7 @@
 	rc = scsi_mode_sense(cd->device, 0, 0x2a, buffer, 128,
 			     SR_TIMEOUT, 3, &data);
 
+	cd->cdi.mask |= CDC_MMC_RW;
 	if (!scsi_status_is_good(rc)) {
 		/* failed, drive doesn't have capabilities mode page */
 		cd->cdi.speed = 1;
@@ -816,6 +822,83 @@
 
 	scsi_release_request(SRpnt);
 	kfree(buffer);
+
+	if (!cd->device->writeable) {
+		printk("%s: scsi3-mmc maybe not writeable\n",
+		       cd->cdi.name);
+		get_configuration(cd);
+	}
+}
+
+static void get_configuration(struct scsi_cd *cd)
+{
+	unsigned char *buffer;
+	int rc, mmc_profile;
+
+	buffer = kmalloc(512, GFP_KERNEL | GFP_DMA);
+	if (!buffer) {
+		printk(KERN_ERR "sr: out of memory.\n");
+		return;
+	}
+
+	rc = scsi_get_configuration(cd->device, buffer, 8,
+			     SR_TIMEOUT, 1);
+	if (!scsi_status_is_good(rc)) {
+		kfree(buffer);
+		return;
+	}
+
+	mmc_profile = ((buffer[6] << 8) + buffer[7]);
+	switch (mmc_profile) {
+	case 0x0001: /* Non-Removable Disk */
+	case 0x0002: /* Removable Disk */
+	case 0x0003: /* Magneto-Optical Erasable */
+	case 0x0005: /* AS-MO */
+	case 0x0012: /* DVD-RAM */
+	case 0x001A: /* DVD+RW */
+	case 0x0022: /* DDCD-RW */
+		printk("%s: scsi3-mmc writable profile: 0x%04x\n",
+		       cd->cdi.name, mmc_profile);
+		cd->device->writeable = 1;
+		if (mmc_profile != 0x001A) { /* != DVD+RW */
+			cd->cdi.mask &= ~CDC_MMC_RW;
+		}
+		break;
+	default:
+		break;
+	}
+
+	kfree(buffer);
+}
+
+/* intended to be block-copy-edit of scsi_lib.c scsi_mode_sense */
+static int
+scsi_get_configuration(struct scsi_device *sdev,
+		  unsigned char *buffer, int len, int timeout, int retries)
+{
+	struct scsi_request *sreq = scsi_allocate_request(sdev, GFP_KERNEL);
+	unsigned char cmd[MAX_COMMAND_SIZE];
+	int ret;
+
+	if (!sreq)
+		return -1;
+
+	memset(&cmd[0], 0, MAX_COMMAND_SIZE);
+	cmd[0] = GET_CONFIGURATION;
+	cmd[8] = len;
+
+	sreq->sr_cmd_len = 0;
+	sreq->sr_sense_buffer[0] = 0;
+	sreq->sr_sense_buffer[2] = 0;
+	sreq->sr_data_direction = DMA_FROM_DEVICE;
+
+	memset(buffer, 0, len);
+
+	scsi_wait_req(sreq, cmd, buffer, len, timeout, retries);
+
+	ret = sreq->sr_result;
+	scsi_release_request(sreq);
+	return ret;
 }
 
 /*

Re: 2 KiB/block loopback found where

[Pat LaVarre]

Can I easily fix my --getss too?

$ sudo losetup /dev/loop0 dd.bin
$
$ dmesg | tail -2
loop: loaded (max 8 devices)
loop:to x800 from lo_blocksize = 0x1000
$
$ sudo blockdev --getss /dev/loop0
512
$ sudo blockdev --getbsz /dev/loop0
4096
$
$ sudo blockdev --getss /dev/scd0
2048
$ sudo blockdev --getbsz /dev/scd0
4096
$

I'm pleased to report, patching drivers/block/loop.c as suggested,
specifically via the patch quoted below, did confuse udf.ko as well with
a loop device as does my real 2 KiB/block device.

Certainly I could and did then relay details to
linux_udf@hpesjro.fc.hp.com in a thread titled "balloc free bit already
set and byte minus one".  But I'd of course like to make my loop device
as perfect a simulation of my real device as practical.  So I ask:

Can I easily fix my --getss too?

Pat LaVarre

diff -Nur linux-2.6.0-test6/drivers/block/loop.c linux/drivers/block/loop.c
--- linux-2.6.0-test6/drivers/block/loop.c	2003-09-27 18:50:29.000000000 -0600
+++ linux/drivers/block/loop.c	2003-10-06 18:06:20.447708984 -0600
@@ -732,7 +732,13 @@
 	mapping_set_gfp_mask(inode->i_mapping,
 			     lo->old_gfp_mask & ~(__GFP_IO|__GFP_FS));
 
+#if 0
 	set_blocksize(bdev, lo_blocksize);
+#else
+	printk(KERN_INFO "loop:"
+		"to x800 from lo_blocksize = 0x%X\n", lo_blocksize);
+	set_blocksize(bdev, 0x800);
+#endif
 
 	lo->lo_bio = lo->lo_biotail = NULL;

Re: writable mmc profiles actually are writable

[Jens Axboe]

On Mon, Oct 06 2003, Pat LaVarre wrote:
> Yep, still happy, same in -test6 as in -test5 e.g.
> 
> $ dmesg | grep ^sr
> sr0: scsi3-mmc drive: 125x/125x caddy
> sr0: scsi3-mmc maybe not writeable
> sr0: scsi3-mmc writable profile: 0x0002
> $
> $ grep MMC /proc/sys/dev/cdrom/info
> Can write other MMC-RW: 1 ...

I don't quite agree with this patch. Please add the GET_CONFIGURATION
stuff to cdrom.c instead so it works with ide-cd as well (ide-scsi is
basically dead, and the number of scsi drives out there isn't exactly
big and increasing). GPCMD_GET_CONFIGURATION already exists. Also, make
DVD-RAM drives set CDC_MMC_RW as well and only check for that at open.

-- 
Jens Axboe

Re: max GiB written per boot

[Jens Axboe]

On Mon, Oct 06 2003, Pat LaVarre wrote:
> With usb2 in linux 2.6.0-test6 just now, ...
> 
> I find what works well for me is /etc/grub.conf:
> 
> title Red Hat Linux (2.6.0-test6)
>         root (hd0,8)
>         kernel /vmlinuz-2.6.0-test6 ro root=LABEL=/
>         initrd /initrd-2.6.0-test6.img
> 
> What chokes often is:
> 
> title Red Hat Linux (2.6.0-test6)
>         root (hd0,8)
>         kernel /vmlinuz-2.6.0-test6 ro root=LABEL=/ hdc=ide-scsi
>         initrd /initrd-2.6.0-test6.img
> 
> Having hdc=ide-scsi matter I think is a clue.
> 
> I don't yet know if this apparent correlation will hold up or not.

Yeah, ide-scsi is broken so you probably shouldn't be using it.

-- 
Jens Axboe

Re: writable mmc profiles actually are writable

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 1109 bytes --]

On Tue, Oct 07, 2003 at 07:38:58AM +0200, Jens Axboe wrote:
> On Mon, Oct 06 2003, Pat LaVarre wrote:
> > Yep, still happy, same in -test6 as in -test5 e.g.
> > 
> > $ dmesg | grep ^sr
> > sr0: scsi3-mmc drive: 125x/125x caddy
> > sr0: scsi3-mmc maybe not writeable
> > sr0: scsi3-mmc writable profile: 0x0002
> > $
> > $ grep MMC /proc/sys/dev/cdrom/info
> > Can write other MMC-RW: 1 ...
> 
> I don't quite agree with this patch. Please add the GET_CONFIGURATION
> stuff to cdrom.c instead so it works with ide-cd as well (ide-scsi is
> basically dead, and the number of scsi drives out there isn't exactly
> big and increasing). GPCMD_GET_CONFIGURATION already exists. Also, make
> DVD-RAM drives set CDC_MMC_RW as well and only check for that at open.

While the number of SCSI CD-ROM drives may be shrinking, the ATAPI drives
behind a USB/ATAPI bridge is skyrocketing.

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

Somebody call an exorcist!
					-- Dust Puppy
User Friendly, 5/16/1998

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: writable mmc profiles actually are writable

[Jens Axboe]

On Mon, Oct 06 2003, Matthew Dharm wrote:
> On Tue, Oct 07, 2003 at 07:38:58AM +0200, Jens Axboe wrote:
> > On Mon, Oct 06 2003, Pat LaVarre wrote:
> > > Yep, still happy, same in -test6 as in -test5 e.g.
> > > 
> > > $ dmesg | grep ^sr
> > > sr0: scsi3-mmc drive: 125x/125x caddy
> > > sr0: scsi3-mmc maybe not writeable
> > > sr0: scsi3-mmc writable profile: 0x0002
> > > $
> > > $ grep MMC /proc/sys/dev/cdrom/info
> > > Can write other MMC-RW: 1 ...
> > 
> > I don't quite agree with this patch. Please add the GET_CONFIGURATION
> > stuff to cdrom.c instead so it works with ide-cd as well (ide-scsi is
> > basically dead, and the number of scsi drives out there isn't exactly
> > big and increasing). GPCMD_GET_CONFIGURATION already exists. Also, make
> > DVD-RAM drives set CDC_MMC_RW as well and only check for that at open.
> 
> While the number of SCSI CD-ROM drives may be shrinking, the ATAPI drives
> behind a USB/ATAPI bridge is skyrocketing.

Doesn't change the picture, if you put it in cdrom.c it will work for
both.

-- 
Jens Axboe

Re: writable mmc profiles actually are writable

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 2145 bytes --]

On Tue, Oct 07, 2003 at 08:48:29AM +0200, Jens Axboe wrote:
> On Mon, Oct 06 2003, Matthew Dharm wrote:
> > On Tue, Oct 07, 2003 at 07:38:58AM +0200, Jens Axboe wrote:
> > > On Mon, Oct 06 2003, Pat LaVarre wrote:
> > > > Yep, still happy, same in -test6 as in -test5 e.g.
> > > > 
> > > > $ dmesg | grep ^sr
> > > > sr0: scsi3-mmc drive: 125x/125x caddy
> > > > sr0: scsi3-mmc maybe not writeable
> > > > sr0: scsi3-mmc writable profile: 0x0002
> > > > $
> > > > $ grep MMC /proc/sys/dev/cdrom/info
> > > > Can write other MMC-RW: 1 ...
> > > 
> > > I don't quite agree with this patch. Please add the GET_CONFIGURATION
> > > stuff to cdrom.c instead so it works with ide-cd as well (ide-scsi is
> > > basically dead, and the number of scsi drives out there isn't exactly
> > > big and increasing). GPCMD_GET_CONFIGURATION already exists. Also, make
> > > DVD-RAM drives set CDC_MMC_RW as well and only check for that at open.
> > 
> > While the number of SCSI CD-ROM drives may be shrinking, the ATAPI drives
> > behind a USB/ATAPI bridge is skyrocketing.
> 
> Doesn't change the picture, if you put it in cdrom.c it will work for
> both.

It will?  I'll have to take your word for that, as I don't immediately see
how that works... then again, much of the higher layers of SCSI are a
mystery to me.

I think I see how this work.... sr.c registers with cdrom.c, and cdrom.c
actually provides the interface to the /dev/scdN node, right?

If that's the case, why is there so much code in sr.c anyway?  The
get_capabilities() function really should be moved, as well as media-change
detection, much of the probe function (dealing with initialization of the
new structre), reading sector sizes, and sr_ioctl.c should probably be
renamed (as it doesn't really do IOCTL processing).

Am I on the right track?  That seems like an awfully long list of things
that suggest against my interpretation....

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

It was a new hope.
					-- Dust Puppy
User Friendly, 12/25/1998

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: writable mmc profiles actually are writable

[Jens Axboe]

On Mon, Oct 06 2003, Pat LaVarre wrote:
> > patches to make the detection more modern ...
> 
> Below appears a copy of an old patch to try op x46 get_configuration any
> time mode page x2a doesn't already say the pdt x05 dvd/cd device is
> writable.

See my comments in another mail.

> > I haven't yet had to deal with dvd+rw or
> > ddcd_rw (is this like mtr?
> 
> Sorry I do not know mtr.

mt rainier, different in spirit but allows the same 2kb writes and
handles the rest in hardware.

> > mo and dvd-ram have been supported for a long
> > time and should be set with CDC_DVD_RAM,
> 
> Sorry I do not understand.  I remember noticing CDC_MO_DRIVE appears
> only drivers/ide/ not in drivers/scsi/.  Weakly I conclude mo doesn't
> work the same in atapi and usb-storage.

IIRC, scsi mo drives are handled by sd, not sr.

> > the disk profiles aren't driven by ide-cd or
> > sr.
> 
> Sorry I do not understand.
> 
> In drivers/scsi/sr.c I find:
> cd->cdi.mask |= CDC_DVD_RAM;
> 
> In drivers/ide/ide-cd.c I find:
> devinfo->mask |= CDC_DVD_RAM;
> 
> I conclude sr and ide-cd tell cdrom whether the device "profile" is
> writable or not.  Is that wrong?

->mask is an inverted capability mask, so if you set CDC_DVD_RAM you
mask that capability.

-- 
Jens Axboe

Re: writable mmc profiles actually are writable

[Jens Axboe]

On Tue, Oct 07 2003, Matthew Dharm wrote:
> On Tue, Oct 07, 2003 at 08:48:29AM +0200, Jens Axboe wrote:
> > On Mon, Oct 06 2003, Matthew Dharm wrote:
> > > On Tue, Oct 07, 2003 at 07:38:58AM +0200, Jens Axboe wrote:
> > > > On Mon, Oct 06 2003, Pat LaVarre wrote:
> > > > > Yep, still happy, same in -test6 as in -test5 e.g.
> > > > > 
> > > > > $ dmesg | grep ^sr
> > > > > sr0: scsi3-mmc drive: 125x/125x caddy
> > > > > sr0: scsi3-mmc maybe not writeable
> > > > > sr0: scsi3-mmc writable profile: 0x0002
> > > > > $
> > > > > $ grep MMC /proc/sys/dev/cdrom/info
> > > > > Can write other MMC-RW: 1 ...
> > > > 
> > > > I don't quite agree with this patch. Please add the GET_CONFIGURATION
> > > > stuff to cdrom.c instead so it works with ide-cd as well (ide-scsi is
> > > > basically dead, and the number of scsi drives out there isn't exactly
> > > > big and increasing). GPCMD_GET_CONFIGURATION already exists. Also, make
> > > > DVD-RAM drives set CDC_MMC_RW as well and only check for that at open.
> > > 
> > > While the number of SCSI CD-ROM drives may be shrinking, the ATAPI drives
> > > behind a USB/ATAPI bridge is skyrocketing.
> > 
> > Doesn't change the picture, if you put it in cdrom.c it will work for
> > both.
> 
> It will?  I'll have to take your word for that, as I don't immediately see
> how that works... then again, much of the higher layers of SCSI are a
> mystery to me.
> 
> I think I see how this work.... sr.c registers with cdrom.c, and cdrom.c
> actually provides the interface to the /dev/scdN node, right?

Precisely.

> If that's the case, why is there so much code in sr.c anyway?  The
> get_capabilities() function really should be moved, as well as media-change
> detection, much of the probe function (dealing with initialization of the
> new structre), reading sector sizes, and sr_ioctl.c should probably be
> renamed (as it doesn't really do IOCTL processing).
> 
> Am I on the right track?  That seems like an awfully long list of things
> that suggest against my interpretation....

Yes you are essentially correct. Some things have been killed from sr a
long time ago, but I generally resist adding new things to sr and ide-cd
unless I really have to. And I've been doing so since 2.2, so I really
don't want to reverse that right now.

It's also confusing why then some things only work if you have ide-cd
loaded, or if you use ide-scsi and sr. It's just a bad idea.

-- 
Jens Axboe

Re: writable mmc profiles actually are writable

[Pat LaVarre]

> From: Jens Axboe ...

Thanks again for your immediate & careful review.

> so it works with ide-cd as well

I do have both usb & atapi sample drives to test, I haven't yet tried
atapi.

> Please add ...
> to cdrom.c instead ... GPCMD_GET_CONFIGURATION

Happily will do.  I'll continue to reply as I progress or not.

Before now, I did not know I could find any one place in the kernel
source to tweak CDC decisions.  To my confused newbie eye, ide-cd.c and
sr.c appeared coded independently to fetch mode page x2A Capabilities
and neglect op x46 Get Configuration in slightly different ways.  I
erroneously had planned to develop an ide-cd.c patch after the sr.c
patch.

> Also, make DVD-RAM drives set CDC_MMC_RW ...

Will do.

> ...

Already I have one reaction to share ...

I'm now vague on how we want /proc/sys/dev/cdrom/info to change?

I see:

In -test6 for dvd_ram we have:

Can write CD-R:         0
Can write CD-RW:        0
Can read DVD:           0
Can write DVD-R:        0
Can write DVD-RAM:      1

I ask:

What do we want the new CDC_MMC_RW line to look like?

Should our new line be a CDC_MMC_RW &~ CDC_DVD_RAM line?  That would
give us the same appearance as the last patch I posted i.e. commonly
different devices would provoke only the two exclusive combinations: 

Can write DVD-RAM:      1
Can write other MMC-RW: 0

Can write DVD-RAM:      0
Can write other MMC-RW: 1

Or should our new line look very different, and appear explicitly set
for any CDC_MMC_RW, perhaps:

Tolerates random write: 1

Or do we prefer some other alternative?

Pat LaVarre

Re: writable mmc profiles actually are writable

[Jens Axboe]

On Tue, Oct 07 2003, Pat LaVarre wrote:
> > Please add ...
> > to cdrom.c instead ... GPCMD_GET_CONFIGURATION
> 
> Happily will do.  I'll continue to reply as I progress or not.

Great, thanks.

> Before now, I did not know I could find any one place in the kernel
> source to tweak CDC decisions.  To my confused newbie eye, ide-cd.c and
> sr.c appeared coded independently to fetch mode page x2A Capabilities
> and neglect op x46 Get Configuration in slightly different ways.  I
> erroneously had planned to develop an ide-cd.c patch after the sr.c
> patch.

You'd end up adding about the same code in both places. The
->generic_packet() hook and cdrom_generic_command was invented to solve
this code duplication. So in cdrom.c you just setup the cgc as needed,
and pass it down to ide-cd or sr. See cdrom_mode_sense() for one of the
many examples.

> Already I have one reaction to share ...
> 
> I'm now vague on how we want /proc/sys/dev/cdrom/info to change?
> 
> I see:
> 
> In -test6 for dvd_ram we have:
> 
> Can write CD-R:         0
> Can write CD-RW:        0
> Can read DVD:           0
> Can write DVD-R:        0
> Can write DVD-RAM:      1
> 
> I ask:
> 
> What do we want the new CDC_MMC_RW line to look like?
> 
> Should our new line be a CDC_MMC_RW &~ CDC_DVD_RAM line?  That would

I think CDC_MMC_RW should be independent of that. The idea was to make
CDC_DVD_RAM really be the hardware type indication and not the pseudo
randomly writable flag that it is now.

> give us the same appearance as the last patch I posted i.e. commonly
> different devices would provoke only the two exclusive combinations: 
> 
> Can write DVD-RAM:      1
> Can write other MMC-RW: 0
> 
> Can write DVD-RAM:      0
> Can write other MMC-RW: 1
> 
> Or should our new line look very different, and appear explicitly set
> for any CDC_MMC_RW, perhaps:
> 
> Tolerates random write: 1
> 
> Or do we prefer some other alternative?

No that is what I had in mind as well, although I would probably have
worded it a bit differently :). But that's detail, please proceed, I
think you are on the right track.

-- 
Jens Axboe

Re: max GiB written per boot

[Randy.Dunlap]

On Tue, 7 Oct 2003 07:57:14 +0200 Jens Axboe <axboe@suse.de> wrote:

| > 
| > Having hdc=ide-scsi matter I think is a clue.
| > 
| > I don't yet know if this apparent correlation will hold up or not.
| 
| Yeah, ide-scsi is broken so you probably shouldn't be using it.

Is there any sense in trying to repair it?
I could spend some time on it...

--
~Randy

Re: max GiB written per boot

[Willem Riede]

On 2003.10.07 18:12, Randy.Dunlap wrote:
> On Tue, 7 Oct 2003 07:57:14 +0200 Jens Axboe <axboe@suse.de> wrote:
> 
> | 
> | Yeah, ide-scsi is broken so you probably shouldn't be using it.
> 
> Is there any sense in trying to repair it?
> I could spend some time on it...

Yes, there is IMHO. The Onstream tape driver (osst) depends on ide-scsi
for controlling the IDE version of the drive (DI-30). 

Fixing ide-scsi is not trivial, though. I've spent some time on it
on and off and have not been all that successful. The biggest problem
I have is the error recovery -- making ide subsystem error handling
happen when the scsi error handler needs it...

Regards, Willem Riede.

Re: max GiB written per boot

[Randy.Dunlap]

On Tue, 7 Oct 2003 18:57:08 -0400 Willem Riede <wrlk@riede.org> wrote:

| On 2003.10.07 18:12, Randy.Dunlap wrote:
| > On Tue, 7 Oct 2003 07:57:14 +0200 Jens Axboe <axboe@suse.de> wrote:
| > 
| > | 
| > | Yeah, ide-scsi is broken so you probably shouldn't be using it.
| > 
| > Is there any sense in trying to repair it?
| > I could spend some time on it...
| 
| Yes, there is IMHO. The Onstream tape driver (osst) depends on ide-scsi
| for controlling the IDE version of the drive (DI-30). 
| 
| Fixing ide-scsi is not trivial, though. I've spent some time on it
| on and off and have not been all that successful. The biggest problem
| I have is the error recovery -- making ide subsystem error handling
| happen when the scsi error handler needs it...

Alan said (a few months back) that he was fixing ide-scsi in 2.4.x-ac
and that it should be forward-ported to 2.6.x, so that is where I
would begin...  I haven't checked it for progress yet.

--
~Randy

Re: max GiB written per boot

[Randy.Dunlap]

On Tue, 7 Oct 2003 18:27:14 -0700 "Randy.Dunlap" <rddunlap@osdl.org> wrote:

| On Tue, 7 Oct 2003 18:57:08 -0400 Willem Riede <wrlk@riede.org> wrote:
| 
| | On 2003.10.07 18:12, Randy.Dunlap wrote:
| | > On Tue, 7 Oct 2003 07:57:14 +0200 Jens Axboe <axboe@suse.de> wrote:
| | > 
| | > | 
| | > | Yeah, ide-scsi is broken so you probably shouldn't be using it.
| | > 
| | > Is there any sense in trying to repair it?
| | > I could spend some time on it...
| | 
| | Yes, there is IMHO. The Onstream tape driver (osst) depends on ide-scsi
| | for controlling the IDE version of the drive (DI-30). 
| | 
| | Fixing ide-scsi is not trivial, though. I've spent some time on it
| | on and off and have not been all that successful. The biggest problem
| | I have is the error recovery -- making ide subsystem error handling
| | happen when the scsi error handler needs it...
| 
| Alan said (a few months back) that he was fixing ide-scsi in 2.4.x-ac
| and that it should be forward-ported to 2.6.x, so that is where I
| would begin...  I haven't checked it for progress yet.

but this hasn't really happened (2.4.x-ac ide-scsi) afaict.

--
~Randy

Re: max GiB written per boot

[Jens Axboe]

On Tue, Oct 07 2003, Randy.Dunlap wrote:
> On Tue, 7 Oct 2003 18:27:14 -0700 "Randy.Dunlap" <rddunlap@osdl.org> wrote:
> 
> | On Tue, 7 Oct 2003 18:57:08 -0400 Willem Riede <wrlk@riede.org> wrote:
> | 
> | | On 2003.10.07 18:12, Randy.Dunlap wrote:
> | | > On Tue, 7 Oct 2003 07:57:14 +0200 Jens Axboe <axboe@suse.de> wrote:
> | | > 
> | | > | 
> | | > | Yeah, ide-scsi is broken so you probably shouldn't be using it.
> | | > 
> | | > Is there any sense in trying to repair it?
> | | > I could spend some time on it...
> | | 
> | | Yes, there is IMHO. The Onstream tape driver (osst) depends on ide-scsi
> | | for controlling the IDE version of the drive (DI-30). 
> | | 
> | | Fixing ide-scsi is not trivial, though. I've spent some time on it
> | | on and off and have not been all that successful. The biggest problem
> | | I have is the error recovery -- making ide subsystem error handling
> | | happen when the scsi error handler needs it...
> | 
> | Alan said (a few months back) that he was fixing ide-scsi in 2.4.x-ac
> | and that it should be forward-ported to 2.6.x, so that is where I
> | would begin...  I haven't checked it for progress yet.
> 
> but this hasn't really happened (2.4.x-ac ide-scsi) afaict.

2.4 and 2.6 also uses different error handling for ide-scsi, so it would
probably not have helped you a lot.

-- 
Jens Axboe

Re: max GiB written per boot

[Pat LaVarre]

> > > Having hdc=ide-scsi matter I think is a clue.
> > > 
> > > I don't yet know if this apparent correlation will hold up or not.
> >
> > Yeah, ide-scsi is broken so you probably shouldn't be using it.
>
> From:  Randy.Dunlap <rddunlap@osdl.org>
> Is there any sense in trying to repair it?
> I could spend some time on it...

Perhaps I may say: yes please!

As an ignorant newbie, myself I'd vote we at least apply the patch:

--- ./post-halloween-2.5.txt	2003-10-09 15:50:29.565344120 -0600
+++ post-halloween.txt	2003-10-09 15:52:28.323290168 -0600
@@ -571,6 +571,8 @@
 - Jens Axboe added the ability to use DMA for writing CDs on
   ATAPI devices. Writing CDs should be much faster than it
   was in 2.4, and also less prone to buffer underruns and the like.
+- Take care to remove ide-scsi.  Booting with hdc=ide-scsi or
+  modprobe'ing ide-scsi may disrupt your kernel.
 - With a recent cdrecord, you also no longer need ide-scsi in order to use
   an IDE CD writer.
 - Ripping audio tracks off of CDs now also uses DMA and should be

If that's not reality, I need more education.

If that is reality, but not what we want to say, then we can balance the
work involved in updating ide-scsi vs. the likely pain of answering all
the later newbies who come here saying I swapped 2.6 into my Red Hat/
etc. 2.4/ 2.5 and life went downhill from there.

Myself I gathered 2.6 experience without ide-scsi only by luck.  I have
a usb drive and an atapi drive, I had to choose one or the other to
begin with, I happened to choose the usb drive, and I happened to
dislike the idea of overriding a kernel default.

I'm no longer sure I remember exactly how I fell back into trying
hdc=ide-scsi.  I know I'm contact with other newbies who take more the
attitude of learning to change as little as possible in how Red Hat
boots, rather than the attitude of learning how to survive with as
little from Red Hat as practical.

Pat LaVarre

Re: writable mmc profiles actually are writable

[Pat LaVarre]

> > Before now, I did not know I could find any
> > one place in the kernel source to tweak CDC
> > decisions.  To my confused newbie eye,
> > ide-cd.c and sr.c appeared coded
> > independently to fetch mode page x2A
> > Capabilities and neglect op x46 ...
>
> The ->generic_packet() hook and
> cdrom_generic_command was invented to solve
> this code duplication.

News to me, thanks.

> So in cdrom.c you just setup the cgc as
> needed, and pass it down to ide-cd or sr. See
> cdrom_mode_sense() for one of the many
> examples ...
> ...
> The idea ... make CDC_DVD_RAM really be the
> hardware type indication and not the pseudo
> randomly writable flag that it is now ...
> ...
> please proceed ...

Will do, now diff'ing against 2.6.0-test7.

> > > I haven't yet had to deal with dvd+rw or
> > > ddcd_rw (is this like mtr?
> > 
> > Sorry I do not know mtr.
> 
> mt rainier, different in spirit

Eh?  Different in spirit than what & how?

> mt rainier, different in spirit but allows the same 2kb
> writes and handles the rest in hardware.

Ah.  Yes some of the buzz re Mt Rainier has reached me before now.  But
I had not heard that writes work well unless actually aligned to more
like 64 KiB boundaries.  I had only heard that the hardware tolerated
misaligned writes, not that misaligned writes work well.

> > mo and dvd-ram have been supported for a long
> > > time and should be set with CDC_DVD_RAM,
> > 
> > ... CDC_MO_DRIVE appears only drivers/ide/ not in
> > drivers/scsi/.  Weakly I conclude mo doesn't
> > work the same in atapi and usb-storage.
> 
> IIRC, scsi mo drives are handled by sd, not sr.

Cluelessly I wonder.  I see if scsi mo works via scsi/sd.c, it works
without benefit of CDC_MO_DRIVE, which makes me wonder why drivers/ide
has a use for CDC_MO_DRIVE.  If I hear no reply, I will assume grep
CDC_MO_DRIVE drivers/ide/* would answer my question for me.  I have no
mo drive, so I do not find the question compelling.

> > > the disk profiles aren't driven by ide-cd or
> > > sr.
> > 
> > In drivers/scsi/sr.c I find: ...
> > In drivers/ide/ide-cd.c I find:
> > ...mask |= CDC_DVD_RAM;
> 
> ->mask is an inverted capability mask, so if you set
> CDC_DVD_RAM you mask that capability.

Ouch, that detail I actually did know already.

Sorry I was unclear.

I meant to say I find a default of & CDC_DVD_RAM == 0 and then the |= to
1 executed only conditionally e.g. sr.c:

        ... [fetch mode page x2A Capabilities] ...
        ...
        if ((buffer[n + 3] & 0x20) == 0) {
                /* can't write DVD-RAM media */
                cd->cdi.mask |= CDC_DVD_RAM;
        } else {
                cd->device->writeable = 1;
        }

In my newbie ignorance, that made me guess that sr_mod.ko, not cdrom.ko,
was deciding whether my usb-storage.ko drive was writable.

Pat LaVarre

Re: writable mmc profiles actually are writable

[Pat LaVarre]

> From: Jens Axboe ...
> Please add the GET_CONFIGURATION stuff to cdrom.c
> instead so it works with ide-cd as well ...

Help!  I do not know Where to add this code.

Tell me again where is the one place I can patch to add discovery of
more capability in both sr and ide-cd devices?

Closer study now again seemingly tells me that ide-cd and sr
independently work to try the op x5A Mode Sense of page x2A Capabilities
and not op x46 Get Configuration.

On this point do we agree?

drivers/ide/ide.cd ide_cdrom_get_capabilities speaks thru
drivers/cdrom/cdrom.c cdrom_mode_sense, aye.

drivers/scsi/sr.c get_capabilities speaks instead thru
drivers/scsi/scsi_lib.c scsi_mode_sense.

printk tells me with a usb-storage.ko drive I get as far as 'dd: opening
`/dev/scd0': Read-only file system' without cdrom.c cdrom_mode_sense
ever being called.  With an ide-cd.ko driver, I do see cdrom_mode_sense
called.  The ide-cd command is -x 5A 00 2A:00:00:00 00 00:18 00 -i x18,
rather than the less polite -x 5A 00 2A:00:00:00 00 00:80 00 -i x80 of
sr.

Does a place exist in drivers/cdrom where I can insert code to run after
ide-cd and sr finish their op x5A Mode Sense?

Have I misunderstood your request?  Did you actually mean to tell me to
go so far as to move the op x5A Mode Sense out of ide-cd and out of sr
and into cdrom?  If indeed I should move those out, to where should I
move them?

> The ->generic_packet() hook and
> cdrom_generic_command was invented to solve
> this code duplication.

Found, thank you:

I see cdrom.c cdrom_mode_sense speaks thru cdo->generic_packet.

I see include/linux/cdrom.h defines a struct cdrom_generic_command that
field-for-field matches other simplifications of scsi/sg.h sg_io_hdr
that appear on the web, such as my own plscsi and gccscsi.

> [op x46] GPCMD_GET_CONFIGURATION already exists.

Found, thank you.

Also op x5A GPCMD_MODE_SENSE_10 and page x2A GPMODE_CAPABILITIES_PAGE in
ide-cd.

Also op x5A MODE_SENSE_10 and anonymous x2a in sr.

> make DVD-RAM drives set CDC_MMC_RW as well
> and only check for that at open.

This part I still think I understand how to do.

> why then some things only work if you have ide-cd
> loaded, or if you use ide-scsi and sr ... a bad idea.

Aye.

Pat LaVarre

Re: max GiB written per boot

[Pat LaVarre]

> > > ...
> >
> > ... always ... an oops ... Must be fixed.
>
> Here's another ...

I remember now where I got the idea people don't want to hear about how
easily I can provoke "kernel NULL pointer dereference":

http://www.tux.org/lkml/#s4-3
includes such not-completely-true claims as:

"(REG) Don't even bother posting an Oops if you haven't run it through
ksymoops to decode the symbol addresses. The report will be ignored
because it contains too little useful information."

That was the convention I was trying to follow before I was told
differently.

Pat LaVarre.

Re: writable mmc profiles actually are writable

[Pat LaVarre]

> > Help!  I do not know Where to add this code.

Just now I weakly concluded somewhere I need to add:

op x46 GPCMD_GET_CONFIGURATION
op x5A GPCMD_MODE_SENSE_10 of page x2A GPMODE_CAPABILITIES_PAGE

To let me proceed to write source without yet knowing where to insert
it, I will insert it to a known wrong place that mostly works.

Specifically, I will add code to drivers/cdrom/cdrom.c cdrom_open, to
try to clear the CDROM_CAN(CDC_MMC_WR) incapability only after some
tries cdrom_open with (fp->f_mode & FMODE_WRITE).

I believe this won't actually work.  I believe this will teach only
cdrom.ko to pass thru the writes, but then sr_mod.ko will still refuse
them.  But I will now go prove or disprove that theory.

Pat LaVarre

Re: writable mmc profiles actually are writable

[Pat LaVarre]

Ouch.

make reminds me cdi->ops->capability is a "read-only-member".

I interpret that gcc warning to mean we intend to write the ->capability
when we allot it, and never again thereafter.

I'm therefore now guessing the patch we want is:

We add a ->capability of CDC_MMC_WR to both ide-cd and sr.  We teach
both of those lower level drivers to ->mask this capability by default,
but sometime thereafter we have cdrom author a few cdb's to decide to
unmask that capability for any device with one of the seven standard mmc
profiles that by definition imply feature x0020 random-writable.

Pat LaVarre

Re: writable mmc profiles actually are writable

[Pat LaVarre]

May our review recommence!

The patch here changes four source files.

Please tell me if I have or I have not correctly patched as many as
three of the four source files:

include/linux/cdrom.h
drivers/ide/ide-cd.c
drivers/scsi/sr.c

I'm confident I have not correctly patched the source file:

drivers/cdrom/cdrom.c

I plead for review by eye of this patch, but I do Not recommend applying
this patch.  As yet this patch erroneously makes all /dev/scd*
writeable, just like the four-line patch that began this thread.

Pat LaVarre

diff -Nur linux-2.6.0-test7/include/linux/cdrom.h linux/include/linux/cdrom.h
--- linux-2.6.0-test7/include/linux/cdrom.h	2003-10-08 13:24:00.000000000 -0600
+++ linux/include/linux/cdrom.h	2003-10-10 15:37:16.000000000 -0600
@@ -388,6 +388,7 @@
 #define CDC_DVD_R		0x10000	/* drive can write DVD-R */
 #define CDC_DVD_RAM		0x20000	/* drive can write DVD-RAM */
 #define CDC_MO_DRIVE		0x40000 /* drive is an MO device */
+#define CDC_MMC_WR		0x80000	/* profile includes random write */
 
 /* drive status possibilities returned by CDROM_DRIVE_STATUS ioctl */
 #define CDS_NO_INFO		0	/* if not implemented */
diff -Nur linux-2.6.0-test7/drivers/ide/ide-cd.c linux/drivers/ide/ide-cd.c
--- linux-2.6.0-test7/drivers/ide/ide-cd.c	2003-10-08 13:24:04.000000000 -0600
+++ linux/drivers/ide/ide-cd.c	2003-10-10 15:36:33.000000000 -0600
@@ -2822,7 +2822,7 @@
 				CDC_MEDIA_CHANGED | CDC_PLAY_AUDIO | CDC_RESET |
 				CDC_IOCTLS | CDC_DRIVE_STATUS | CDC_CD_R |
 				CDC_CD_RW | CDC_DVD | CDC_DVD_R| CDC_DVD_RAM |
-				CDC_GENERIC_PACKET | CDC_MO_DRIVE,
+				CDC_GENERIC_PACKET | CDC_MO_DRIVE | CDC_MMC_WR,
 	.generic_packet		= ide_cdrom_packet,
 };
 
@@ -2832,7 +2832,7 @@
 	struct cdrom_device_info *devinfo = &info->devinfo;
 
 	devinfo->ops = &ide_cdrom_dops;
-	devinfo->mask = 0;
+	devinfo->mask = CDC_MMC_WR;
 	devinfo->speed = CDROM_STATE_FLAGS(drive)->current_speed;
 	devinfo->capacity = nslots;
 	devinfo->handle = (void *) drive;
diff -Nur linux-2.6.0-test7/drivers/scsi/sr.c linux/drivers/scsi/sr.c
--- linux-2.6.0-test7/drivers/scsi/sr.c	2003-10-08 13:24:03.000000000 -0600
+++ linux/drivers/scsi/sr.c	2003-10-10 15:48:12.322686720 -0600
@@ -67,7 +67,8 @@
 	(CDC_CLOSE_TRAY|CDC_OPEN_TRAY|CDC_LOCK|CDC_SELECT_SPEED| \
 	 CDC_SELECT_DISC|CDC_MULTI_SESSION|CDC_MCN|CDC_MEDIA_CHANGED| \
 	 CDC_PLAY_AUDIO|CDC_RESET|CDC_IOCTLS|CDC_DRIVE_STATUS| \
-	 CDC_CD_R|CDC_CD_RW|CDC_DVD|CDC_DVD_R|CDC_DVD_RAM|CDC_GENERIC_PACKET)
+	 CDC_CD_R|CDC_CD_RW|CDC_DVD|CDC_DVD_R|CDC_DVD_RAM|CDC_GENERIC_PACKET| \
+	 CDC_MMC_WR)
 
 static int sr_probe(struct device *);
 static int sr_remove(struct device *);
@@ -327,8 +328,12 @@
 	}
 
 	if (rq_data_dir(SCpnt->request) == WRITE) {
-		if (!cd->device->writeable)
-			return 0;
+		if (!cd->device->writeable) {
+			if ((cd->cdi.mask & CDC_MMC_WR) != 0) {
+				return 0;
+			}
+			cd->device->writeable = 1;
+		}
 		SCpnt->cmnd[0] = WRITE_10;
 		SCpnt->sc_data_direction = SCSI_DATA_WRITE;
 	} else if (rq_data_dir(SCpnt->request) == READ) {
@@ -550,7 +555,7 @@
 
 	cd->cdi.ops = &sr_dops;
 	cd->cdi.handle = cd;
-	cd->cdi.mask = 0;
+	cd->cdi.mask = CDC_MMC_WR;
 	cd->cdi.capacity = 1;
 	sprintf(cd->cdi.name, "sr%d", minor);
 
diff -Nur linux-2.6.0-test7/drivers/cdrom/cdrom.c linux/drivers/cdrom/cdrom.c
--- linux-2.6.0-test7/drivers/cdrom/cdrom.c	2003-10-08 13:24:02.000000000 -0600
+++ linux/drivers/cdrom/cdrom.c	2003-10-10 15:45:59.648856208 -0600
@@ -408,6 +408,24 @@
 	return 0;
 }
 
+/* Say if profile includes random write or not.
+ */
+static int cdrom_cdc_mmc_wr(struct cdrom_device_info *cdi)
+{
+	printk(KERN_INFO "cdrom_cdc_mmc_wr\n");
+	if (CDROM_CAN(CDC_DVD_RAM)) {
+		return 1;
+	}
+#if 0 /* FIXME */
+#error "try op x5A GPCMD_MODE_SENSE_10 of page x2A GPMODE_CAPABILITIES_PAGE"
+#error "return 0 if failed"
+#error "try op x46 GPGPCMD_GET_CONFIGURATION"
+#error "return 0 if failed"
+#error "return 0 if not random writable profile"
+#endif
+	return 1;
+}
+
 /* We use the open-option O_NONBLOCK to indicate that the
  * purpose of opening is only for subsequent ioctl() calls; no device
  * integrity checks are performed.
@@ -426,8 +444,12 @@
 	if ((fp->f_flags & O_NONBLOCK) && (cdi->options & CDO_USE_FFLAGS))
 		ret = cdi->ops->open(cdi, 1);
 	else {
-		if ((fp->f_mode & FMODE_WRITE) && !CDROM_CAN(CDC_DVD_RAM))
-			return -EROFS;
+		if ((fp->f_mode & FMODE_WRITE) && !CDROM_CAN(CDC_MMC_WR)) {
+			if (!cdrom_cdc_mmc_wr(cdi)) {
+				return -EROFS;
+			}
+			cdi->mask &= ~CDC_MMC_WR;
+		}
 
 		ret = open_for_data(cdi);
 	}
@@ -2406,6 +2428,10 @@
 	for (cdi=topCdromPtr;cdi!=NULL;cdi=cdi->next)
 	    pos += sprintf(info+pos, "\t%d", CDROM_CAN(CDC_DVD_RAM) != 0);
 
+	pos += sprintf(info+pos, "\nTolerates random write:");
+	for (cdi=topCdromPtr;cdi!=NULL;cdi=cdi->next)
+	    pos += sprintf(info+pos, "\t%d", CDROM_CAN(CDC_MMC_WR) != 0);
+
 	strcpy(info+pos,"\n\n");
 		
         return proc_dostring(ctl, write, filp, buffer, lenp);

Re: writable mmc profiles actually are writable

[Pat LaVarre]

I have a new guess of precisely when we want cdrom to probe for
capabilities beyond those discovered by ide-cd and sr.

I'm now guessing we want to author additional cdb's at the end of
register_cdrom, just after such optional messages as 'cdrom: drive
"/dev/sr0" registered'.

To leap to that conclusion, I reasoned as follows.  Somebody tell me how
wrong I am?

Pat LaVarre

1)

I found cdrom.c ERRLOGMASK.  I see our default is to pass thru just the
CD_WARNING cdrom: messages.  I see the last commented-out #define means
pass thru all the defined types of cdrom: messages.

2)

I see with usb cdb's unmasked, hot plug spews the dmesg:

Initializing USB Mass Storage driver...
scsi0 : SCSI emulation for USB Mass Storage devices
x 12 00 00 00 24 00
x 12 00 00 00 C0 00
  Vendor: Iomega    Model: RRD               Rev: 36.D
  Type:   CD-ROM                             ANSI SCSI revision: 02
WARNING: USB Mass Storage data integrity not assured
USB Mass Storage device found at 2
drivers/usb/core/usb.c: registered new driver usb-storage
USB Mass Storage support registered.
updfstab: numerical sysctl 1 23 is obsolete.

3)

I see with cdrom messages also unmasked, the one block write `sudo dd
of=/dev/scd0 bs=2K count=1 if=/dev/zero` provokes the spew:

x 00 00 00 00 00 00
x 03 00 00 00 12 00
x 00 00 00 00 00 00
x 5A 00 2A 00 00 00 00 00 80 00
sr0: scsi3-mmc drive: 125x/125x caddy
cdrom: entering register_cdrom
Uniform CD-ROM driver Revision: 3.12
cdrom: drive "/dev/sr0" registered
Attached scsi CD-ROM sr0 at scsi0, channel 0, id 0, lun 0
cdrom: entering cdrom_open
cdrom_cdc_mmc_wr
cdrom: entering open_for_data
x 00 00 00 00 00 00
cdrom: drive_status=4
cdrom: entering cdrom_count_tracks
x 43 00 00 00 00 00 00 00 0C 00
x 43 02 00 00 00 00 01 00 0C 00
cdrom: track 1: format=2, ctrl=4
cdrom: disc has 1 tracks: 0=audio 1=data 0=Cd-I 0=XA
cdrom: all seems well, opening the device.
x 25 00 00 00 00 00 00 00 00 00
cdrom: opening the device gave me 0.
x 1E 00 00 00 01 00
cdrom: door locked.
cdrom: device opened successfully.
cdrom: Use count for "/dev/sr0" now 1
x 00 00 00 00 00 00
x 43 00 00 00 00 00 00 00 0C 40
x 43 00 00 00 00 00 00 00 0C 00
x 43 00 00 00 00 00 01 00 0C 00
x 28 00 00 00 00 00 00 00 02 00
x 2A 00 00 00 00 00 00 00 02 00
cdrom: entering cdrom_release
cdrom: Use count for "/dev/sr0" now zero
cdrom: Unlocking door!
x 1E 00 00 00 00 00

Re: writable mmc profiles actually are writable

[Pat LaVarre]

I have begun again with the new patch you saw me almost finish
developing here.

List: linux-scsi
Subject: [PATCH] CDC_MMC_WR
http://marc.theaimsgroup.com/?l=linux-scsi&m=106583271114657

Pat LaVarre

Re: aligned /dev/scd$n reads less rare how

[Pat LaVarre]

> Anything easy I can do
> to make aligned reads more common?

Instead we first could/ should ask:

How does Linux align write/read of cd-rw/dvd+rw?

We know 2.6 vs. 2.4 newly connects cd-rw as writeable, by
http://kniggit.net/wwol26.html of l=kernelnewbies.  And "everybody
knows" cd-rw needs 64 KiB write alignment, dvd+rw needs 32 KiB write
alignment, and both prefer write alignment in reads, right?  How then
does Linux hint at the mounted fs to align write/read to something other
than the 0.5 KiB that was optimal for legacy hdd?

> I ask because often thru /dev/scd$n
> I see seemingly pointless misalignment,
> ...
> $ sudo blockdev --flushbufs /dev/scd0
> $ time sudo dd if=/dev/scd0 bs=1M
> ...
> usb-storage:  x 28 00 00 00 00 00 00 00 20 00

Initially we see aligned in address and length to x20 blocks = 64 KiB.

> usb-storage:  x 28 00 00 00 00 20 00 00 24 00

Soon we see aligned in address but, whoops, not in length.

> usb-storage:  x 28 00 00 00 00 44 00 00 40 00
> usb-storage:  x 28 00 00 00 00 84 00 00 40 00
> usb-storage:  x 28 00 00 00 00 C4 00 00 40 00
> ...
> usb-storage:  x 28 00 00 07 FE C4 00 00 40 00

Mostly we see contiguous after a misaligned access, so not aligned in
address though yes aligned in length.  Ouch.

Pat LaVarre