fdisk does not allow overwrite default/suggested value

fdisk does not allow overwrite default/suggested value

[Istvan Gabor]

Dear util-linux developers:

I'v been using cfdisk and fdisk for a long time.
Currently I have util-linux 2.31.1 in openSUSE Leap 42.3.

Recently I've run into this issue:

I use USB hard drive docker and the system gets
false optimal_io_size values for hard drives.

For a detailed description of the problem, see:

Linux SSD partition alignment – problems with external USB-to-SATA 
controllers – I

https://linux-blog.anracom.com/2018/12/03/linux-ssd-partition-alignment-problems-with-external-usb-to-sata-controllers-i/

I wanted to use fdisk for partitioning a 1 TB disk.
fdisk wanted to start the first partition at sector
65535 (based on the wrong optimal_io_size value, I
guess). I tried to overwrite the suggested value with
2048 but got "value out of range" error.

My point is that fdisk insist to use its suggested
value and don't let the user set other value.

In my opinion it is a bad policy, the program
should allow to overwrite the suggested value
and use it. If it is dangerous, then there should
be a specific option to enable the behavior or
double check if the user really wants to use the
value entered. But it should be possible to use
other values than the offered ones, especially if the
offered value is wrong.

Best regards,

Istvan

Re: fdisk does not allow overwrite default/suggested value

[Martin K. Petersen]

Istvan,

> https://linux-blog.anracom.com/2018/12/03/linux-ssd-partition-alignment-problems-with-external-usb-to-sata-controllers-i/

First of all, the ATA protocol can only express logical and physical
block sizes. So the optimal I/O size is always 0 when a disk is
connected via a "native" ATA controller.

USB storage, however, is a SCSI protocol. In that configuration it is up
to the USB bridge device to translate properties reported by the
attached ATA device to the relevant fields in the SCSI protocol. And
this is where several devices get it wrong (despite a fairly
comprehensive document describing how each field should be translated).

The SCSI block storage protocol permits reporting an optimal I/O
size. Since there is no corresponding ATA parameter, it means that the
value in your case inevitably comes from the USB bridge firmware. There
is nothing to base this value on from the ATA device itself, so the
bridge firmware authors either made it up or forgot to zero the field.

As a result of reports similar to the one above, I added an additional
heuristic to the SCSI disk driver code a while ago. The kernel will now
reject a device-reported optimal transfer size if it is not a multiple
of the physical block size. So in the buggy USB-SATA bridge case above
the optimal transfer size should be reported to user space as 0 and
therefore fdisk should align on a 4096-byte (physical block size)
boundary.

Note that this is somewhat orthogonal to your request about being able
to override the reported values in fdisk. I just thought some of the
text in the links you posted warranted clarification.

The question is why your kernel does not have the patch adding this
additional heuristic?

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: fdisk does not allow overwrite default/suggested value

[Karel Zak]

On Tue, Jul 16, 2019 at 03:10:26PM +0200, Istvan Gabor wrote:
> Dear util-linux developers:
> 
> I'v been using cfdisk and fdisk for a long time.
> Currently I have util-linux 2.31.1 in openSUSE Leap 42.3.
> 
> Recently I've run into this issue:
> 
> I use USB hard drive docker and the system gets
> false optimal_io_size values for hard drives.
> 
> For a detailed description of the problem, see:
> 
> Linux SSD partition alignment – problems with external USB-to-SATA
> controllers – I
> 
> https://linux-blog.anracom.com/2018/12/03/linux-ssd-partition-alignment-problems-with-external-usb-to-sata-controllers-i/
> 
> I wanted to use fdisk for partitioning a 1 TB disk.
> fdisk wanted to start the first partition at sector
> 65535 (based on the wrong optimal_io_size value, I
> guess). I tried to overwrite the suggested value with
> 2048 but got "value out of range" error.

Since v2.27 we use this patch:
https://github.com/karelzak/util-linux/commit/acb7651f8897ae73d0f45dd75bc87630001c61b9

to ignore misaligned optimal I/O size, so the question is why your
fdisk follows 65535 (it would be nice to have output from 
"LIBFDISK_DEBUG=all fdisk -l <device>").

Anyway, you can move begin of the first partition in expert menu ('x')
by command 'b'.

It's also possible to use --sector-size, this option overrides logical
and physical sector size, and internally used io-size.

> My point is that fdisk insist to use its suggested
> value and don't let the user set other value.
> 
> In my opinion it is a bad policy, the program
> should allow to overwrite the suggested value
> and use it. If it is dangerous, then there should

It would be possible to add --set-io-{min,opt,...} command line
options, I'll think about it but it's Pandora's box...

> be a specific option to enable the behavior or
> double check if the user really wants to use the
> value entered. But it should be possible to use
> other values than the offered ones, especially if the
> offered value is wrong.

The problem is that the rest of the system (mkfs, LUKS, DM/lvm, ...)
still see wrong IO limits. The right way is to fix kernel.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com