dd, close(), sync, and the Linux disk cache

dd, close(), sync, and the Linux disk cache

[Stephen Warren]

Jens et. al.,

If I dd to an SD card (e.g. via a USB SD card reader), is it required to 
run sync afterward in order to guarantee that all written data is 
written to the SD card? I'm running dd with a simple command-line like 
"dd if=file.img of=/dev/sdc".

In practice on my system, strace shows me that dd completes its write 
operations relatively quickly, since many are buffered in the kernel. 
However, dd's call to close() takes a very long time. I believe this 
delay is due to the kernel flushing all the buffers to disk, and so a 
sync is not required. I found code in the kernel that does appear to do 
this:

https://elixir.bootlin.com/linux/v5.5/source/fs/block_dev.c#L2145
dev_blk_fops.release = blkdev_close
blkdev_close calls blkdev_put
blkdev_put calls __blkdev_put
__blkdev_put calls sync_blockdev if the device open count is 0
sync_blockdev calls __sync_blockdev with the wait flag set
__sync_blockdev calls filemap_write_and_wait

However, Simon finds that when he dd's to an SD card, he needs to 
execute sync afterward to avoid a corrupted SD card. I'm trying to 
understand what the difference is, and whether sync is the correct 
solution for this issue, or just something that happens to take some 
time (e.g. to flush buffers to the sytem's main hard disk) during which 
some other issue is typically resolved.

For reference, I've seen the "no sync required" behaviour on my laptop 
with a built-in PCIe SD reader which shows up as /dev/mmcblkN and on my 
desktop using a USB SD reader which shows up as /dev/sdN. Simon is I 
believe using a USB SD card reader, which is actually a mux device that 
allows switching the SD card between hosts, with the host->host switch 
happening immediately after the dd (or dd+sync) completes.

Re: dd, close(), sync, and the Linux disk cache

[Christoph Hellwig]

On Wed, Feb 12, 2020 at 05:02:43PM -0700, Stephen Warren wrote:
> Jens et. al.,
> 
> If I dd to an SD card (e.g. via a USB SD card reader), is it required to run
> sync afterward in order to guarantee that all written data is written to the
> SD card? I'm running dd with a simple command-line like "dd if=file.img
> of=/dev/sdc".

Yes.  Or use of=dsync on the dd command line.

Re: dd, close(), sync, and the Linux disk cache

[Stephen Warren]

On 2/19/20 9:32 AM, Christoph Hellwig wrote:
> On Wed, Feb 12, 2020 at 05:02:43PM -0700, Stephen Warren wrote:
>> Jens et. al.,
>>
>> If I dd to an SD card (e.g. via a USB SD card reader), is it required to run
>> sync afterward in order to guarantee that all written data is written to the
>> SD card? I'm running dd with a simple command-line like "dd if=file.img
>> of=/dev/sdc".
> 
> Yes.  Or use of=dsync on the dd command line.

Can you explain further why it's necessary given that the kernel 
explicitly blocks execution of close() to flush buffers to disk, 
assuming the process is the last one with the device open? Am I 
misinterpreting the code path I mentioned later in my email? In 
practice, I can see this happening when I use dd.

Re: dd, close(), sync, and the Linux disk cache

[Alberto Faria]

Hi,

> Can you explain further why it's necessary given that the kernel
> explicitly blocks execution of close() to flush buffers to disk,
> assuming the process is the last one with the device open? Am I
> misinterpreting the code path I mentioned later in my email? In
> practice, I can see this happening when I use dd.

From what I could gather empirically, close() does write dirty pages
from the block device's page cache to disk but does not submit a
"flush" request to the device driver. Depending on the driver and
device, this may be necessary to ensure that data is persistently
stored (e.g., if the device features an internal write-back cache).
Performing a sync command with the block device as an argument, or
reopening the device and invoking fsync() or fdatasync(), submits
and awaits that request.

The existence or non-existence of a device-internal write-back
cache, and differences in the timing and policies of such a cache,
may explain why in some cases the sync appears necessary and in
others it doesn't.

Alberto Faria