Re: Misbehavior of md-raid RAID on failed NVMe.

[Pavel <pavel2000@areainter.net>]

08.06.2022 23:52, Wol пишет:
> On 08/06/2022 04:48, Pavel wrote:
>
> Did you dd the raid device (/dev/md0 for example), or the individual 
> nvme devices?

There was LVM over /dev/md0, and dd transferred LVM volumes data.

>> While data in transfer, kernel started IO errors reporting on one of 
>> NVMe devices. (dmesg output below)
>> But md-raid not reacted on them in any way. RAID array not went into 
>> any failed state, and "clean" state reported all the time.
>
> This is actually normal, correct and expected behaviour. If the raid 
> layer does not report a problem to dd, the data should have copied 
> correctly. And raid really only reports problems if it gets write 
> failures.

Yes, but data was not copied correctly.

> Unfortunately, you're missing a lot of detail to help us diagnose the 
> problem. What raid level are you using, for starters. It sounds like 
> there is a problem, but as Mariusz implies, it looks like a faulty 
> nVME device. And if that device is lying to linux, as appears likely 
> (my guess is that raid is trying to fix the data, and the drive is 
> just losing the writes),

Feel free to ask. Raid level: RAID 1, built over partitions on two NVMe 
devices.

Yes, drive is "just" losing the writes. But there is nothing "to fix" on 
RAID level.
 From my user POV, RAID should detect the loss and take appropriate 
actions (mark device as failed).

I don't know, if NVMe layer lies to kernel or not, but I clearly see
"I/O error, dev nvme0n1, sector 1297536456 op 0x1:(WRITE) flags 0x0 
phys_seg 1 prio class 0"
messages, and I expect they clearly mean write failure.

> then there is precious little we can do about it.

As a kernel user, I did all I might to do - posted an report here.
As a kernel developers, you can do a bit more, than users.

Thanks for your answers.

Regards,
Pavel.