* XFS issues with NVMe SSD
@ 2017-01-22 1:20 Bond Masuda
2017-01-23 13:41 ` Eric Sandeen
0 siblings, 1 reply; 9+ messages in thread
From: Bond Masuda @ 2017-01-22 1:20 UTC (permalink / raw)
To: linux-xfs
Hi linux-xfs list:
I've been having XFS file system corruption issues when used with Intel
NVMe SSD in Fedora 25 Linux as reported here:
https://bugzilla.redhat.com/show_bug.cgi?id=1402533
I don't know if this is XFS issue or not, but ext4 on the same Intel
NVMe SSD doesn't appear to be a problem. Are linux/xfs devs already
aware of the problem?
Bond
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: XFS issues with NVMe SSD
2017-01-22 1:20 XFS issues with NVMe SSD Bond Masuda
@ 2017-01-23 13:41 ` Eric Sandeen
2017-01-23 13:50 ` Brian Foster
0 siblings, 1 reply; 9+ messages in thread
From: Eric Sandeen @ 2017-01-23 13:41 UTC (permalink / raw)
To: Bond Masuda, linux-xfs
On 1/21/17 7:20 PM, Bond Masuda wrote:
> Hi linux-xfs list:
>
> I've been having XFS file system corruption issues when used with
> Intel NVMe SSD in Fedora 25 Linux as reported here:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1402533
>
> I don't know if this is XFS issue or not, but ext4 on the same Intel
> NVMe SSD doesn't appear to be a problem. Are linux/xfs devs already
> aware of the problem?
>
> Bond
Well, the bug languished under anaconda, so no, we haven't seen it.
It's pretty light on details. Please provide the install logs, dmesg output,
xfs_repair output etc. I'll update the bug.
-Eric
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: XFS issues with NVMe SSD
2017-01-23 13:41 ` Eric Sandeen
@ 2017-01-23 13:50 ` Brian Foster
2017-01-23 13:53 ` Christoph Hellwig
0 siblings, 1 reply; 9+ messages in thread
From: Brian Foster @ 2017-01-23 13:50 UTC (permalink / raw)
To: Eric Sandeen; +Cc: Bond Masuda, linux-xfs
On Mon, Jan 23, 2017 at 07:41:58AM -0600, Eric Sandeen wrote:
> On 1/21/17 7:20 PM, Bond Masuda wrote:
> > Hi linux-xfs list:
> >
> > I've been having XFS file system corruption issues when used with
> > Intel NVMe SSD in Fedora 25 Linux as reported here:
> >
> > https://bugzilla.redhat.com/show_bug.cgi?id=1402533
> >
> > I don't know if this is XFS issue or not, but ext4 on the same Intel
> > NVMe SSD doesn't appear to be a problem. Are linux/xfs devs already
> > aware of the problem?
> >
> > Bond
>
> Well, the bug languished under anaconda, so no, we haven't seen it.
>
> It's pretty light on details. Please provide the install logs, dmesg output,
> xfs_repair output etc. I'll update the bug.
>
Since this is limited hardware (I don't have access to such hardware, at
least), it might also be helpful to see if you can isolate the problem
from a full distro install. For example, can you format such a device
external to the rootfs, mount, copy some stuff and reproduce the
corruption after some sequence of unmount/remount/reboot operations?
>From the bug, it does seem like a reboot is a consistent factor to
reproduce the corruption. I wonder whether it's enough to have the
device mounted at reboot time, or if being a rootfs device is a primary
factor...
Brian
> -Eric
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: XFS issues with NVMe SSD
2017-01-23 13:50 ` Brian Foster
@ 2017-01-23 13:53 ` Christoph Hellwig
2017-01-23 21:25 ` Chris Murphy
0 siblings, 1 reply; 9+ messages in thread
From: Christoph Hellwig @ 2017-01-23 13:53 UTC (permalink / raw)
To: Brian Foster; +Cc: Eric Sandeen, Bond Masuda, linux-xfs
On Mon, Jan 23, 2017 at 08:50:48AM -0500, Brian Foster wrote:
> Since this is limited hardware (I don't have access to such hardware, at
> least), it might also be helpful to see if you can isolate the problem
> from a full distro install. For example, can you format such a device
> external to the rootfs, mount, copy some stuff and reproduce the
> corruption after some sequence of unmount/remount/reboot operations?
I have plenty of NVMe hardware, although none of them is Intel and I've
never seen corruption like that. I also don't run Fedora, though :)
Does anyone know if the Fedora installer does a fstrim run or something
similar? I remember that most Intel NVMe devices had some pretty severe
deallocate (aka discard on NVMe) bugs. A firmware update might be a
good start in that case.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: XFS issues with NVMe SSD
2017-01-23 13:53 ` Christoph Hellwig
@ 2017-01-23 21:25 ` Chris Murphy
2017-01-23 21:27 ` Eric Sandeen
0 siblings, 1 reply; 9+ messages in thread
From: Chris Murphy @ 2017-01-23 21:25 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Brian Foster, Eric Sandeen, Bond Masuda, linux-xfs
On Mon, Jan 23, 2017 at 6:53 AM, Christoph Hellwig <hch@infradead.org> wrote:
> On Mon, Jan 23, 2017 at 08:50:48AM -0500, Brian Foster wrote:
>> Since this is limited hardware (I don't have access to such hardware, at
>> least), it might also be helpful to see if you can isolate the problem
>> from a full distro install. For example, can you format such a device
>> external to the rootfs, mount, copy some stuff and reproduce the
>> corruption after some sequence of unmount/remount/reboot operations?
>
> I have plenty of NVMe hardware, although none of them is Intel and I've
> never seen corruption like that. I also don't run Fedora, though :)
>
> Does anyone know if the Fedora installer does a fstrim run or something
> similar? I remember that most Intel NVMe devices had some pretty severe
> deallocate (aka discard on NVMe) bugs. A firmware update might be a
> good start in that case.
The installer doesn't do fstrim or blkdiscard before installation; but
on my SSDs at least mkfs.xfs and mkfs.btrfs result in trim. So one
possible test would be to manually mkfs with -k, and then if it's a
live media installation, install manually from the booted live
environment with:
rsync -pogAXtlHrDx --exclude /dev/ --exclude /proc/ --exclude /sys/
--exclude /run/ --exclude /boot/*rescue* --exclude /etc/machine-id
/run/install/source/ /mnt/sysimage
--
Chris Murphy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: XFS issues with NVMe SSD
2017-01-23 21:25 ` Chris Murphy
@ 2017-01-23 21:27 ` Eric Sandeen
2017-01-23 21:33 ` Chris Murphy
0 siblings, 1 reply; 9+ messages in thread
From: Eric Sandeen @ 2017-01-23 21:27 UTC (permalink / raw)
To: Chris Murphy, Christoph Hellwig; +Cc: Brian Foster, Bond Masuda, linux-xfs
On 1/23/17 3:25 PM, Chris Murphy wrote:
> On Mon, Jan 23, 2017 at 6:53 AM, Christoph Hellwig <hch@infradead.org> wrote:
>> On Mon, Jan 23, 2017 at 08:50:48AM -0500, Brian Foster wrote:
>>> Since this is limited hardware (I don't have access to such hardware, at
>>> least), it might also be helpful to see if you can isolate the problem
>>> from a full distro install. For example, can you format such a device
>>> external to the rootfs, mount, copy some stuff and reproduce the
>>> corruption after some sequence of unmount/remount/reboot operations?
>>
>> I have plenty of NVMe hardware, although none of them is Intel and I've
>> never seen corruption like that. I also don't run Fedora, though :)
>>
>> Does anyone know if the Fedora installer does a fstrim run or something
>> similar? I remember that most Intel NVMe devices had some pretty severe
>> deallocate (aka discard on NVMe) bugs. A firmware update might be a
>> good start in that case.
>
> The installer doesn't do fstrim or blkdiscard before installation; but
> on my SSDs at least mkfs.xfs and mkfs.btrfs result in trim. So one
> possible test would be to manually mkfs with -k, and then if it's a
> live media installation, install manually from the booted live
> environment with:
>
> rsync -pogAXtlHrDx --exclude /dev/ --exclude /proc/ --exclude /sys/
> --exclude /run/ --exclude /boot/*rescue* --exclude /etc/machine-id
> /run/install/source/ /mnt/sysimage
Right, mkfs does do trim...
Could also pre-mkfs, and tell anaconda to re-use the partition without
re-mkfs - is that possible?
-Eric
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: XFS issues with NVMe SSD
2017-01-23 21:27 ` Eric Sandeen
@ 2017-01-23 21:33 ` Chris Murphy
2017-01-24 2:13 ` Bond Masuda
0 siblings, 1 reply; 9+ messages in thread
From: Chris Murphy @ 2017-01-23 21:33 UTC (permalink / raw)
To: Eric Sandeen
Cc: Chris Murphy, Christoph Hellwig, Brian Foster, Bond Masuda, linux-xfs
On Mon, Jan 23, 2017 at 2:27 PM, Eric Sandeen <sandeen@sandeen.net> wrote:
>
>
> On 1/23/17 3:25 PM, Chris Murphy wrote:
>> On Mon, Jan 23, 2017 at 6:53 AM, Christoph Hellwig <hch@infradead.org> wrote:
>>> On Mon, Jan 23, 2017 at 08:50:48AM -0500, Brian Foster wrote:
>>>> Since this is limited hardware (I don't have access to such hardware, at
>>>> least), it might also be helpful to see if you can isolate the problem
>>>> from a full distro install. For example, can you format such a device
>>>> external to the rootfs, mount, copy some stuff and reproduce the
>>>> corruption after some sequence of unmount/remount/reboot operations?
>>>
>>> I have plenty of NVMe hardware, although none of them is Intel and I've
>>> never seen corruption like that. I also don't run Fedora, though :)
>>>
>>> Does anyone know if the Fedora installer does a fstrim run or something
>>> similar? I remember that most Intel NVMe devices had some pretty severe
>>> deallocate (aka discard on NVMe) bugs. A firmware update might be a
>>> good start in that case.
>>
>> The installer doesn't do fstrim or blkdiscard before installation; but
>> on my SSDs at least mkfs.xfs and mkfs.btrfs result in trim. So one
>> possible test would be to manually mkfs with -k, and then if it's a
>> live media installation, install manually from the booted live
>> environment with:
>>
>> rsync -pogAXtlHrDx --exclude /dev/ --exclude /proc/ --exclude /sys/
>> --exclude /run/ --exclude /boot/*rescue* --exclude /etc/machine-id
>> /run/install/source/ /mnt/sysimage
>
> Right, mkfs does do trim...
>
> Could also pre-mkfs, and tell anaconda to re-use the partition without
> re-mkfs - is that possible?
Anaconda requires reformat of any partition used for rootfs, except
Btrfs where it requires a new subvolume for rootfs.
I've got one of these with Fedora 25 on it, variably using kernels
4.5.5 through 4.10-rc4 and haven't had any corruption issues with
either XFS or Btrfs
Samsung Electronics Co Ltd NVMe SSD Controller SM951/PM951 [144d:a802]
--
Chris Murphy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: XFS issues with NVMe SSD
2017-01-23 21:33 ` Chris Murphy
@ 2017-01-24 2:13 ` Bond Masuda
2017-01-24 8:14 ` Gabriel VLASIU
0 siblings, 1 reply; 9+ messages in thread
From: Bond Masuda @ 2017-01-24 2:13 UTC (permalink / raw)
To: Chris Murphy, Eric Sandeen; +Cc: Christoph Hellwig, Brian Foster, linux-xfs
On 01/23/2017 01:33 PM, Chris Murphy wrote:
>
> Anaconda requires reformat of any partition used for rootfs, except
> Btrfs where it requires a new subvolume for rootfs.
>
> I've got one of these with Fedora 25 on it, variably using kernels
> 4.5.5 through 4.10-rc4 and haven't had any corruption issues with
> either XFS or Btrfs
> Samsung Electronics Co Ltd NVMe SSD Controller SM951/PM951 [144d:a802]
>
There are only 2 people I know of that have run into this issue,
including myself. Maybe the issue is hardware specific? If so, the SSD
model I had this problem with is Intel SSDPEKKW512G7X1. I'm not sure
what the "other" person who reported this problem has, other than it too
was an Intel SSD.
The system I had this issue with got shipped to Japan 2 weeks ago and I
no longer have possession of it. Ended up not putting the OS install on
the Intel NVMe SSD, but on a SATA SSD. I may have another system
similarly configured soon - when I get a chance, I'll try to reproduce
the problem again and try to collect more information.
Bond
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: XFS issues with NVMe SSD
2017-01-24 2:13 ` Bond Masuda
@ 2017-01-24 8:14 ` Gabriel VLASIU
0 siblings, 0 replies; 9+ messages in thread
From: Gabriel VLASIU @ 2017-01-24 8:14 UTC (permalink / raw)
To: Bond Masuda; +Cc: Eric Sandeen, linux-xfs
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
On Mon, 23 Jan 2017, Bond Masuda wrote:
> There are only 2 people I know of that have run into this issue, including
> myself.
Well, no. I have the same problem about a week ago.
Intel 6000p (INTEL SSDPEKKF256G7).
RHEL 7 (actually CentOS 7), kernel 3.10.0-514.
I had the root partition on raid 5 for many, many years. I decided to
switch to nvme ssd for root fs. I copied the content of / partition on
ssd, made the required changes to fstab and grub, put selinux in
permissive mode. Reboot. Everything was fine. Touch /.autorelabel, reboot
again. Everything worked up to relabel point during the boot. Errors.
Instructed-me to run fsck. Reboot with the old root fs. xfs.fsck was not
been able to do anything on the ssd partition. In the end, I formatted the
nvme partition with ext4 and seems to works.
Maybe a selinux problem with xfs again? I don't know.
Sincerely,
Gabriel
- --
// Gabriel VLASIU
//
// OpenGPG-KeyID : 44952F15
// OpenGPG-Fingerprint: 4AC5 7C26 2FE9 02DA 4906 24B2 D32B 7ED7 4495 2F15
// OpenGPG-URL : http://www.vlasiu.net/public.key
-----BEGIN PGP SIGNATURE-----
iQIbBAEBCgAGBQJYhwzJAAoJENMrftdElS8VEkQP90a6bW8keyLc31ctNFJ/wgcC
DsXuw9SbG9zI62I5TvJVBDU5wVstZX/C70edbrNPDOqRJSNRCbDorJ3AuPhZ6wBm
PfjNXu/Dz4j1CRQBt0cbQ4bGP0U1FgTlMbt5QqNcAwSlXOZ44QIe7uGMbKoiW84S
pfqr1x+FolMPRUMWrNAI0+Yml0gOMmKg7fNpdEkRnB4NfAveD8zaKh2wrDCkglDn
EaoixjQpkj6mVLmCzkMRs6afb90ycEcZCN0+BB/rC2TOQ+YKAWgqppDhf9enB4CF
7rFaUNrjZ4M22S8sA60D/SvnF76XS9hvABlmn81p378qQiiVsG+uW+a7vkYQjz4I
WL0HLWhJdy5HPCgQ4eo36TpTzlMOsMkHaV6aJLH+biRFQo/mxIBgqvr4Fd3oEQMT
+28DqTbAWCZDIG7o8d+tKuU72Cse3rUa2QbhSN5YwzVi4RZOSP56NJvVmXBPq5+6
vplvOPSF7zDIon7ik7J4IHjfg/5dy6JKevx3QekO7vW/0Aeh7GLI3qfzzxuAW8Ww
MjELXSrcFnhFRG/2YxZavlZCtXjYxr6AuaxQT22OWfz9fvulPZWrLPA2G/pDpNzW
njYg5MR2dGaHtq5kg23t2KjpCI/2FxCD/nd3/loeZLXzVrZKeWSRPO8ckxD083+K
cmSHlv8gdk/e6fCr+54=
=dzNp
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2017-01-24 8:23 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-01-22 1:20 XFS issues with NVMe SSD Bond Masuda
2017-01-23 13:41 ` Eric Sandeen
2017-01-23 13:50 ` Brian Foster
2017-01-23 13:53 ` Christoph Hellwig
2017-01-23 21:25 ` Chris Murphy
2017-01-23 21:27 ` Eric Sandeen
2017-01-23 21:33 ` Chris Murphy
2017-01-24 2:13 ` Bond Masuda
2017-01-24 8:14 ` Gabriel VLASIU
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.