From mboxrd@z Thu Jan 1 00:00:00 1970 From: Amir Goldstein Subject: Re: Regression in handling power cuts since 3a1e819b4e80 ("ovl: store file handle of lower inode on copy up") Date: Mon, 22 Oct 2018 11:57:20 +0300 Message-ID: References: <3000620.g91H2S8sUk@blindfold> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <3000620.g91H2S8sUk@blindfold> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-mtd" Errors-To: linux-mtd-bounces+gldm-linux-mtd-36=gmane.org@lists.infradead.org To: Richard Weinberger Cc: Artem Bityutskiy , =?UTF-8?B?UmFmYcWCIE1pxYJlY2tp?= , Adrian Hunter , overlayfs , Miklos Szeredi , linux-mtd@lists.infradead.org, russell@personaltelco.net, linux-fsdevel , openwrt-devel@lists.openwrt.org List-Id: linux-unionfs@vger.kernel.org T24gTW9uLCBPY3QgMjIsIDIwMTggYXQgMTE6MjYgQU0gUmljaGFyZCBXZWluYmVyZ2VyIDxyaWNo YXJkQG5vZC5hdD4gd3JvdGU6Cj4KPiBBbSBNb250YWcsIDIyLiBPa3RvYmVyIDIwMTgsIDA5OjE0 OjA4IENFU1Qgc2NocmllYiBSYWZhxYIgTWnFgmVja2k6Cj4gPiBPbiBGcmksIDE5IE9jdCAyMDE4 IGF0IDE0OjMxLCBSYWZhxYIgTWnFgmVja2kgPHphamVjNUBnbWFpbC5jb20+IHdyb3RlOgo+ID4g PiBTaW5jZSBPcGVuV3J0IHN3aXRjaCBmcm9tIGtlcm5lbCA0LjkgdG8gNC4xNCB1c2VycyBzdGFy dGVkIHJhbmRvbWx5Cj4gPiA+IHJlcG9ydGluZyBmaWxlIHN5c3RlbSBjb3JydXB0aW9ucy4gT3Bl bldydCB1c2VzIG92ZXJsYXkoZnMpIHdpdGgKPiA+ID4gc3F1YXNoZnMgYXMgbG93ZXJkaXIgYW5k IHViaWZzIGFzIHVwcGVyZGlyLiBSdXNzZWxsIG1hbmFnZWQgdG8gaXNvbGF0ZQo+ID4gPiAmIGRl c2NyaWJlIHRlc3QgY2FzZSBmb3IgcmVwcm9kdWNpbmcgY29ycnVwdGlvbiB3aGVuIGRvaW5nIGEg cG93ZXIgY3V0Cj4gPiA+IGFmdGVyIGZpcnN0IGJvb3QuCj4gPiA+Cj4gPiA+ICguLi4pCj4gPiA+ Cj4gPiA+IENhbiBJIGFzayB5b3UgdG8gY2hlY2sgaWYgdGhlcmUgaXMgc29tZXRoaW5nIHBvc3Np Ymx5IHdyb25nIHdpdGggdGhlCj4gPiA+IGFib3ZlIG92bCBjb21taXQ/IE9yIGRvZXMgaXQgZXhw b3NlIHNvbWUgcHJvYmxlbSB3aXRoIHRoZSB1Ymlmcz8gT3IKPiA+ID4gbWF5YmUgdGhlIHdob2xl IFVCST8KPiA+ID4KPiA+ID4gRldJVyB0ZXN0aW5nIGFib3ZlIGNvbW1pdCAoYW5kIG9uZSBiZWZv cmUgaXQpIGFsd2F5cyByZXN1bHRzIGluIHNpbmdsZQo+ID4gPiBlcnJvciBpbiB0aGUga2VybmVs IGxvZzoKPiA+ID4gWyAgIDE0LjI1MDE4NF0gVUJJRlMgZXJyb3IgKHViaTA6MSBwaWQgNjM3KTog dWJpZnNfYWRkX29ycGhhbjogb3JwaGFuZWQgdHdpY2UKPiA+ID4KPiA+ID4gVGhhdCBVQklGUyBl cnJvciBkb2Vzbid0IG9jY3VyIHdpdGggNC4xMi4xNC4gVW5mb3J0dW5hdGVseSBpdCdzCj4gPiA+ IGltcG9zc2libGUgdG8gY2xlYW5seSByZXZlcnQgM2ExZTgxOWI0ZTgwIGZyb20gdGhlIHRvcCBv ZiA0LjEyLjE0Lgo+ID4KPiA+IExldCBtZSBwcm92aWRlIGEgc3VtbWFyeSBvZiBhbGwgcmVsZXZh bnQgY29tbWl0cyAmIHRlc3RzOgo+ID4KPiA+IEJ5ICJDb3JydXB0aW9uIiBJIG1lYW4gZmlsZSBz eXN0ZW0gY29ycnVwdGlvbiBhZnRlciBwb3dlciBjdXQKPgo+IFdlbGwsIGlzIHRoZSBmaWxlc3lz dGVtIG5vdCBjb25zaXN0ZW50IGFueW1vcmU/Cj4gRnJvbSB3aGF0IFJ1c3NlbCBleHBsYWluZWQg dG8gbWUsIEkgdGhvdWdodCB0aGUgbWFpbiBwcm9ibGVtIGlzIHRoYXQgbm8gd3JpdGUgYmFjayBo YXBwZW5zLgo+IElPVyB0aGUgaW5vZGUgaXMgcHJlc2VudCwgaGFzIGNvcnJlY3QgbGVuZ3RoLCBi dXQgbm8gY29udGVudCBpcyB0aGVyZSAoYWxsIHplcm9zKS4KPgo+IEp1c3QgbGlrZSB0aGUgdHlw aWNhbCBjYXNlIHdoZXJlIHVzZXJzcGFjZSBkb2VzIG5vdCBmc3luYy4KPiBCdXQgaW4geW91ciBj YXNlIHNvb25lciBvciBsYXRlciB3cml0ZSBiYWNrIHNob3VsZCBoYXZlIGhhcHBlbmVkIGJlY2F1 c2UgdGhlIHdyaXRlYmFjayB0aW1lcgo+IGZpcmVzIGF0IHNvbWUgcG9pbnQuCj4KCkZvciB0aGUg cmVjb3JkcyBvdmVybGF5ZnMgZG9lczoKLSBvcGVuKE9fVE1QRklMRSkKLSBzZXR4YXR0cigpIFt3 aXRoIDNhMWU4MTliNGU4MF0KLSB3cml0ZSB0byB0bXBmaWxlCi0gZnN5bmMgdG1wZmlsZQotIGxp bmsgdG1wZmlsZQoKSSBzdWdnZXN0IHRoYXQgeW91IHRyeSB0aGUgc2FtZSBmcm9tIHVzZXIgc3Bh Y2Ugb24gdWJpZnMuCgpUaGFua3MsCkFtaXIuCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX18KTGludXggTVREIGRpc2N1c3Npb24gbWFpbGluZyBs aXN0Cmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtbXRk Lwo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yb1-f196.google.com ([209.85.219.196]:44073 "EHLO mail-yb1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727218AbeJVRPJ (ORCPT ); Mon, 22 Oct 2018 13:15:09 -0400 MIME-Version: 1.0 References: <3000620.g91H2S8sUk@blindfold> In-Reply-To: <3000620.g91H2S8sUk@blindfold> From: Amir Goldstein Date: Mon, 22 Oct 2018 11:57:20 +0300 Message-ID: Subject: Re: Regression in handling power cuts since 3a1e819b4e80 ("ovl: store file handle of lower inode on copy up") To: Richard Weinberger Cc: =?UTF-8?B?UmFmYcWCIE1pxYJlY2tp?= , Miklos Szeredi , overlayfs , linux-fsdevel , Artem Bityutskiy , Adrian Hunter , linux-mtd@lists.infradead.org, russell@personaltelco.net, openwrt-devel@lists.openwrt.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Mon, Oct 22, 2018 at 11:26 AM Richard Weinberger wrote: > > Am Montag, 22. Oktober 2018, 09:14:08 CEST schrieb Rafa=C5=82 Mi=C5=82eck= i: > > On Fri, 19 Oct 2018 at 14:31, Rafa=C5=82 Mi=C5=82ecki wrote: > > > Since OpenWrt switch from kernel 4.9 to 4.14 users started randomly > > > reporting file system corruptions. OpenWrt uses overlay(fs) with > > > squashfs as lowerdir and ubifs as upperdir. Russell managed to isolat= e > > > & describe test case for reproducing corruption when doing a power cu= t > > > after first boot. > > > > > > (...) > > > > > > Can I ask you to check if there is something possibly wrong with the > > > above ovl commit? Or does it expose some problem with the ubifs? Or > > > maybe the whole UBI? > > > > > > FWIW testing above commit (and one before it) always results in singl= e > > > error in the kernel log: > > > [ 14.250184] UBIFS error (ubi0:1 pid 637): ubifs_add_orphan: orphan= ed twice > > > > > > That UBIFS error doesn't occur with 4.12.14. Unfortunately it's > > > impossible to cleanly revert 3a1e819b4e80 from the top of 4.12.14. > > > > Let me provide a summary of all relevant commits & tests: > > > > By "Corruption" I mean file system corruption after power cut > > Well, is the filesystem not consistent anymore? > From what Russel explained to me, I thought the main problem is that no w= rite back happens. > IOW the inode is present, has correct length, but no content is there (al= l zeros). > > Just like the typical case where userspace does not fsync. > But in your case sooner or later write back should have happened because = the writeback timer > fires at some point. > For the records overlayfs does: - open(O_TMPFILE) - setxattr() [with 3a1e819b4e80] - write to tmpfile - fsync tmpfile - link tmpfile I suggest that you try the same from user space on ubifs. Thanks, Amir.