From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D780C433EF for ; Wed, 27 Oct 2021 13:41:19 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 67DC860C51 for ; Wed, 27 Oct 2021 13:41:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 67DC860C51 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.ozlabs.org Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HfVG50hBzz2yRf for ; Thu, 28 Oct 2021 00:41:17 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ILKqtnlR; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2a00:1450:4864:20::435; helo=mail-wr1-x435.google.com; envelope-from=t.i.ivanov@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ILKqtnlR; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HfVFx1F7bz2y7K for ; Thu, 28 Oct 2021 00:41:07 +1100 (AEDT) Received: by mail-wr1-x435.google.com with SMTP id d3so4233887wrh.8 for ; Wed, 27 Oct 2021 06:41:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=cpB8/v2ixQKySz87e9xrf81v5+WjxH7CHCvUY5EMWLE=; b=ILKqtnlRNY+paHHF3ljsOGjyPO3FlYYGqzatBTlB+i/dR8YP2i3/y6REkNsaGeAgQJ vGh6F9H+Aw/sWxoO3aHwsvsNSa4nrWYILEvhiwmJJ7iEOz4G9Y+drOd9hvz/T1MHiXD7 r3BL1HIKYT8MUBJRhCFqbLsmy2q22wTU9PRYJEPyn9OGeB8Ifq2ghTVXFusYMIh3ZdDG Vczcq66M3X2f3wyag8kCRhMA3TgSg73UgzRX7REmICiYlKlm1NYeAi61TPGxkduVMXNU 6Q7GJSkrrplgMXh/a/gYZ6kJPVAizADE39V7WyXepU14yyEQc4CcveVsW1VkPPCaV/0o +wZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=cpB8/v2ixQKySz87e9xrf81v5+WjxH7CHCvUY5EMWLE=; b=axzqvCAaO58VPUWIb1u9aFFpGjzUFp8KMl8uiT8QCMOZHfzyi3j+SHdB9aEDFINnOk kFigup16eOnBE5zXMERZCA78Ck10H/PCpbHgvxkHhRHfJobeUH+RnIzeJ3WY14XGjLZy PQ+NI1jdALfYcpCYIMZJSYslO0g13vCBQQJ+pW+LvrWSNx8z7imKQ0IeiRi6TtQmwCis jsypaw+T65zR5SKWRi0UklAMwVOihHdwr0/CzqBQxfZlu8OmaSQt6ZKNPBRo4nNFskzF R4GAtMARGTcGvERF6UHX08QRLB3xYxuJ59rOJwMhNQ/WqHWzD2DwSOoCAa8aHom7XZf8 w8Pw== X-Gm-Message-State: AOAM531vy+hFkAXyQf99dbtYBq5wNZonAqgiVvS2iojaoFPujkqL+SAr KjRCkL02Bo/CXowuwx0/v0A0gnuljXftj48TymS8Yj+LJyQ= X-Google-Smtp-Source: ABdhPJy6n76Qp+Kdpu4mZaPf31y8Q/Hf3sq2LxyUE6FZs4+Q3dFlpVaIZ7PXj5WIzGUUrDNUkBz2mNwZtOw+Xs2EY/E= X-Received: by 2002:adf:f202:: with SMTP id p2mr2880285wro.93.1635342054777; Wed, 27 Oct 2021 06:40:54 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Todor Ivanov Date: Wed, 27 Oct 2021 16:40:28 +0300 Message-ID: Subject: Re: Question about mkfs.erofs and reproducible builds To: Gao Xiang Content-Type: multipart/alternative; boundary="00000000000042e44b05cf55be4e" X-BeenThere: linux-erofs@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development of Linux EROFS file system List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-erofs@lists.ozlabs.org Errors-To: linux-erofs-bounces+linux-erofs=archiver.kernel.org@lists.ozlabs.org Sender: "Linux-erofs" --00000000000042e44b05cf55be4e Content-Type: text/plain; charset="UTF-8" Hi, Gao, Indeed c_version:[1.3-g9fe440d0], seems to fix the problem :) Thank you very much for the prompt and accurate feedback! Do you think we can use the dev branch for our images or is it better until it is merged into master? What do you think? Kind regards, Todor On Wed, Oct 27, 2021 at 2:51 PM Gao Xiang wrote: > On Wed, Oct 27, 2021 at 02:41:54PM +0300, Todor Ivanov wrote: > > Hi, Gao, > > > > This is how I installed mkfs.erofs on debian10: > > > > apt-get install pkg-config liblz4-dev gawk > > wget > > > http://ftp.debian.org/debian/pool/main/e/erofs-utils/erofs-utils_1.3.orig.tar.gz > > tar xvzpf erofs-utils_1.3.orig.tar.gz > > cd erofs-utils-1.3/ > > ./autogen.sh > > ./configure > > make > > make install > > > > Can you tell me where do I clone it from and if build instructions are > > different? > > You could get the latest dev branch from: > git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git -b dev > > We fixed some reproducable build issues recently, I think it might > be related to > > https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?id=6324fac820c28c6a946f595fa58a0abba0f48eb4 > > Thanks, > Gao Xiang > > > > > Kind regards, > > Todor > > > > On Wed, Oct 27, 2021 at 2:27 PM Gao Xiang > > wrote: > > > > > Hi Todor, > > > > > > On Wed, Oct 27, 2021 at 02:11:24PM +0300, Todor Ivanov wrote: > > > > Hello, > > > > > > > > We are trying to replace squashfs with erofs and face an > issue > > > with > > > > reproducing the build from one and the same source folder. The source > > > > folder is "/etc" actually taken from an offline ubuntu 20.04 image > and > > > > mounted as read-only. > > > > I managed to narrow down the scope and it turns out that the > > > issue > > > > is when you have a file starting with "." (dot) in this folder. I.e.: > > > > > > > > etc/.anyfilename > > > > > > > > If I remove this file the erofs image of "etc" is reproducible (-T > and -U > > > > are used as well) > > > > > > > > The issue is somehow related to the other 76 subfolders of etc and > this > > > > file starting with dot. For example if I create such .anyfilename in > usr > > > or > > > > var, there is no issue. Also if I create this file under > > > > etc/xdg/.anyfilename, this is fine as well. > > > > I also tried with etc from debian10 and the result is the same. > Removing > > > > any file that starts with dot directly under etc, makes the erofs > build > > > > reproducible. > > > > Do you have any advice on this? > > > > > > In principle filenames starting with '.' won't impact anything about > > > reproducible builds... > > > > > > Let me investigate it now... But may I ask which erofs-utils version > > > is used? Does it still happen on the latest dev branch? > > > > > > Thanks, > > > Gao Xiang > > > > > > > > > > > Regards, > > > > Todor > > > > --00000000000042e44b05cf55be4e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

=C2=A0=C2=A0 Hi, Gao,
=

=C2=A0=C2=A0 Indeed c_version:[1.3-g9fe440d0], seems to= fix the problem :) Thank you very much for the prompt and accurate feedbac= k! Do you think we can use the dev branch for our images or is it better un= til it is merged into master? What do you think?

Kind regards,
Todor


On Wed, Oct 27, 2= 021 at 2:51 PM Gao Xiang <hsiangkao@linux.alibaba.com> wrote:
On Wed, Oct 27, 2021 at 02:41:54PM +0300, Todor= Ivanov wrote:
>=C2=A0 =C2=A0 =C2=A0Hi, Gao,
>
>=C2=A0 =C2=A0 =C2=A0This is how I installed mkfs.erofs on debian10:
>
> apt-get install pkg-config liblz4-dev gawk
> wget
> http://ftp.debi= an.org/debian/pool/main/e/erofs-utils/erofs-utils_1.3.orig.tar.gz
> tar xvzpf erofs-utils_1.3.orig.tar.gz
> cd erofs-utils-1.3/
> ./autogen.sh
> ./configure
> make
> make install
>
> Can you tell me where do I clone it from and if build instructions are=
> different?

You could get the latest dev branch from:
git://git.kernel.org/pub/scm/lin= ux/kernel/git/xiang/erofs-utils.git -b dev

We fixed some reproducable build issues recently, I think it might
be related to
https://git.kernel.org/pub/scm/linux/kernel/git/xian= g/erofs-utils.git/commit/?id=3D6324fac820c28c6a946f595fa58a0abba0f48eb4=

Thanks,
Gao Xiang

>
> Kind regards,
> Todor
>
> On Wed, Oct 27, 2021 at 2:27 PM Gao Xiang <hsiangkao@linux.alibaba.com>= ;
> wrote:
>
> > Hi Todor,
> >
> > On Wed, Oct 27, 2021 at 02:11:24PM +0300, Todor Ivanov wrote:
> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Hello,
> > >
> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0We are trying to replace sq= uashfs with erofs and face an issue
> > with
> > > reproducing the build from one and the same source folder. T= he source
> > > folder is "/etc" actually taken from an offline ub= untu 20.04 image and
> > > mounted as read-only.
> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0I managed to narrow down th= e scope and it turns out that the
> > issue
> > > is when you have a file starting with "." (dot) in= this folder. I.e.:
> > >
> > > etc/.anyfilename
> > >
> > > If I remove this file the erofs image of "etc" is = reproducible (-T and -U
> > > are used as well)
> > >
> > > The issue is somehow related to the other 76 subfolders of e= tc and this
> > > file starting with dot. For example if I create such .anyfil= ename in usr
> > or
> > > var, there is no issue. Also if I create this file under
> > > etc/xdg/.anyfilename, this is fine as well.
> > > I also tried with etc from debian10 and the result is the sa= me. Removing
> > > any file that starts with dot directly under etc, makes the = erofs build
> > > reproducible.
> > > Do you have any advice on this?
> >
> > In principle filenames starting with '.' won't impact= anything about
> > reproducible builds...
> >
> > Let me investigate it now... But may I ask which erofs-utils vers= ion
> > is used? Does it still happen on the latest dev branch?
> >
> > Thanks,
> > Gao Xiang
> >
> > >
> > > Regards,
> > > Todor
> >
--00000000000042e44b05cf55be4e--