From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.17.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8A116AD0 for ; Tue, 16 Aug 2022 20:51:18 +0000 (UTC) Received: from [192.168.1.138] ([37.4.248.80]) by mrelayeu.kundenserver.de (mreue109 [212.227.15.183]) with ESMTPSA (Nemesis) id 1MO9r5-1o0Dxc2Fk9-00OTr7; Tue, 16 Aug 2022 22:45:49 +0200 Message-ID: Date: Tue, 16 Aug 2022 22:45:48 +0200 Precedence: bulk X-Mailing-List: regressions@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Subject: Re: [Regression] ext4: changes to mb_optimize_scan cause issues on Raspberry Pi Content-Language: en-US To: Jan Kara Cc: linux-ext4@vger.kernel.org, Ojaswin Mujoo , Harshad Shirwadkar , Theodore Ts'o , Ritesh Harjani , linux-fsdevel@vger.kernel.org, Linux Kernel Mailing List , Geetika.Moolchandani1@ibm.com, regressions@lists.linux.dev, Florian Fainelli References: <0d81a7c2-46b7-6010-62a4-3e6cfc1628d6@i2se.com> <20220728100055.efbvaudwp3ofolpi@quack3> <64b7899f-d84d-93de-f9c5-49538bd080d0@i2se.com> <20220816093421.ok26tcyvf6bm3ngy@quack3> From: Stefan Wahren In-Reply-To: <20220816093421.ok26tcyvf6bm3ngy@quack3> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K1:QATVIxUIzO4XNP67B+8HP4QQNAu7SDUyzpY148ZnYDJfs7tnRBJ P4nHE8TnbMh+WgAS4rs/lgc7XTshocNV1OpZ44YQvy18WcgxRaLBk+kobb43uc74GoOFyUp gR19howWA1OqDL8Aq7YNa5EhexZE8b5NFYrK0PM4hMG49wBDM/sICkgNrOWNn1soXhDp+O8 TqJAbNdm7pr3HJsnooNqA== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:jI94R+XpoKU=:YODqq+lF/DlxYRBssKasq8 lWZmNmQPfTx1wSQHz1nHkBbIQnrJarg2TiYnLSYPIktGkSK5Rse4JoTC4GX5KU6xmDgsyQTtk 78fdG/0LT0Pa1piFV+dWiaCp/Ja8Wji2j6JYTNu4MzdJhxRrT0bTAGAa34Lt17irxaObS4Kvr quC2X7qeYVIf699jVQr0GHaqndBxAspAEsMn8OZsi8JIAwBmhLfm1Qmjp9UWDDf8BbBKjpQDe lrl0/gu403lPKXfSxVDaTCCR7MxLfDqP4i9FovwrexaSumaps/yQiKgRmeSR03O3GtJfbTZP+ F2i74fZNLMSzMd20/yLgQwBTGusKWXWa1ib0I98iE8dwNizezXXa8SYRKsE6OXZYO67pgT0an FAxRUMKB51R03NDhlcoZlPK+vVVjSEYY4HUg0yr4j2OMi/5XOiyTV7sIOTRd2aoicVa9j03iB qpGG57d361DqBNRO++smr1Cp306I+rsbtU4iGfs4mhrghz9mO8Ea+WnEuRAoFOD/8Er+IFe2k x9axGZTgy6nQV2sqIa5ERhuhXKbW4fW0W7obbxm67y7ppRWgw+oCF15A0JhqSW294V8xAfodg Vg2FAd60t2+fIqWMiAE9sJMwIBMF2rLZy6lmdVJDiRAmD7xg1O8Y8C6CvkrgYalZrjcjy+ni4 O/YJWQCImPbsdOiE/CoeCZk4SaB7QnYLcOOjFVZbNohaf4xavsN9+1CF8Mymfvan51ptFfNVp Mnw8Vdr5oXVGK/5G0QY+XWJkDfq+3If5Ir/hP3Ecc1pa09ZPcrGH3CHJG0o= Hi Jan, Am 16.08.22 um 11:34 schrieb Jan Kara: > Hi Stefan! > So this is interesting. We can see the card is 100% busy. The IO submitted > to the card is formed by small requests - 18-38 KB per request - and each > request takes 0.3-0.5s to complete. So the resulting throughput is horrible > - only tens of KB/s. Also we can see there are many IOs queued for the > device in parallel (aqu-sz columnt). This does not look like load I would > expect to be generated by download of a large file from the web. > > You have mentioned in previous emails that with dd(1) you can do couple > MB/s writing to this card which is far more than these tens of KB/s. So the > file download must be doing something which really destroys the IO pattern > (and with mb_optimize_scan=0 ext4 happened to be better dealing with it and > generating better IO pattern). Can you perhaps strace the process doing the > download (or perhaps strace -f the whole rpi-update process) so that we can > see how does the load generated on the filesystem look like? Thanks! i didn't create the strace yet, but i looked at the source of rpi-update. At the end the download phase is a curl call to download a tar archive and pipe it directly to tar. You can find the content list of the tar file here: https://raw.githubusercontent.com/lategoodbye/mb_optimize_scan_regress/main/rpi-firmware-tar-content-list.txt Best regards > > Honza