All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2] lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number
@ 2023-07-07  8:13 Mattijs Korpershoek
  2023-07-07 10:54 ` qianfan
  2023-07-25 21:12 ` Tom Rini
  0 siblings, 2 replies; 4+ messages in thread
From: Mattijs Korpershoek @ 2023-07-07  8:13 UTC (permalink / raw)
  To: qianfan Zhao, Sean Anderson
  Cc: Guillaume La Roque, Gary Bisson, Troy Kisky, u-boot, Mattijs Korpershoek

Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")
fixed cache alignment for systems with a D-CACHE.

However it introduced some performance regressions [1] on system
flashing huge images, such as Android.

On AM62x SK EVM, we also observe such performance penalty:
Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.954s]
Writing 'super'                                    OKAY [ 75.926s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.641s]
Writing 'super'                                    OKAY [ 62.849s]
Finished. Total time: 182.474s

The reason for this is that we use an arbitrary small buffer
(info->blksz * 100) for transferring.

Fix it by using a bigger buffer (info->blksz * FASTBOOT_MAX_BLK_WRITE)
as suggested in the original's patch review [2].

With this patch, performance impact is mitigated:
Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.912s]
Writing 'super'                                    OKAY [ 15.780s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.581s]
Writing 'super'                                    OKAY [ 17.192s]
Finished. Total time: 76.569s

[1] https://lore.kernel.org/r/20221118121323.4009193-1-gary.bisson@boundarydevices.com
[2] https://lore.kernel.org/r/all/43e4c17c-4483-ec8e-f843-9b4c5569bd18@seco.com/

Fixes: 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")
Signed-off-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
---
Changes in v2:
- Use FASTBOOT_MAX_BLK_WRITE instead of blkcnt (Qianfan)
- Link to v1: https://lore.kernel.org/r/20230616-sparse-flash-fix-v1-1-6bafeacc567b@baylibre.com
---
 drivers/fastboot/fb_mmc.c | 2 --
 include/image-sparse.h    | 2 ++
 lib/image-sparse.c        | 3 ++-
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/fastboot/fb_mmc.c b/drivers/fastboot/fb_mmc.c
index 9d25c402028a..060918e49109 100644
--- a/drivers/fastboot/fb_mmc.c
+++ b/drivers/fastboot/fb_mmc.c
@@ -19,8 +19,6 @@
 #include <linux/compat.h>
 #include <android_image.h>
 
-#define FASTBOOT_MAX_BLK_WRITE 16384
-
 #define BOOT_PARTITION_NAME "boot"
 
 struct fb_mmc_sparse {
diff --git a/include/image-sparse.h b/include/image-sparse.h
index 0572dbd0a283..282a0b256498 100644
--- a/include/image-sparse.h
+++ b/include/image-sparse.h
@@ -7,6 +7,8 @@
 #include <part.h>
 #include <sparse_format.h>
 
+#define FASTBOOT_MAX_BLK_WRITE 16384
+
 #define ROUNDUP(x, y)	(((x) + ((y) - 1)) & ~((y) - 1))
 
 struct sparse_storage {
diff --git a/lib/image-sparse.c b/lib/image-sparse.c
index 5ec0f94ab3eb..8f8a67e15804 100644
--- a/lib/image-sparse.c
+++ b/lib/image-sparse.c
@@ -55,7 +55,8 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,
 				       void *data,
 				       char *response)
 {
-	lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100;
+	lbaint_t n = blkcnt, write_blks, blks = 0;
+	lbaint_t aligned_buf_blks = FASTBOOT_MAX_BLK_WRITE;
 	uint32_t *aligned_buf = NULL;
 
 	if (CONFIG_IS_ENABLED(SYS_DCACHE_OFF)) {

---
base-commit: 923de765ee1a5b26310f02cb42dcbad9e2b011c5
change-id: 20230616-sparse-flash-fix-9c2852aa8d16

Best regards,
-- 
Mattijs Korpershoek <mkorpershoek@baylibre.com>


^ permalink raw reply related	[flat|nested] 4+ messages in thread
* Re: [PATCH v2] lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number
  2023-07-07  8:13 [PATCH v2] lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number Mattijs Korpershoek
@ 2023-07-07 10:54 ` qianfan
  2023-07-07 14:01   ` Mattijs Korpershoek
  2023-07-25 21:12 ` Tom Rini
  1 sibling, 1 reply; 4+ messages in thread
From: qianfan @ 2023-07-07 10:54 UTC (permalink / raw)
  To: Mattijs Korpershoek, Sean Anderson
  Cc: Guillaume La Roque, Gary Bisson, Troy Kisky, u-boot



在 2023/7/7 16:13, Mattijs Korpershoek 写道:
> Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")
> fixed cache alignment for systems with a D-CACHE.
>
> However it introduced some performance regressions [1] on system
> flashing huge images, such as Android.
>
> On AM62x SK EVM, we also observe such performance penalty:
> Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.954s]
> Writing 'super'                                    OKAY [ 75.926s]
> Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.641s]
> Writing 'super'                                    OKAY [ 62.849s]
> Finished. Total time: 182.474s
>
> The reason for this is that we use an arbitrary small buffer
> (info->blksz * 100) for transferring.
>
> Fix it by using a bigger buffer (info->blksz * FASTBOOT_MAX_BLK_WRITE)
> as suggested in the original's patch review [2].
>
> With this patch, performance impact is mitigated:
> Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.912s]
> Writing 'super'                                    OKAY [ 15.780s]
> Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.581s]
> Writing 'super'                                    OKAY [ 17.192s]
> Finished. Total time: 76.569s
>
> [1] https://lore.kernel.org/r/20221118121323.4009193-1-gary.bisson@boundarydevices.com
> [2] https://lore.kernel.org/r/all/43e4c17c-4483-ec8e-f843-9b4c5569bd18@seco.com/
>
> Fixes: 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")
> Signed-off-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
> ---
> Changes in v2:
> - Use FASTBOOT_MAX_BLK_WRITE instead of blkcnt (Qianfan)
> - Link to v1: https://lore.kernel.org/r/20230616-sparse-flash-fix-v1-1-6bafeacc567b@baylibre.com
> ---
>   drivers/fastboot/fb_mmc.c | 2 --
>   include/image-sparse.h    | 2 ++
>   lib/image-sparse.c        | 3 ++-
>   3 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/fastboot/fb_mmc.c b/drivers/fastboot/fb_mmc.c
> index 9d25c402028a..060918e49109 100644
> --- a/drivers/fastboot/fb_mmc.c
> +++ b/drivers/fastboot/fb_mmc.c
> @@ -19,8 +19,6 @@
>   #include <linux/compat.h>
>   #include <android_image.h>
>   
> -#define FASTBOOT_MAX_BLK_WRITE 16384
> -
>   #define BOOT_PARTITION_NAME "boot"
>   
>   struct fb_mmc_sparse {
> diff --git a/include/image-sparse.h b/include/image-sparse.h
> index 0572dbd0a283..282a0b256498 100644
> --- a/include/image-sparse.h
> +++ b/include/image-sparse.h
> @@ -7,6 +7,8 @@
>   #include <part.h>
>   #include <sparse_format.h>
>   
> +#define FASTBOOT_MAX_BLK_WRITE 16384
Hi:

Just a personal suggestion, define sometings like FASTBOOT_MAX_BLK_WRITE in
image-sparse.c is very strange.

Or maybe define a marco such as SPARSE_MAX_BLK_WRITE and set a default
value to 16384, and leave some comments for why we choice this value.

Thanks.
> +
>   #define ROUNDUP(x, y)	(((x) + ((y) - 1)) & ~((y) - 1))
>   
>   struct sparse_storage {
> diff --git a/lib/image-sparse.c b/lib/image-sparse.c
> index 5ec0f94ab3eb..8f8a67e15804 100644
> --- a/lib/image-sparse.c
> +++ b/lib/image-sparse.c
> @@ -55,7 +55,8 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,
>   				       void *data,
>   				       char *response)
>   {
> -	lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100;
> +	lbaint_t n = blkcnt, write_blks, blks = 0;
> +	lbaint_t aligned_buf_blks = FASTBOOT_MAX_BLK_WRITE;
>   	uint32_t *aligned_buf = NULL;
>   
>   	if (CONFIG_IS_ENABLED(SYS_DCACHE_OFF)) {
>
> ---
> base-commit: 923de765ee1a5b26310f02cb42dcbad9e2b011c5
> change-id: 20230616-sparse-flash-fix-9c2852aa8d16
>
> Best regards,


^ permalink raw reply	[flat|nested] 4+ messages in thread
* Re: [PATCH v2] lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number
  2023-07-07 10:54 ` qianfan
@ 2023-07-07 14:01   ` Mattijs Korpershoek
  0 siblings, 0 replies; 4+ messages in thread
From: Mattijs Korpershoek @ 2023-07-07 14:01 UTC (permalink / raw)
  To: qianfan, Sean Anderson
  Cc: Guillaume La Roque, Gary Bisson, Troy Kisky, u-boot

Hi Qianfan,

Thank you for your review.

On ven., juil. 07, 2023 at 18:54, qianfan <qianfanguijin@163.com> wrote:

> 在 2023/7/7 16:13, Mattijs Korpershoek 写道:
>> Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")
>> fixed cache alignment for systems with a D-CACHE.
>>
>> However it introduced some performance regressions [1] on system
>> flashing huge images, such as Android.
>>
>> On AM62x SK EVM, we also observe such performance penalty:
>> Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.954s]
>> Writing 'super'                                    OKAY [ 75.926s]
>> Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.641s]
>> Writing 'super'                                    OKAY [ 62.849s]
>> Finished. Total time: 182.474s
>>
>> The reason for this is that we use an arbitrary small buffer
>> (info->blksz * 100) for transferring.
>>
>> Fix it by using a bigger buffer (info->blksz * FASTBOOT_MAX_BLK_WRITE)
>> as suggested in the original's patch review [2].
>>
>> With this patch, performance impact is mitigated:
>> Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.912s]
>> Writing 'super'                                    OKAY [ 15.780s]
>> Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.581s]
>> Writing 'super'                                    OKAY [ 17.192s]
>> Finished. Total time: 76.569s
>>
>> [1] https://lore.kernel.org/r/20221118121323.4009193-1-gary.bisson@boundarydevices.com
>> [2] https://lore.kernel.org/r/all/43e4c17c-4483-ec8e-f843-9b4c5569bd18@seco.com/
>>
>> Fixes: 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")
>> Signed-off-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
>> ---
>> Changes in v2:
>> - Use FASTBOOT_MAX_BLK_WRITE instead of blkcnt (Qianfan)
>> - Link to v1: https://lore.kernel.org/r/20230616-sparse-flash-fix-v1-1-6bafeacc567b@baylibre.com
>> ---
>>   drivers/fastboot/fb_mmc.c | 2 --
>>   include/image-sparse.h    | 2 ++
>>   lib/image-sparse.c        | 3 ++-
>>   3 files changed, 4 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/fastboot/fb_mmc.c b/drivers/fastboot/fb_mmc.c
>> index 9d25c402028a..060918e49109 100644
>> --- a/drivers/fastboot/fb_mmc.c
>> +++ b/drivers/fastboot/fb_mmc.c
>> @@ -19,8 +19,6 @@
>>   #include <linux/compat.h>
>>   #include <android_image.h>
>>   
>> -#define FASTBOOT_MAX_BLK_WRITE 16384
>> -
>>   #define BOOT_PARTITION_NAME "boot"
>>   
>>   struct fb_mmc_sparse {
>> diff --git a/include/image-sparse.h b/include/image-sparse.h
>> index 0572dbd0a283..282a0b256498 100644
>> --- a/include/image-sparse.h
>> +++ b/include/image-sparse.h
>> @@ -7,6 +7,8 @@
>>   #include <part.h>
>>   #include <sparse_format.h>
>>   
>> +#define FASTBOOT_MAX_BLK_WRITE 16384
> Hi:
>
> Just a personal suggestion, define sometings like FASTBOOT_MAX_BLK_WRITE in
> image-sparse.c is very strange.
>
> Or maybe define a marco such as SPARSE_MAX_BLK_WRITE and set a default
> value to 16384, and leave some comments for why we choice this value.

I don't agree with having a duplicating between SPARSE_MAX_BLK_WRITE
and FASTBOOT_MAX_BLK_WRITE. code comments can rot as well.

And FASTBOOT_MAX_BLK_WRITE is used for both sparse and unsparsed image,
which is why I chose to not rename it.

>
> Thanks.
>> +
>>   #define ROUNDUP(x, y)	(((x) + ((y) - 1)) & ~((y) - 1))
>>   
>>   struct sparse_storage {
>> diff --git a/lib/image-sparse.c b/lib/image-sparse.c
>> index 5ec0f94ab3eb..8f8a67e15804 100644
>> --- a/lib/image-sparse.c
>> +++ b/lib/image-sparse.c
>> @@ -55,7 +55,8 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,
>>   				       void *data,
>>   				       char *response)
>>   {
>> -	lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100;
>> +	lbaint_t n = blkcnt, write_blks, blks = 0;
>> +	lbaint_t aligned_buf_blks = FASTBOOT_MAX_BLK_WRITE;
>>   	uint32_t *aligned_buf = NULL;
>>   
>>   	if (CONFIG_IS_ENABLED(SYS_DCACHE_OFF)) {
>>
>> ---
>> base-commit: 923de765ee1a5b26310f02cb42dcbad9e2b011c5
>> change-id: 20230616-sparse-flash-fix-9c2852aa8d16
>>
>> Best regards,

^ permalink raw reply	[flat|nested] 4+ messages in thread
* Re: [PATCH v2] lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number
  2023-07-07  8:13 [PATCH v2] lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number Mattijs Korpershoek
  2023-07-07 10:54 ` qianfan
@ 2023-07-25 21:12 ` Tom Rini
  1 sibling, 0 replies; 4+ messages in thread
From: Tom Rini @ 2023-07-25 21:12 UTC (permalink / raw)
  To: Mattijs Korpershoek
  Cc: qianfan Zhao, Sean Anderson, Guillaume La Roque, Gary Bisson,
	Troy Kisky, u-boot

[-- Attachment #1: Type: text/plain, Size: 1675 bytes --]

On Fri, Jul 07, 2023 at 10:13:34AM +0200, Mattijs Korpershoek wrote:

> Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")
> fixed cache alignment for systems with a D-CACHE.
> 
> However it introduced some performance regressions [1] on system
> flashing huge images, such as Android.
> 
> On AM62x SK EVM, we also observe such performance penalty:
> Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.954s]
> Writing 'super'                                    OKAY [ 75.926s]
> Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.641s]
> Writing 'super'                                    OKAY [ 62.849s]
> Finished. Total time: 182.474s
> 
> The reason for this is that we use an arbitrary small buffer
> (info->blksz * 100) for transferring.
> 
> Fix it by using a bigger buffer (info->blksz * FASTBOOT_MAX_BLK_WRITE)
> as suggested in the original's patch review [2].
> 
> With this patch, performance impact is mitigated:
> Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.912s]
> Writing 'super'                                    OKAY [ 15.780s]
> Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.581s]
> Writing 'super'                                    OKAY [ 17.192s]
> Finished. Total time: 76.569s
> 
> [1] https://lore.kernel.org/r/20221118121323.4009193-1-gary.bisson@boundarydevices.com
> [2] https://lore.kernel.org/r/all/43e4c17c-4483-ec8e-f843-9b4c5569bd18@seco.com/
> 
> Fixes: 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")
> Signed-off-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>

Applied to u-boot/master, thanks!

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-07-25 21:13 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-07-07  8:13 [PATCH v2] lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number Mattijs Korpershoek
2023-07-07 10:54 ` qianfan
2023-07-07 14:01   ` Mattijs Korpershoek
2023-07-25 21:12 ` Tom Rini

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.