All of lore.kernel.org
 help / color / mirror / Atom feed
* reiser4[StorageManager(2383)]: lzo1_alloc...
@ 2017-08-20 17:15 Metztli Information Technology
  2017-08-22 18:49 ` Edward Shishkin
  0 siblings, 1 reply; 9+ messages in thread
From: Metztli Information Technology @ 2017-08-20 17:15 UTC (permalink / raw)
  To: reiserfs-devel

Niltze, Ed-

Although I am using your latest Reiser4 patch for Linux kernel 4.12.x on a test 1.2TB (transparent compression) reiser4 root fs, I have seen
dmesg generate the following in lesser version patches and in much smaller partitions, i.e., 20Gb -- with transparent compression, too.

Your input would be greatly appreciated:

[ 3449.944653] reiser4[StorageManager(2383)]: lzo1_alloc (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 3449.944674] reiser4[StorageManager(2383)]: lzo1_alloc (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 3449.944694] reiser4[StorageManager(2383)]: lzo1_alloc (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 3449.944715] reiser4[StorageManager(2383)]: lzo1_alloc (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[snip]
( the above replicate for about two(2) thousand lines )

Please note that the error above is not immediate but is triggered possibly by multiple events run simultaneously, i.e.,
 plugged 1TB USB hard disk with multiple partitions where copy operation to local disk is being run; another copy operation to-from local hard
disk itself on same filesystem; installing additional packages, building local software, etc.

Subsequently system slows down in latency and fan use increases in all instances.

Thank you in advance for your insight.


Best Professional Regards.

^ permalink raw reply	[flat|nested] 9+ messages in thread
* Re: reiser4[StorageManager(2383)]: lzo1_alloc...
  2017-08-20 17:15 reiser4[StorageManager(2383)]: lzo1_alloc Metztli Information Technology
@ 2017-08-22 18:49 ` Edward Shishkin
  2017-08-30  9:29   ` Edward Shishkin
  0 siblings, 1 reply; 9+ messages in thread
From: Edward Shishkin @ 2017-08-22 18:49 UTC (permalink / raw)
  To: Reiserfs development mailing list; +Cc: Mathieu Belanger

[-- Attachment #1: Type: text/plain, Size: 2399 bytes --]

Hello,

Please, try the attached patches.
The first patch improves responsiveness to vm subsystem
(modified version of ->migratepage() from Ivan Shapovalov).
The second patch performs memory allocation in the critical
place with __GFP_NOFAIL flag.
Let us know about results.

Thanks,
Edward.

On 08/20/2017 07:15 PM, Metztli Information Technology wrote:
> Niltze, Ed-
>
> Although I am using your latest Reiser4 patch for Linux kernel 4.12.x on a test 1.2TB (transparent compression) reiser4 root fs, I have seen
> dmesg generate the following in lesser version patches and in much smaller partitions, i.e., 20Gb -- with transparent compression, too.
>
> Your input would be greatly appreciated:
>
> [ 3449.944653] reiser4[StorageManager(2383)]: lzo1_alloc (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 3449.944674] reiser4[StorageManager(2383)]: lzo1_alloc (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 3449.944694] reiser4[StorageManager(2383)]: lzo1_alloc (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 3449.944715] reiser4[StorageManager(2383)]: lzo1_alloc (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [snip]
> ( the above replicate for about two(2) thousand lines )
>
> Please note that the error above is not immediate but is triggered possibly by multiple events run simultaneously, i.e.,
>   plugged 1TB USB hard disk with multiple partitions where copy operation to local disk is being run; another copy operation to-from local hard
> disk itself on same filesystem; installing additional packages, building local software, etc.
>
> Subsequently system slows down in latency and fan use increases in all instances.
>
> Thank you in advance for your insight.
>
>
> Best Professional Regards.
> --
> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
> the body of a message tomajordomo@vger.kernel.org
> More majordomo info athttp://vger.kernel.org/majordomo-info.html


[-- Attachment #2: reiser4-migratepage.patch --]
[-- Type: text/x-patch, Size: 1459 bytes --]

diff --git a/as_ops.c b/as_ops.c
index 393e9d1..df856df 100644
--- a/as_ops.c
+++ b/as_ops.c
@@ -309,9 +309,45 @@ int reiser4_releasepage(struct page *page, gfp_t gfp UNUSED_ARG)
 int reiser4_migratepage(struct address_space *mapping, struct page *newpage,
 			struct page *page, enum migrate_mode mode)
 {
-	/* TODO: implement movable mapping
+	int result;
+	jnode *node;
+
+	assert("intelfx-62", PageLocked(page));
+	assert("intelfx-63", !PageWriteback(page));
+	assert("intelfx-64", reiser4_schedulable());
+	assert("intelfx-65", page->mapping != NULL);
+	assert("intelfx-66", page->mapping->host != NULL);
+
+	if (!PagePrivate(page))
+		/*
+		 * anonymous, not yet captured page
+		 */
+		return migrate_page(mapping, newpage, page, mode);
+	/*
+	 * page has an attached jnode. That jnode should be
+	 * linked with the new page. Otherwise, everyone
+	 * calling jnode_page(), etc will get invalid data
 	 */
-	return -EIO;
+	node = jnode_by_page(page);
+	spin_lock_jnode(node);
+	spin_lock(&(node->load));
+
+	page_clear_jnode(page, node);
+	set_page_private(newpage, 0ul);
+
+	result = migrate_page(mapping, newpage, page, mode);
+	if (unlikely(result))
+		/*
+		 * migration failed - reattach the old page
+		 */
+		jnode_attach_page(node, page);
+	else
+		jnode_attach_page(node, newpage);
+	spin_unlock(&(node->load));
+	spin_unlock_jnode(node);
+
+	assert("intelfx-67", reiser4_schedulable());
+	return result;
 }
 #endif /* CONFIG_MIGRATION */
 



[-- Attachment #3: reiser4-alloc-compression-workspace-with-nofail-flag.patch --]
[-- Type: text/x-patch, Size: 865 bytes --]

diff --git a/plugin/item/ctail.c b/plugin/item/ctail.c
index 3f46c38..42a3134 100644
--- a/plugin/item/ctail.c
+++ b/plugin/item/ctail.c
@@ -1565,6 +1565,7 @@ static int assign_conversion_mode(flush_pos_t * pos, ctail_convert_mode_t *mode)
 	if (!convert_data_attached(pos)) {
 		if (should_attach_convert_idata(pos)) {
 			struct inode *inode;
+			gfp_t old_mask = get_current_context()->gfp_mask;
 
 			assert("edward-264", pos->child != NULL);
 			assert("edward-265", jnode_page(pos->child) != NULL);
@@ -1577,7 +1578,9 @@ static int assign_conversion_mode(flush_pos_t * pos, ctail_convert_mode_t *mode)
 			/*
 			 * attach new convert item info
 			 */
+			get_current_context()->gfp_mask |= __GFP_NOFAIL;
 			ret = attach_convert_idata(pos, inode);
+			get_current_context()->gfp_mask = old_mask;
 			pos->child = NULL;
 			if (ret == -E_REPEAT) {
 				/*



^ permalink raw reply related	[flat|nested] 9+ messages in thread
* Re: reiser4[StorageManager(2383)]: lzo1_alloc...
  2017-08-22 18:49 ` Edward Shishkin
@ 2017-08-30  9:29   ` Edward Shishkin
  0 siblings, 0 replies; 9+ messages in thread
From: Edward Shishkin @ 2017-08-30  9:29 UTC (permalink / raw)
  To: Reiserfs development mailing list; +Cc: Mathieu Belanger

On 08/22/2017 08:49 PM, Edward Shishkin wrote:
> Hello,
>
> Please, try the attached patches.
> The first patch improves responsiveness to vm subsystem
> (modified version of ->migratepage()

Hi all,

I found that the first patch leads to system lockups after ~1 hour of 
stress-testing.
This is a side effect of appearing of non-parsed jnodes in writeback 
queues for some
reasons. So please, drop it for now..

Thanks,
Edward.

> The second patch performs memory allocation in the critical
> place with __GFP_NOFAIL flag.
> Let us know about results.
>
> Thanks,
> Edward.
>
> On 08/20/2017 07:15 PM, Metztli Information Technology wrote:
>> Niltze, Ed-
>>
>> Although I am using your latest Reiser4 patch for Linux kernel 4.12.x 
>> on a test 1.2TB (transparent compression) reiser4 root fs, I have seen
>> dmesg generate the following in lesser version patches and in much 
>> smaller partitions, i.e., 20Gb -- with transparent compression, too.
>>
>> Your input would be greatly appreciated:
>>
>> [ 3449.944653] reiser4[StorageManager(2383)]: lzo1_alloc 
>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) 
>> failed
>>
>> [ 3449.944674] reiser4[StorageManager(2383)]: lzo1_alloc 
>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) 
>> failed
>>
>> [ 3449.944694] reiser4[StorageManager(2383)]: lzo1_alloc 
>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) 
>> failed
>>
>> [ 3449.944715] reiser4[StorageManager(2383)]: lzo1_alloc 
>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) 
>> failed
>>
>> [snip]
>> ( the above replicate for about two(2) thousand lines )
>>
>> Please note that the error above is not immediate but is triggered 
>> possibly by multiple events run simultaneously, i.e.,
>>   plugged 1TB USB hard disk with multiple partitions where copy 
>> operation to local disk is being run; another copy operation to-from 
>> local hard
>> disk itself on same filesystem; installing additional packages, 
>> building local software, etc.
>>
>> Subsequently system slows down in latency and fan use increases in 
>> all instances.
>>
>> Thank you in advance for your insight.
>>
>>
>> Best Professional Regards.
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe 
>> reiserfs-devel" in
>> the body of a message tomajordomo@vger.kernel.org
>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
>


^ permalink raw reply	[flat|nested] 9+ messages in thread
* Re: reiser4[StorageManager(2383)]: lzo1_alloc
  2017-08-26  7:15 Metztli Information Technology
@ 2017-08-28 13:15 ` Edward Shishkin
  0 siblings, 0 replies; 9+ messages in thread
From: Edward Shishkin @ 2017-08-28 13:15 UTC (permalink / raw)
  To: Metztli Information Technology, b747xx, reiserfs-devel

[-- Attachment #1: Type: text/plain, Size: 12935 bytes --]

Please, try the attached patch,
it allows to proceed flushing even workspace allocation was failed.

Thanks,
Edward.

On 08/26/2017 09:15 AM, Metztli Information Technology wrote:
> On Fri, Aug 25, 2017 at 9:22 AM, Edward Shishkin <edward.shishkin@gmail.com> wrote:
>> As a possible fixup we can leave data uncompressed.
>> I think it is rather rare event (the flood of warnings is because
>> of inability to allocate workspace for the same chunk of data.
> If left unfixed in subsequent reiser4 kernel patches, (unseen) stream of 'flood of warnings' has the potential to crash the kernel.
>
>> Edward.
>>
>> On 08/25/2017 06:49 AM, Mathieu Bélanger wrote:
>>> I did have that bug specifically with Chromium too, I did initial try to
>>> test by disabling ALSR (I was suspecting the Ryzen silicon bug that finally
>>> can be RMA).
>>> But disabling ALSR caused more problem so ...
>>>
>>> I did "fix" the issue by recompiling Chromium with -O2 instead of -O3.
>>>
>>> And I do have similar problem with Firefox 57 compiled with -O3 (Again -O2
>>> fix it and Firefox 55 with -O3 is not affected)
> Thing is pre-built Firefox from Debian repositories did not produce warnings -- as far as random captured 2,000 chunks of dmesg output log showed -- only Chromium did.
>>> On Thu, Aug 24, 2017 at 11:42 PM, Metztli Information Technology
>>> <jose@huitzilopochtli.metztli-it.com
>>> <mailto:jose@huitzilopochtli.metztli-it.com>> wrote:
>>>
>>>
>>>
>>>      On Thu, Aug 24, 2017 at 6:01 AM, Edward Shishkin
>>>      <edward.shishkin@gmail.com <mailto:edward.shishkin@gmail.com>> wrote:
>>>      > So, memory allocation policy got changed in the upstream,
>>>      > and we need to perform pre-allocation to not fail at flush time.
>>>      > I am sorry, but right now I don't have a time for this..
>>>      No worries, sir. I simply fulfilled your request for feedback.
>>>
>>>      Background for this test was to evaluate 2TB maximum slice allowed
>>>      by Google Compute Engine in cloud instances, specifically Debian
>>>      with transparent compression reiser4.
>>>
>>>      Currently, (default) transparent compression reiser4 formatting is
>>>      not available[1] in my custom Debian-Installer (d-i) but planned
>>>      to make available in a future implementation.
>>>
>>>      >
>>>      >
>>>      > On 08/24/2017 06:59 AM, Metztli Information Technology wrote:
>>>      >>
>>>      >> Much appreciated, Ed-
>>>      >>
>>>      >> Noticed improvement, notwithstanding...
>>>      >>
>>>      >> Context:
>>>      >>
>>>      >> uname -a
>>>      >> Linux huitzilopochtli 4.12.0-1+reiser4.0.1-amd64 #1 SMP Debian
>>>      >> 4.12.6-3+reiser4.0.1 (2017-08-14) x86_64 GNU/Linux
>>>      >>
>>>      >> (I have reinstalled same kernel two times after patching so the
>>>      above
>>>      >> string retained the older original kernel installation date.
>>>      >> but
>>>      >> uname -v
>>>      >> #1 SMP Debian 4.12.6-3...[means upgrade '-3' reflects fact that
>>>      I rebuilt
>>>      >> fs with your latest two(2) patches to address the issue.])
>>>      >>
>>>      >>
>>>      >>   ls -ltc /lib/modules/4.12.0-1*64/kernel
>>>      >> total 18
>>>      >> drwxr-xr-x 14 root root 16 Aug 23 02:14 sound/
>>>      >> drwxr-xr-x  5 root root 24 Aug 23 02:13 lib/
>>>      >> drwxr-xr-x  2 root root  4 Aug 23 02:13 mm/
>>>      >> drwxr-xr-x 60 root root 62 Aug 23 02:13 fs/
>>>      >> drwxr-xr-x  3 root root 73 Aug 23 02:13 crypto/
>>>      >> drwxr-xr-x  2 root root  4 Aug 23 02:13 block/alloc workspace
>>>
>>>      >> drwxr-xr-x 51 root root 51 Aug 18 17:02 net/
>>>      >> drwxr-xr-x  3 root root  3 Aug 18 17:02 virt/
>>>      >> drwxr-xr-x 70 root root 70 Aug 18 17:02 drivers/
>>>      >> drwxr-xr-x  3 root root  3 Aug 18 17:02 arch/
>>>      >>
>>>      >> After applying (fs/) patches and rebooting, I began to apply
>>>      load to the
>>>      >> machine where with previous kernel I had already built
>>>      GCC-5-branch and
>>>      >> Apache OpenOffice. Memory is limited to 16Gb RAM; copy
>>>      operations were
>>>      >> started from 1TB USB disk to local reiser4 transparent
>>>      compression, a  16Gb
>>>      >> data copy to same local filesystem, began a 2Gb svn download,
>>>      etc.; I had
>>>      >> Firefox open with several tabs open and launched chromium
>>>      browser -- which
>>>      >> began producing relevant feedback. I have set a two(2) thousand
>>>      lines limit
>>>      >> output in my shell so that is the reason *all* of the below
>>>      WARNINGS repeat
>>>      >> that number of times.
>>>      >>
>>>      >> Chromium browser launched but app did not open in the GUI and
>>>      got stuck
>>>      >> with un-killable processes (memory starvation?):
>>>      >>
>>>      >> % kill -9 $(pgrep chromium)
>>>      >> % pgrep chromium
>>>      >> 6320
>>>      >> 6743
>>>      >> 6814
>>>      >> 7050
>>>      >>
>>>      >> Subsequently dmesg showed (<process number>) none necessarily
>>>      in the order
>>>      >> below as dmesg alternated in producing sequence:
>>>      >>
>>>      >> [ 6175.145234] reiser4[chromium(7050)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 6175.145248] reiser4[chromium(7050)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 6175.145261] reiser4[chromium(7050)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 6175.145275] reiser4[chromium(7050)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> (snip)
>>>      >>
>>>      >> [ 7116.052780] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 7116.053021] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 7116.053044] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 7116.055925] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> (snip)
> It is not only Chromium, as 'TaskSchedulerBa...' aggregated to the warning stream. Accordingly, application(s) generating the warning may vary.  It just happened that I decided to launch Chromium in this particular instance for elaboration.
>
>>>      >>
>>>      >> [ 7309.117294] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 7309.117305] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 7309.117316] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 7309.117327] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 7309.117338] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> (snip)
>>>      >>
>>>      >> [ 7550.849425] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 7550.849436] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 7550.849446] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >>
>>>      >> [ 7550.849457] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>>>      >>
>>>     
>>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>>      1) failed
>>>      >> (snip)
>>>      >>
>>>      >>
>>>      >> On Tue, Aug 22, 2017 at 11:49 AM, Edward Shishkin
>>>      >> <edward.shishkin@gmail.com <mailto:edward.shishkin@gmail.com>>
>>>
>>>      wrote:
>>>      >>>
>>>      >>> Hello,
>>>      >>>
>>>      >>> Please, try the attached patches.
>>>      >>> The first patch improves responsiveness to vm subsystem
>>>      >>> (modified version of ->migratepage() from Ivan Shapovalov).
>>>      >>> The second patch performs memory allocation in the critical
>>>      >>> place with __GFP_NOFAIL flag.
>>>      >>> Let us know about results.
>>>      >>>
>>>      >>> Thanks,
>>>      >>> Edward.
>>>      >>
>>>      >> []
>>>      >>>>
>>>      >>>> Your input would be greatly appreciated:
>>>      >>>>
>>>      >>>> [ 3449.944653] reiser4[StorageManager(2383)]: lzo1_alloc
>>>      >>>>
>>>      >>>>
>>>     
>>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>>>                  WARNING: alloc workspace for lzo1 (tfm
>>>      action = 1)
>>>      >>>> failed
>>>      >>>>
>>>      >>>> [ 3449.944674] reiser4[StorageManager(2383)]: lzo1_alloc
>>>      >>>>
>>>      >>>>
>>>     
>>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>>>                  WARNING: alloc workspace for lzo1 (tfm
>>>      action = 1)
>>>      >>>> failed
>>>      >>>>
>>>      >>>> [ 3449.944694] reiser4[StorageManager(2383)]: lzo1_alloc
>>>      >>>>
>>>      >>>>
>>>     
>>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>>>                  WARNING: alloc workspace for lzo1 (tfm
>>>      action = 1)
>>>      >>>> failed
>>>      >>>>
>>>      >>>> [ 3449.944715] reiser4[StorageManager(2383)]: lzo1_alloc
>>>      >>>>
>>>      >>>>
>>>     
>>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>      >>>>                  WARNING: alloc workspace for lzo1 (tfm
>>>      action = 1)
>>>      >>>> failed
>>>      >>>>
>>>      >>>> [snip]
>>>      >>
>>>      >> This time dmesg did not output any references to
>>>      [StorageManager(<pid>)]
>>>      >>
>>>      >>
>>>      [1] but default reiser4 formatting can be performed from the
>>>      command line in another virtual terminal.
>>>
> In other developments, I am engaged in a GRUB2 hack quest which, if successful, has the potential to add native reiser4 boot detection to GRUB2; that way, a separate ext2 /boot partition will not be required.
>
>
> Best Professional Regards.
>


[-- Attachment #2: reiser4-dont-prealloc-workspace-for-compression.patch --]
[-- Type: text/x-patch, Size: 2540 bytes --]

diff --git a/plugin/compress/compress.c b/plugin/compress/compress.c
index a574933..2a263ea 100644
--- a/plugin/compress/compress.c
+++ b/plugin/compress/compress.c
@@ -80,15 +80,10 @@ static coa_t gzip1_alloc(tfm_action act)
 		}
 		break;
 	default:
-		impossible("edward-767",
-			   "trying to alloc workspace for unknown tfm action");
+		impossible("edward-767", "unknown tfm action");
 	}
-	if (ret) {
-		warning("edward-768",
-			"alloc workspace for gzip1 (tfm action = %d) failed\n",
-			act);
+	if (ret)
 		return ERR_PTR(ret);
-	}
 	return coa;
 }
 
@@ -232,15 +227,10 @@ static coa_t lzo1_alloc(tfm_action act)
 	case TFMA_READ:		/* decompress */
 		break;
 	default:
-		impossible("edward-877",
-			   "trying to alloc workspace for unknown tfm action");
+		impossible("edward-877", "unknown tfm action");
 	}
-	if (ret) {
-		warning("edward-878",
-			"alloc workspace for lzo1 (tfm action = %d) failed\n",
-			act);
+	if (ret)
 		return ERR_PTR(ret);
-	}
 	return coa;
 }
 
diff --git a/plugin/file/cryptcompress.c b/plugin/file/cryptcompress.c
index e1a3449..694680b 100644
--- a/plugin/file/cryptcompress.c
+++ b/plugin/file/cryptcompress.c
@@ -1103,12 +1103,12 @@ int reiser4_deflate_cluster(struct cluster_handle * clust, struct inode * inode)
 		assert("edward-1423", coplug->compress != NULL);
 
 		result = grab_coa(tc, coplug);
-		if (result) {
-		    warning("edward-1424",
-			    "alloc_coa failed with ret=%d, skipped compression",
-			    result);
-		    goto cipher;
-		}
+		if (result)
+			/*
+			 * can not allocate memory to perform
+			 * compression, leave data uncompressed
+			 */
+			goto cipher;
 		result = grab_tfm_stream(inode, tc, OUTPUT_STREAM);
 		if (result) {
 		    warning("edward-1425",
diff --git a/plugin/item/ctail.c b/plugin/item/ctail.c
index 3f46c38..dc1aa9d 100644
--- a/plugin/item/ctail.c
+++ b/plugin/item/ctail.c
@@ -1252,7 +1252,6 @@ static int attach_convert_idata(flush_pos_t * pos, struct inode *inode)
 	struct convert_item_info *info;
 	struct cluster_handle *clust;
 	file_plugin *fplug = inode_file_plugin(inode);
-	compression_plugin *cplug = inode_compression_plugin(inode);
 
 	assert("edward-248", pos != NULL);
 	assert("edward-249", pos->child != NULL);
@@ -1273,9 +1272,7 @@ static int attach_convert_idata(flush_pos_t * pos, struct inode *inode)
 		reset_convert_data(pos->sq);
 
 	clust = &pos->sq->clust;
-	ret = grab_coa(&clust->tc, cplug);
-	if (ret)
-		goto err;
+
 	ret = set_cluster_by_page(clust,
 				  jnode_page(pos->child),
 				  MAX_CLUSTER_NRPAGES);

^ permalink raw reply related	[flat|nested] 9+ messages in thread
* Re: reiser4[StorageManager(2383)]: lzo1_alloc
@ 2017-08-26  7:15 Metztli Information Technology
  2017-08-28 13:15 ` Edward Shishkin
  0 siblings, 1 reply; 9+ messages in thread
From: Metztli Information Technology @ 2017-08-26  7:15 UTC (permalink / raw)
  To: edward.shishkin, b747xx, reiserfs-devel

On Fri, Aug 25, 2017 at 9:22 AM, Edward Shishkin <edward.shishkin@gmail.com> wrote:
> As a possible fixup we can leave data uncompressed.
> I think it is rather rare event (the flood of warnings is because
> of inability to allocate workspace for the same chunk of data.

If left unfixed in subsequent reiser4 kernel patches, (unseen) stream of 'flood of warnings' has the potential to crash the kernel.

>
> Edward.
>
> On 08/25/2017 06:49 AM, Mathieu Bélanger wrote:
>>
>> I did have that bug specifically with Chromium too, I did initial try to
>> test by disabling ALSR (I was suspecting the Ryzen silicon bug that finally
>> can be RMA).
>> But disabling ALSR caused more problem so ...
>>
>> I did "fix" the issue by recompiling Chromium with -O2 instead of -O3.
>>
>> And I do have similar problem with Firefox 57 compiled with -O3 (Again -O2
>> fix it and Firefox 55 with -O3 is not affected)

Thing is pre-built Firefox from Debian repositories did not produce warnings -- as far as random captured 2,000 chunks of dmesg output log showed -- only Chromium did.
>>
>> On Thu, Aug 24, 2017 at 11:42 PM, Metztli Information Technology
>> <jose@huitzilopochtli.metztli-it.com
>> <mailto:jose@huitzilopochtli.metztli-it.com>> wrote:
>>
>>
>>
>>     On Thu, Aug 24, 2017 at 6:01 AM, Edward Shishkin
>>     <edward.shishkin@gmail.com <mailto:edward.shishkin@gmail.com>> wrote:
>>     > So, memory allocation policy got changed in the upstream,
>>     > and we need to perform pre-allocation to not fail at flush time.
>>     > I am sorry, but right now I don't have a time for this..
>>     No worries, sir. I simply fulfilled your request for feedback.
>>
>>     Background for this test was to evaluate 2TB maximum slice allowed
>>     by Google Compute Engine in cloud instances, specifically Debian
>>     with transparent compression reiser4.
>>
>>     Currently, (default) transparent compression reiser4 formatting is
>>     not available[1] in my custom Debian-Installer (d-i) but planned
>>     to make available in a future implementation.
>>
>>     >
>>     >
>>     > On 08/24/2017 06:59 AM, Metztli Information Technology wrote:
>>     >>
>>     >> Much appreciated, Ed-
>>     >>
>>     >> Noticed improvement, notwithstanding...
>>     >>
>>     >> Context:
>>     >>
>>     >> uname -a
>>     >> Linux huitzilopochtli 4.12.0-1+reiser4.0.1-amd64 #1 SMP Debian
>>     >> 4.12.6-3+reiser4.0.1 (2017-08-14) x86_64 GNU/Linux
>>     >>
>>     >> (I have reinstalled same kernel two times after patching so the
>>     above
>>     >> string retained the older original kernel installation date.
>>     >> but
>>     >> uname -v
>>     >> #1 SMP Debian 4.12.6-3...[means upgrade '-3' reflects fact that
>>     I rebuilt
>>     >> fs with your latest two(2) patches to address the issue.])
>>     >>
>>     >>
>>     >>   ls -ltc /lib/modules/4.12.0-1*64/kernel
>>     >> total 18
>>     >> drwxr-xr-x 14 root root 16 Aug 23 02:14 sound/
>>     >> drwxr-xr-x  5 root root 24 Aug 23 02:13 lib/
>>     >> drwxr-xr-x  2 root root  4 Aug 23 02:13 mm/
>>     >> drwxr-xr-x 60 root root 62 Aug 23 02:13 fs/
>>     >> drwxr-xr-x  3 root root 73 Aug 23 02:13 crypto/
>>     >> drwxr-xr-x  2 root root  4 Aug 23 02:13 block/alloc workspace
>>
>>     >> drwxr-xr-x 51 root root 51 Aug 18 17:02 net/
>>     >> drwxr-xr-x  3 root root  3 Aug 18 17:02 virt/
>>     >> drwxr-xr-x 70 root root 70 Aug 18 17:02 drivers/
>>     >> drwxr-xr-x  3 root root  3 Aug 18 17:02 arch/
>>     >>
>>     >> After applying (fs/) patches and rebooting, I began to apply
>>     load to the
>>     >> machine where with previous kernel I had already built
>>     GCC-5-branch and
>>     >> Apache OpenOffice. Memory is limited to 16Gb RAM; copy
>>     operations were
>>     >> started from 1TB USB disk to local reiser4 transparent
>>     compression, a  16Gb
>>     >> data copy to same local filesystem, began a 2Gb svn download,
>>     etc.; I had
>>     >> Firefox open with several tabs open and launched chromium
>>     browser -- which
>>     >> began producing relevant feedback. I have set a two(2) thousand
>>     lines limit
>>     >> output in my shell so that is the reason *all* of the below
>>     WARNINGS repeat
>>     >> that number of times.
>>     >>
>>     >> Chromium browser launched but app did not open in the GUI and
>>     got stuck
>>     >> with un-killable processes (memory starvation?):
>>     >>
>>     >> % kill -9 $(pgrep chromium)
>>     >> % pgrep chromium
>>     >> 6320
>>     >> 6743
>>     >> 6814
>>     >> 7050
>>     >>
>>     >> Subsequently dmesg showed (<process number>) none necessarily
>>     in the order
>>     >> below as dmesg alternated in producing sequence:
>>     >>
>>     >> [ 6175.145234] reiser4[chromium(7050)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 6175.145248] reiser4[chromium(7050)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 6175.145261] reiser4[chromium(7050)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 6175.145275] reiser4[chromium(7050)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> (snip)
>>     >>
>>     >> [ 7116.052780] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 7116.053021] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 7116.053044] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 7116.055925] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> (snip)

It is not only Chromium, as 'TaskSchedulerBa...' aggregated to the warning stream. Accordingly, application(s) generating the warning may vary.  It just happened that I decided to launch Chromium in this particular instance for elaboration.

>>     >>
>>     >> [ 7309.117294] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 7309.117305] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 7309.117316] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 7309.117327] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 7309.117338] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> (snip)
>>     >>
>>     >> [ 7550.849425] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 7550.849436] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 7550.849446] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >>
>>     >> [ 7550.849457] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>>     >>
>>    
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>>     1) failed
>>     >> (snip)
>>     >>
>>     >>
>>     >> On Tue, Aug 22, 2017 at 11:49 AM, Edward Shishkin
>>     >> <edward.shishkin@gmail.com <mailto:edward.shishkin@gmail.com>>
>>
>>     wrote:
>>     >>>
>>     >>> Hello,
>>     >>>
>>     >>> Please, try the attached patches.
>>     >>> The first patch improves responsiveness to vm subsystem
>>     >>> (modified version of ->migratepage() from Ivan Shapovalov).
>>     >>> The second patch performs memory allocation in the critical
>>     >>> place with __GFP_NOFAIL flag.
>>     >>> Let us know about results.
>>     >>>
>>     >>> Thanks,
>>     >>> Edward.
>>     >>
>>     >> []
>>     >>>>
>>     >>>> Your input would be greatly appreciated:
>>     >>>>
>>     >>>> [ 3449.944653] reiser4[StorageManager(2383)]: lzo1_alloc
>>     >>>>
>>     >>>>
>>    
>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>>>                  WARNING: alloc workspace for lzo1 (tfm
>>     action = 1)
>>     >>>> failed
>>     >>>>
>>     >>>> [ 3449.944674] reiser4[StorageManager(2383)]: lzo1_alloc
>>     >>>>
>>     >>>>
>>    
>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>>>                  WARNING: alloc workspace for lzo1 (tfm
>>     action = 1)
>>     >>>> failed
>>     >>>>
>>     >>>> [ 3449.944694] reiser4[StorageManager(2383)]: lzo1_alloc
>>     >>>>
>>     >>>>
>>    
>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>>>                  WARNING: alloc workspace for lzo1 (tfm
>>     action = 1)
>>     >>>> failed
>>     >>>>
>>     >>>> [ 3449.944715] reiser4[StorageManager(2383)]: lzo1_alloc
>>     >>>>
>>     >>>>
>>    
>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>     >>>>                  WARNING: alloc workspace for lzo1 (tfm
>>     action = 1)
>>     >>>> failed
>>     >>>>
>>     >>>> [snip]
>>     >>
>>     >> This time dmesg did not output any references to
>>     [StorageManager(<pid>)]
>>     >>
>>     >>
>>     [1] but default reiser4 formatting can be performed from the
>>     command line in another virtual terminal.
>>

In other developments, I am engaged in a GRUB2 hack quest which, if successful, has the potential to add native reiser4 boot detection to GRUB2; that way, a separate ext2 /boot partition will not be required.


Best Professional Regards.

-- 
Jose R R
http://metztli.it
---------------------------------------------------------------------------------------------
Download Metztli Reiser4: Debian Stretch with Linux 4.12
---------------------------------------------------------------------------------------------
for AMD64 https://sf.net/projects/metztli-reiser4/
---------------------------------------------------------------------------------------------
Try at no charge http://b2evolution.net for http://OpenShift.com PaaS
---------------------------------------------------------------------------------------------
from our GitHub http://Nepohualtzintzin.com repository. Cloud the easy way!

^ permalink raw reply	[flat|nested] 9+ messages in thread
* Re: reiser4[StorageManager(2383)]: lzo1_alloc
       [not found] ` <CAMDqdwxyR7nFy66chkkfVCRVHKA_PHoTosMrynXvvRRiDB-q7Q@mail.gmail.com>
@ 2017-08-25 16:22   ` Edward Shishkin
  0 siblings, 0 replies; 9+ messages in thread
From: Edward Shishkin @ 2017-08-25 16:22 UTC (permalink / raw)
  To: Mathieu Bélanger, Metztli Information Technology
  Cc: ReiserFS development mailing list

As a possible fixup we can leave data uncompressed.
I think it is rather rare event (the flood of warnings is because
of inability to allocate workspace for the same chunk of data.

Edward.

On 08/25/2017 06:49 AM, Mathieu Bélanger wrote:
> I did have that bug specifically with Chromium too, I did initial try 
> to test by disabling ALSR (I was suspecting the Ryzen silicon bug that 
> finally can be RMA).
> But disabling ALSR caused more problem so ...
>
> I did "fix" the issue by recompiling Chromium with -O2 instead of -O3.
>
> And I do have similar problem with Firefox 57 compiled with -O3 (Again 
> -O2 fix it and Firefox 55 with -O3 is not affected)
>
> On Thu, Aug 24, 2017 at 11:42 PM, Metztli Information Technology 
> <jose@huitzilopochtli.metztli-it.com 
> <mailto:jose@huitzilopochtli.metztli-it.com>> wrote:
>
>
>
>     On Thu, Aug 24, 2017 at 6:01 AM, Edward Shishkin
>     <edward.shishkin@gmail.com <mailto:edward.shishkin@gmail.com>> wrote:
>     > So, memory allocation policy got changed in the upstream,
>     > and we need to perform pre-allocation to not fail at flush time.
>     > I am sorry, but right now I don't have a time for this..
>     No worries, sir. I simply fulfilled your request for feedback.
>
>     Background for this test was to evaluate 2TB maximum slice allowed
>     by Google Compute Engine in cloud instances, specifically Debian
>     with transparent compression reiser4.
>
>     Currently, (default) transparent compression reiser4 formatting is
>     not available[1] in my custom Debian-Installer (d-i) but planned
>     to make available in a future implementation.
>
>     >
>     >
>     > On 08/24/2017 06:59 AM, Metztli Information Technology wrote:
>     >>
>     >> Much appreciated, Ed-
>     >>
>     >> Noticed improvement, notwithstanding...
>     >>
>     >> Context:
>     >>
>     >> uname -a
>     >> Linux huitzilopochtli 4.12.0-1+reiser4.0.1-amd64 #1 SMP Debian
>     >> 4.12.6-3+reiser4.0.1 (2017-08-14) x86_64 GNU/Linux
>     >>
>     >> (I have reinstalled same kernel two times after patching so the
>     above
>     >> string retained the older original kernel installation date.
>     >> but
>     >> uname -v
>     >> #1 SMP Debian 4.12.6-3...[means upgrade '-3' reflects fact that
>     I rebuilt
>     >> fs with your latest two(2) patches to address the issue.])
>     >>
>     >>
>     >>   ls -ltc /lib/modules/4.12.0-1*64/kernel
>     >> total 18
>     >> drwxr-xr-x 14 root root 16 Aug 23 02:14 sound/
>     >> drwxr-xr-x  5 root root 24 Aug 23 02:13 lib/
>     >> drwxr-xr-x  2 root root  4 Aug 23 02:13 mm/
>     >> drwxr-xr-x 60 root root 62 Aug 23 02:13 fs/
>     >> drwxr-xr-x  3 root root 73 Aug 23 02:13 crypto/
>     >> drwxr-xr-x  2 root root  4 Aug 23 02:13 block/alloc workspace
>     >> drwxr-xr-x 51 root root 51 Aug 18 17:02 net/
>     >> drwxr-xr-x  3 root root  3 Aug 18 17:02 virt/
>     >> drwxr-xr-x 70 root root 70 Aug 18 17:02 drivers/
>     >> drwxr-xr-x  3 root root  3 Aug 18 17:02 arch/
>     >>
>     >> After applying (fs/) patches and rebooting, I began to apply
>     load to the
>     >> machine where with previous kernel I had already built
>     GCC-5-branch and
>     >> Apache OpenOffice. Memory is limited to 16Gb RAM; copy
>     operations were
>     >> started from 1TB USB disk to local reiser4 transparent
>     compression, a  16Gb
>     >> data copy to same local filesystem, began a 2Gb svn download,
>     etc.; I had
>     >> Firefox open with several tabs open and launched chromium
>     browser -- which
>     >> began producing relevant feedback. I have set a two(2) thousand
>     lines limit
>     >> output in my shell so that is the reason *all* of the below
>     WARNINGS repeat
>     >> that number of times.
>     >>
>     >> Chromium browser launched but app did not open in the GUI and
>     got stuck
>     >> with un-killable processes (memory starvation?):
>     >>
>     >> % kill -9 $(pgrep chromium)
>     >> % pgrep chromium
>     >> 6320
>     >> 6743
>     >> 6814
>     >> 7050
>     >>
>     >> Subsequently dmesg showed (<process number>) none necessarily
>     in the order
>     >> below as dmesg alternated in producing sequence:
>     >>
>     >> [ 6175.145234] reiser4[chromium(7050)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 6175.145248] reiser4[chromium(7050)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 6175.145261] reiser4[chromium(7050)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 6175.145275] reiser4[chromium(7050)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> (snip)
>     >>
>     >> [ 7116.052780] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 7116.053021] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 7116.053044] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 7116.055925] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> (snip)
>     >>
>     >> [ 7309.117294] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 7309.117305] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 7309.117316] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 7309.117327] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 7309.117338] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> (snip)
>     >>
>     >> [ 7550.849425] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 7550.849436] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 7550.849446] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >>
>     >> [ 7550.849457] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>     >>
>     (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>                 WARNING: alloc workspace for lzo1 (tfm action =
>     1) failed
>     >> (snip)
>     >>
>     >>
>     >> On Tue, Aug 22, 2017 at 11:49 AM, Edward Shishkin
>     >> <edward.shishkin@gmail.com <mailto:edward.shishkin@gmail.com>>
>     wrote:
>     >>>
>     >>> Hello,
>     >>>
>     >>> Please, try the attached patches.
>     >>> The first patch improves responsiveness to vm subsystem
>     >>> (modified version of ->migratepage() from Ivan Shapovalov).
>     >>> The second patch performs memory allocation in the critical
>     >>> place with __GFP_NOFAIL flag.
>     >>> Let us know about results.
>     >>>
>     >>> Thanks,
>     >>> Edward.
>     >>
>     >> []
>     >>>>
>     >>>> Your input would be greatly appreciated:
>     >>>>
>     >>>> [ 3449.944653] reiser4[StorageManager(2383)]: lzo1_alloc
>     >>>>
>     >>>>
>     (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>>>                  WARNING: alloc workspace for lzo1 (tfm
>     action = 1)
>     >>>> failed
>     >>>>
>     >>>> [ 3449.944674] reiser4[StorageManager(2383)]: lzo1_alloc
>     >>>>
>     >>>>
>     (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>>>                  WARNING: alloc workspace for lzo1 (tfm
>     action = 1)
>     >>>> failed
>     >>>>
>     >>>> [ 3449.944694] reiser4[StorageManager(2383)]: lzo1_alloc
>     >>>>
>     >>>>
>     (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>>>                  WARNING: alloc workspace for lzo1 (tfm
>     action = 1)
>     >>>> failed
>     >>>>
>     >>>> [ 3449.944715] reiser4[StorageManager(2383)]: lzo1_alloc
>     >>>>
>     >>>>
>     (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>     >>>>                  WARNING: alloc workspace for lzo1 (tfm
>     action = 1)
>     >>>> failed
>     >>>>
>     >>>> [snip]
>     >>
>     >> This time dmesg did not output any references to
>     [StorageManager(<pid>)]
>     >>
>     >>
>     [1] but default reiser4 formatting can be performed from the
>     command line in another virtual terminal.
>
>     Best Professional Regards.
>
>     --
>     Jose R R
>     http://metztli.it
>     ---------------------------------------------------------------------------------------------
>     Download Metztli Reiser4: Debian Stretch with Linux 4.12
>     ---------------------------------------------------------------------------------------------
>     for AMD64 https://sf.net/projects/metztli-reiser4/
>     <https://sf.net/projects/metztli-reiser4/>
>     ---------------------------------------------------------------------------------------------
>     Try at no charge http://b2evolution.net for http://OpenShift.com PaaS
>     ---------------------------------------------------------------------------------------------
>     from our GitHub http://Nepohualtzintzin.com repository. Cloud the
>     easy way!
>     --
>     To unsubscribe from this list: send the line "unsubscribe
>     reiserfs-devel" in
>     the body of a message to majordomo@vger.kernel.org
>     <mailto:majordomo@vger.kernel.org>
>     More majordomo info at http://vger.kernel.org/majordomo-info.html
>     <http://vger.kernel.org/majordomo-info.html>
>
>


^ permalink raw reply	[flat|nested] 9+ messages in thread
* Re: reiser4[StorageManager(2383)]: lzo1_alloc
@ 2017-08-25  4:42 Metztli Information Technology
       [not found] ` <CAMDqdwxyR7nFy66chkkfVCRVHKA_PHoTosMrynXvvRRiDB-q7Q@mail.gmail.com>
  0 siblings, 1 reply; 9+ messages in thread
From: Metztli Information Technology @ 2017-08-25  4:42 UTC (permalink / raw)
  To: edward.shishkin, reiserfs-devel



On Thu, Aug 24, 2017 at 6:01 AM, Edward Shishkin <edward.shishkin@gmail.com> wrote:
> So, memory allocation policy got changed in the upstream,
> and we need to perform pre-allocation to not fail at flush time.
> I am sorry, but right now I don't have a time for this..
No worries, sir. I simply fulfilled your request for feedback.

Background for this test was to evaluate 2TB maximum slice allowed by Google Compute Engine in cloud instances, specifically Debian with transparent compression reiser4.

Currently, (default) transparent compression reiser4 formatting is not available[1] in my custom Debian-Installer (d-i) but planned to make available in a future implementation.

>
>
> On 08/24/2017 06:59 AM, Metztli Information Technology wrote:
>>
>> Much appreciated, Ed-
>>
>> Noticed improvement, notwithstanding...
>>
>> Context:
>>
>> uname -a
>> Linux huitzilopochtli 4.12.0-1+reiser4.0.1-amd64 #1 SMP Debian
>> 4.12.6-3+reiser4.0.1 (2017-08-14) x86_64 GNU/Linux
>>
>> (I have reinstalled same kernel two times after patching so the above
>> string retained the older original kernel installation date.
>> but
>> uname -v
>> #1 SMP Debian 4.12.6-3...[means upgrade '-3' reflects fact that I rebuilt
>> fs with your latest two(2) patches to address the issue.])
>>
>>
>>   ls -ltc /lib/modules/4.12.0-1*64/kernel
>> total 18
>> drwxr-xr-x 14 root root 16 Aug 23 02:14 sound/
>> drwxr-xr-x  5 root root 24 Aug 23 02:13 lib/
>> drwxr-xr-x  2 root root  4 Aug 23 02:13 mm/
>> drwxr-xr-x 60 root root 62 Aug 23 02:13 fs/
>> drwxr-xr-x  3 root root 73 Aug 23 02:13 crypto/
>> drwxr-xr-x  2 root root  4 Aug 23 02:13 block/
>> drwxr-xr-x 51 root root 51 Aug 18 17:02 net/
>> drwxr-xr-x  3 root root  3 Aug 18 17:02 virt/
>> drwxr-xr-x 70 root root 70 Aug 18 17:02 drivers/
>> drwxr-xr-x  3 root root  3 Aug 18 17:02 arch/
>>
>> After applying (fs/) patches and rebooting, I began to apply load to the
>> machine where with previous kernel I had already built GCC-5-branch and
>> Apache OpenOffice. Memory is limited to 16Gb RAM; copy operations were
>> started from 1TB USB disk to local reiser4 transparent compression, a  16Gb
>> data copy to same local filesystem, began a 2Gb svn download, etc.; I had
>> Firefox open with several tabs open and launched chromium browser -- which
>> began producing relevant feedback. I have set a two(2) thousand lines limit
>> output in my shell so that is the reason *all* of the below WARNINGS repeat
>> that number of times.
>>
>> Chromium browser launched but app did not open in the GUI and got stuck
>> with un-killable processes (memory starvation?):
>>
>> % kill -9 $(pgrep chromium)
>> % pgrep chromium
>> 6320
>> 6743
>> 6814
>> 7050
>>
>> Subsequently dmesg showed (<process number>) none necessarily in the order
>> below as dmesg alternated in producing sequence:
>>
>> [ 6175.145234] reiser4[chromium(7050)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 6175.145248] reiser4[chromium(7050)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 6175.145261] reiser4[chromium(7050)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 6175.145275] reiser4[chromium(7050)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> (snip)
>>
>> [ 7116.052780] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 7116.053021] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 7116.053044] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 7116.055925] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> (snip)
>>
>> [ 7309.117294] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 7309.117305] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 7309.117316] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 7309.117327] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 7309.117338] reiser4[Chrome_DBThread(6796)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> (snip)
>>
>> [ 7550.849425] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 7550.849436] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 7550.849446] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 7550.849457] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc
>> (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>> (snip)
>>
>>
>> On Tue, Aug 22, 2017 at 11:49 AM, Edward Shishkin
>> <edward.shishkin@gmail.com> wrote:
>>>
>>> Hello,
>>>
>>> Please, try the attached patches.
>>> The first patch improves responsiveness to vm subsystem
>>> (modified version of ->migratepage() from Ivan Shapovalov).
>>> The second patch performs memory allocation in the critical
>>> place with __GFP_NOFAIL flag.
>>> Let us know about results.
>>>
>>> Thanks,
>>> Edward.
>>
>> []
>>>>
>>>> Your input would be greatly appreciated:
>>>>
>>>> [ 3449.944653] reiser4[StorageManager(2383)]: lzo1_alloc
>>>>
>>>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>>                  WARNING: alloc workspace for lzo1 (tfm action = 1)
>>>> failed
>>>>
>>>> [ 3449.944674] reiser4[StorageManager(2383)]: lzo1_alloc
>>>>
>>>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>>                  WARNING: alloc workspace for lzo1 (tfm action = 1)
>>>> failed
>>>>
>>>> [ 3449.944694] reiser4[StorageManager(2383)]: lzo1_alloc
>>>>
>>>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>>                  WARNING: alloc workspace for lzo1 (tfm action = 1)
>>>> failed
>>>>
>>>> [ 3449.944715] reiser4[StorageManager(2383)]: lzo1_alloc
>>>>
>>>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>>                  WARNING: alloc workspace for lzo1 (tfm action = 1)
>>>> failed
>>>>
>>>> [snip]
>>
>> This time dmesg did not output any references to [StorageManager(<pid>)]
>>
>>
[1] but default reiser4 formatting can be performed from the command line in another virtual terminal.

Best Professional Regards.

-- 
Jose R R
http://metztli.it
---------------------------------------------------------------------------------------------
Download Metztli Reiser4: Debian Stretch with Linux 4.12
---------------------------------------------------------------------------------------------
for AMD64 https://sf.net/projects/metztli-reiser4/
---------------------------------------------------------------------------------------------
Try at no charge http://b2evolution.net for http://OpenShift.com PaaS
---------------------------------------------------------------------------------------------
from our GitHub http://Nepohualtzintzin.com repository. Cloud the easy way!

^ permalink raw reply	[flat|nested] 9+ messages in thread
* Re: reiser4[StorageManager(2383)]: lzo1_alloc
  2017-08-24  4:59 Metztli Information Technology
@ 2017-08-24 13:01 ` Edward Shishkin
  0 siblings, 0 replies; 9+ messages in thread
From: Edward Shishkin @ 2017-08-24 13:01 UTC (permalink / raw)
  To: Metztli Information Technology, reiserfs-devel

So, memory allocation policy got changed in the upstream,
and we need to perform pre-allocation to not fail at flush time.
I am sorry, but right now I don't have a time for this..

Thanks,
Edward.

On 08/24/2017 06:59 AM, Metztli Information Technology wrote:
> Much appreciated, Ed-
>
> Noticed improvement, notwithstanding...
>
> Context:
>
> uname -a
> Linux huitzilopochtli 4.12.0-1+reiser4.0.1-amd64 #1 SMP Debian 4.12.6-3+reiser4.0.1 (2017-08-14) x86_64 GNU/Linux
>
> (I have reinstalled same kernel two times after patching so the above string retained the older original kernel installation date.
> but
> uname -v
> #1 SMP Debian 4.12.6-3...[means upgrade '-3' reflects fact that I rebuilt fs with your latest two(2) patches to address the issue.])
>
>
>   ls -ltc /lib/modules/4.12.0-1*64/kernel
> total 18
> drwxr-xr-x 14 root root 16 Aug 23 02:14 sound/
> drwxr-xr-x  5 root root 24 Aug 23 02:13 lib/
> drwxr-xr-x  2 root root  4 Aug 23 02:13 mm/
> drwxr-xr-x 60 root root 62 Aug 23 02:13 fs/
> drwxr-xr-x  3 root root 73 Aug 23 02:13 crypto/
> drwxr-xr-x  2 root root  4 Aug 23 02:13 block/
> drwxr-xr-x 51 root root 51 Aug 18 17:02 net/
> drwxr-xr-x  3 root root  3 Aug 18 17:02 virt/
> drwxr-xr-x 70 root root 70 Aug 18 17:02 drivers/
> drwxr-xr-x  3 root root  3 Aug 18 17:02 arch/
>
> After applying (fs/) patches and rebooting, I began to apply load to the machine where with previous kernel I had already built GCC-5-branch and Apache OpenOffice. Memory is limited to 16Gb RAM; copy operations were started from 1TB USB disk to local reiser4 transparent compression, a  16Gb data copy to same local filesystem, began a 2Gb svn download, etc.; I had Firefox open with several tabs open and launched chromium browser -- which began producing relevant feedback. I have set a two(2) thousand lines limit output in my shell so that is the reason *all* of the below WARNINGS repeat that number of times.
>
> Chromium browser launched but app did not open in the GUI and got stuck with un-killable processes (memory starvation?):
>
> % kill -9 $(pgrep chromium)
> % pgrep chromium
> 6320
> 6743
> 6814
> 7050
>
> Subsequently dmesg showed (<process number>) none necessarily in the order below as dmesg alternated in producing sequence:
>
> [ 6175.145234] reiser4[chromium(7050)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 6175.145248] reiser4[chromium(7050)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 6175.145261] reiser4[chromium(7050)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 6175.145275] reiser4[chromium(7050)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> (snip)
>
> [ 7116.052780] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 7116.053021] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 7116.053044] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 7116.055925] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> (snip)
>
> [ 7309.117294] reiser4[Chrome_DBThread(6796)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 7309.117305] reiser4[Chrome_DBThread(6796)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 7309.117316] reiser4[Chrome_DBThread(6796)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 7309.117327] reiser4[Chrome_DBThread(6796)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 7309.117338] reiser4[Chrome_DBThread(6796)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> (snip)
>
> [ 7550.849425] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 7550.849436] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 7550.849446] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>
> [ 7550.849457] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
> (snip)
>
>
> On Tue, Aug 22, 2017 at 11:49 AM, Edward Shishkin <edward.shishkin@gmail.com> wrote:
>> Hello,
>>
>> Please, try the attached patches.
>> The first patch improves responsiveness to vm subsystem
>> (modified version of ->migratepage() from Ivan Shapovalov).
>> The second patch performs memory allocation in the critical
>> place with __GFP_NOFAIL flag.
>> Let us know about results.
>>
>> Thanks,
>> Edward.
> []
>>> Your input would be greatly appreciated:
>>>
>>> [ 3449.944653] reiser4[StorageManager(2383)]: lzo1_alloc
>>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>                  WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>>
>>> [ 3449.944674] reiser4[StorageManager(2383)]: lzo1_alloc
>>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>                  WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>>
>>> [ 3449.944694] reiser4[StorageManager(2383)]: lzo1_alloc
>>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>                  WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>>
>>> [ 3449.944715] reiser4[StorageManager(2383)]: lzo1_alloc
>>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>>                  WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>>
>>> [snip]
> This time dmesg did not output any references to [StorageManager(<pid>)]
>
>
> Best Professional Regards.
>


^ permalink raw reply	[flat|nested] 9+ messages in thread
* Re: reiser4[StorageManager(2383)]: lzo1_alloc
@ 2017-08-24  4:59 Metztli Information Technology
  2017-08-24 13:01 ` Edward Shishkin
  0 siblings, 1 reply; 9+ messages in thread
From: Metztli Information Technology @ 2017-08-24  4:59 UTC (permalink / raw)
  To: reiserfs-devel, edward.shishkin

Much appreciated, Ed-

Noticed improvement, notwithstanding...

Context:

uname -a
Linux huitzilopochtli 4.12.0-1+reiser4.0.1-amd64 #1 SMP Debian 4.12.6-3+reiser4.0.1 (2017-08-14) x86_64 GNU/Linux

(I have reinstalled same kernel two times after patching so the above string retained the older original kernel installation date.
but
uname -v
#1 SMP Debian 4.12.6-3...[means upgrade '-3' reflects fact that I rebuilt fs with your latest two(2) patches to address the issue.])


 ls -ltc /lib/modules/4.12.0-1*64/kernel
total 18
drwxr-xr-x 14 root root 16 Aug 23 02:14 sound/
drwxr-xr-x  5 root root 24 Aug 23 02:13 lib/
drwxr-xr-x  2 root root  4 Aug 23 02:13 mm/
drwxr-xr-x 60 root root 62 Aug 23 02:13 fs/
drwxr-xr-x  3 root root 73 Aug 23 02:13 crypto/
drwxr-xr-x  2 root root  4 Aug 23 02:13 block/
drwxr-xr-x 51 root root 51 Aug 18 17:02 net/
drwxr-xr-x  3 root root  3 Aug 18 17:02 virt/
drwxr-xr-x 70 root root 70 Aug 18 17:02 drivers/
drwxr-xr-x  3 root root  3 Aug 18 17:02 arch/

After applying (fs/) patches and rebooting, I began to apply load to the machine where with previous kernel I had already built GCC-5-branch and Apache OpenOffice. Memory is limited to 16Gb RAM; copy operations were started from 1TB USB disk to local reiser4 transparent compression, a  16Gb data copy to same local filesystem, began a 2Gb svn download, etc.; I had Firefox open with several tabs open and launched chromium browser -- which began producing relevant feedback. I have set a two(2) thousand lines limit output in my shell so that is the reason *all* of the below WARNINGS repeat that number of times.

Chromium browser launched but app did not open in the GUI and got stuck with un-killable processes (memory starvation?):

% kill -9 $(pgrep chromium)
% pgrep chromium
6320
6743
6814
7050

Subsequently dmesg showed (<process number>) none necessarily in the order below as dmesg alternated in producing sequence:

[ 6175.145234] reiser4[chromium(7050)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 6175.145248] reiser4[chromium(7050)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 6175.145261] reiser4[chromium(7050)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 6175.145275] reiser4[chromium(7050)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

(snip)

[ 7116.052780] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 7116.053021] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 7116.053044] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 7116.055925] reiser4[TaskSchedulerBa(6793)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

(snip)

[ 7309.117294] reiser4[Chrome_DBThread(6796)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 7309.117305] reiser4[Chrome_DBThread(6796)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 7309.117316] reiser4[Chrome_DBThread(6796)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 7309.117327] reiser4[Chrome_DBThread(6796)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 7309.117338] reiser4[Chrome_DBThread(6796)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

(snip)

[ 7550.849425] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 7550.849436] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 7550.849446] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed

[ 7550.849457] reiser4[Chrome_HistoryT(6828)]: lzo1_alloc (/usr/src/build/kernel/build/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
               WARNING: alloc workspace for lzo1 (tfm action = 1) failed
(snip)


On Tue, Aug 22, 2017 at 11:49 AM, Edward Shishkin <edward.shishkin@gmail.com> wrote:
> Hello,
>
> Please, try the attached patches.
> The first patch improves responsiveness to vm subsystem
> (modified version of ->migratepage() from Ivan Shapovalov).
> The second patch performs memory allocation in the critical
> place with __GFP_NOFAIL flag.
> Let us know about results.
>
> Thanks,
> Edward.
[]
>> Your input would be greatly appreciated:
>>
>> [ 3449.944653] reiser4[StorageManager(2383)]: lzo1_alloc
>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 3449.944674] reiser4[StorageManager(2383)]: lzo1_alloc
>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 3449.944694] reiser4[StorageManager(2383)]: lzo1_alloc
>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [ 3449.944715] reiser4[StorageManager(2383)]: lzo1_alloc
>> (/mnt/chiucuetetl/usr/src/linux/fs/reiser4/plugin/compress/compress.c:241)[edward-878]:
>>                 WARNING: alloc workspace for lzo1 (tfm action = 1) failed
>>
>> [snip]

This time dmesg did not output any references to [StorageManager(<pid>)]


Best Professional Regards.

-- 
Jose R R
http://metztli.it
---------------------------------------------------------------------------------------------
Download Metztli Reiser4: Debian Stretch with Linux 4.12
---------------------------------------------------------------------------------------------
for AMD64 https://sf.net/projects/metztli-reiser4/
---------------------------------------------------------------------------------------------
Try at no charge http://b2evolution.net for http://OpenShift.com PaaS
---------------------------------------------------------------------------------------------
from our GitHub http://Nepohualtzintzin.com repository. Cloud the easy way!

^ permalink raw reply	[flat|nested] 9+ messages in thread
end of thread, other threads:[~2017-08-30  9:29 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-20 17:15 reiser4[StorageManager(2383)]: lzo1_alloc Metztli Information Technology
2017-08-22 18:49 ` Edward Shishkin
2017-08-30  9:29   ` Edward Shishkin
2017-08-24  4:59 Metztli Information Technology
2017-08-24 13:01 ` Edward Shishkin
2017-08-25  4:42 Metztli Information Technology
     [not found] ` <CAMDqdwxyR7nFy66chkkfVCRVHKA_PHoTosMrynXvvRRiDB-q7Q@mail.gmail.com>
2017-08-25 16:22   ` Edward Shishkin
2017-08-26  7:15 Metztli Information Technology
2017-08-28 13:15 ` Edward Shishkin

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.