Re: [Bug 42578] Kernel crash "Out of memory error by X" when using NTFS file system on external USB Hard drive

Re: [Bug 42578] Kernel crash "Out of memory error by X" when using NTFS file system on external USB Hard drive

[Andrew Morton]

(switched to email.  Please respond via emailed reply-to-all, not via the
bugzilla web interface).

On Wed, 18 Jan 2012 09:22:12 GMT
bugzilla-daemon@bugzilla.kernel.org wrote:

> https://bugzilla.kernel.org/show_bug.cgi?id=42578

Stuart has an 8GB x86_32 machine.  It has large amounts of NTFS
pagecache in highmem.  NTFS is using 512-byte buffer_heads.  All of the
machine's lowmem is being consumed by struct buffer_heads which are
attached to the highmem pagecache and the machine is dead in the water,
getting a storm of ooms.

A regression, I think.  A box-killing one on a pretty simple workload
on a not uncommon machine.

We used to handle this by scanning highmem even when there was plenty
of free highmem and the request is for a lowmmem pages.  We have made a
few changes in this area and I guess that's what broke it.


I think a suitable fix here would be to extend the
buffer_heads_over_limit special-case.  If buffer_heads_over_limit is
true, both direct-reclaimers and kswapd should scan the highmem zone
regardless of incoming gfp_mask and regardless of the highmem free
pages count.

In this mode, we only scan the file lru.  We should perform writeback
as well, because the buffer_heads might be dirty.

[aside: If all of a page's buffer_heads are dirty we can in fact
reclaim them and mark the entire page dirty.  If some of the
buffer_heads are dirty and the others are uptodate we can even reclaim
them in this case, and mark the entire page dirty, causing extra I/O
later.  But try_to_release_page() doesn't do these things.]


I think it is was always wrong that we only strip buffer_heads when
moving pages to the inactive list.  What happens if those 600MB of
buffer_heads are all attached to inactive pages?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Bug 42578] Kernel crash "Out of memory error by X" when using NTFS file system on external USB Hard drive

[Mel Gorman]

On Thu, Jan 19, 2012 at 12:24:48PM -0800, Andrew Morton wrote:
> 
> (switched to email.  Please respond via emailed reply-to-all, not via the
> bugzilla web interface).
> 
> On Wed, 18 Jan 2012 09:22:12 GMT
> bugzilla-daemon@bugzilla.kernel.org wrote:
> 
> > https://bugzilla.kernel.org/show_bug.cgi?id=42578
> 

Sorry again for taking so long to look at this.

> Stuart has an 8GB x86_32 machine. 

The bugzilla talks about a 16G machine. Is 8G a typo?

> It has large amounts of NTFS
> pagecache in highmem.  NTFS is using 512-byte buffer_heads.  All of the
> machine's lowmem is being consumed by struct buffer_heads which are
> attached to the highmem pagecache and the machine is dead in the water,
> getting a storm of ooms.
> 

Ok, I was at least able to confirm with an 8G machine that there are a lot
of buffer_heads allocated as you'd expect but it did not crash. I suspect
it's because the ratio of highmem/normal was insufficient to trigger the
bug. Stuart, if this is a 16G machine, can you test booting with mem=8G
to confirm the ratio of highmem/normal is the important factor please?

> A regression, I think.  A box-killing one on a pretty simple workload
> on a not uncommon machine.
> 

Because of the trigger, it's the type of bug that could have existed for
a long time without being noticed. When I went to reproduce this, I found
that my distro by default was using fuse to access the NTFS partition
which could have also contributed to hiding this.

> We used to handle this by scanning highmem even when there was plenty
> of free highmem and the request is for a lowmmem pages.  We have made a
> few changes in this area and I guess that's what broke it.
> 

I don't have much time to look at this unfortunately so I didn't dig too
deep but this assessment looks accurate. In direct reclaim for example,
we used to always scan all zones unconditionally. Now we filter what zones
we reclaim from based on the gfp mask of the caller.

> I think a suitable fix here would be to extend the
> buffer_heads_over_limit special-case.  If buffer_heads_over_limit is
> true, both direct-reclaimers and kswapd should scan the highmem zone
> regardless of incoming gfp_mask and regardless of the highmem free
> pages count.
> 

I've included a quick hatchet job below to test the basic theory. It has
not been tested properly I'm afraid but the basic idea is there.

> In this mode, we only scan the file lru.  We should perform writeback
> as well, because the buffer_heads might be dirty.
> 

With this patch against 3.3-rc3, it won't immediately initiate writeback by
kswapd. Direct reclaim cannot initiate writeback at all so there is still
a risk that enough dirty pages could exist to pin low memory and go OOM but
the machine would need at least 30G of machine and running in 32-bit mode.

> [aside: If all of a page's buffer_heads are dirty we can in fact
> reclaim them and mark the entire page dirty.  If some of the
> buffer_heads are dirty and the others are uptodate we can even reclaim
> them in this case, and mark the entire page dirty, causing extra I/O
> later.  But try_to_release_page() doesn't do these things.]
> 

Good tip.

> I think it is was always wrong that we only strip buffer_heads when
> moving pages to the inactive list.  What happens if those 600MB of
> buffer_heads are all attached to inactive pages?
> 

I wondered the same thing myself. With some use-once logic, there is
no guarantee that they even get promoted to the active list in the
first place. It's "always" been like this but we've changed how pages gets
promoted quite a bit and this use case could have been easily missed.

diff --git a/mm/vmscan.c b/mm/vmscan.c
index c52b235..3622765 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2235,6 +2235,14 @@ static bool shrink_zones(int priority, struct zonelist *zonelist,
 	unsigned long nr_soft_scanned;
 	bool aborted_reclaim = false;
 
+	/*
+	 * If the number of buffer_heads in the machine exceeds the maximum
+	 * allowed level, force direct reclaim to scan the highmem zone as
+	 * highmem pages could be pinning lowmem pages storing buffer_heads
+	 */
+	if (buffer_heads_over_limit)
+		sc->gfp_mask |= __GFP_HIGHMEM;
+
 	for_each_zone_zonelist_nodemask(zone, z, zonelist,
 					gfp_zone(sc->gfp_mask), sc->nodemask) {
 		if (!populated_zone(zone))
@@ -2724,6 +2732,17 @@ loop_again:
 			 */
 			age_active_anon(zone, &sc, priority);
 
+			/*
+			 * If the number of buffer_heads in the machine
+			 * exceeds the maximum allowed level and this node
+			 * has a highmem zone, force kswapd to reclaim from
+			 * it to relieve lowmem pressure.
+			 */
+			if (buffer_heads_over_limit && is_highmem_idx(i)) {
+				end_zone = i;
+				break;
+			}
+
 			if (!zone_watermark_ok_safe(zone, order,
 					high_wmark_pages(zone), 0, 0)) {
 				end_zone = i;
@@ -2786,7 +2805,8 @@ loop_again:
 				(zone->present_pages +
 					KSWAPD_ZONE_BALANCE_GAP_RATIO-1) /
 				KSWAPD_ZONE_BALANCE_GAP_RATIO);
-			if (!zone_watermark_ok_safe(zone, order,
+			if ((buffer_heads_over_limit && is_highmem_idx(i)) ||
+				    !zone_watermark_ok_safe(zone, order,
 					high_wmark_pages(zone) + balance_gap,
 					end_zone, 0)) {
 				shrink_zone(priority, zone, &sc);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Bug 42578] Kernel crash "Out of memory error by X" when using NTFS file system on external USB Hard drive

[Rik van Riel]

On 02/10/2012 11:37 AM, Mel Gorman wrote:
> On Thu, Jan 19, 2012 at 12:24:48PM -0800, Andrew Morton wrote:

>> I think it is was always wrong that we only strip buffer_heads when
>> moving pages to the inactive list.  What happens if those 600MB of
>> buffer_heads are all attached to inactive pages?
>>
>
> I wondered the same thing myself. With some use-once logic, there is
> no guarantee that they even get promoted to the active list in the
> first place. It's "always" been like this but we've changed how pages gets
> promoted quite a bit and this use case could have been easily missed.

It may be possible to also strip the buffer heads from
pages when they are moved to the active list, in
activate_page().

-- 
All rights reversed

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Bug 42578] Kernel crash "Out of memory error by X" when using NTFS file system on external USB Hard drive

[Stuart Foster]

On 02/10/12 16:37, Mel Gorman wrote:
> On Thu, Jan 19, 2012 at 12:24:48PM -0800, Andrew Morton wrote:
>>
>> (switched to email.  Please respond via emailed reply-to-all, not via the
>> bugzilla web interface).
>>
>> On Wed, 18 Jan 2012 09:22:12 GMT
>> bugzilla-daemon@bugzilla.kernel.org wrote:
>>
>>> https://bugzilla.kernel.org/show_bug.cgi?id=42578
>>
>
> Sorry again for taking so long to look at this.
>
>> Stuart has an 8GB x86_32 machine.
>
> The bugzilla talks about a 16G machine. Is 8G a typo?
>
>> It has large amounts of NTFS
>> pagecache in highmem.  NTFS is using 512-byte buffer_heads.  All of the
>> machine's lowmem is being consumed by struct buffer_heads which are
>> attached to the highmem pagecache and the machine is dead in the water,
>> getting a storm of ooms.
>>
>
> Ok, I was at least able to confirm with an 8G machine that there are a lot
> of buffer_heads allocated as you'd expect but it did not crash. I suspect
> it's because the ratio of highmem/normal was insufficient to trigger the
> bug. Stuart, if this is a 16G machine, can you test booting with mem=8G
> to confirm the ratio of highmem/normal is the important factor please?
>
>> A regression, I think.  A box-killing one on a pretty simple workload
>> on a not uncommon machine.
>>
>
> Because of the trigger, it's the type of bug that could have existed for
> a long time without being noticed. When I went to reproduce this, I found
> that my distro by default was using fuse to access the NTFS partition
> which could have also contributed to hiding this.
>
>> We used to handle this by scanning highmem even when there was plenty
>> of free highmem and the request is for a lowmmem pages.  We have made a
>> few changes in this area and I guess that's what broke it.
>>
>
> I don't have much time to look at this unfortunately so I didn't dig too
> deep but this assessment looks accurate. In direct reclaim for example,
> we used to always scan all zones unconditionally. Now we filter what zones
> we reclaim from based on the gfp mask of the caller.
>
>> I think a suitable fix here would be to extend the
>> buffer_heads_over_limit special-case.  If buffer_heads_over_limit is
>> true, both direct-reclaimers and kswapd should scan the highmem zone
>> regardless of incoming gfp_mask and regardless of the highmem free
>> pages count.
>>
>
> I've included a quick hatchet job below to test the basic theory. It has
> not been tested properly I'm afraid but the basic idea is there.
>
>> In this mode, we only scan the file lru.  We should perform writeback
>> as well, because the buffer_heads might be dirty.
>>
>
> With this patch against 3.3-rc3, it won't immediately initiate writeback by
> kswapd. Direct reclaim cannot initiate writeback at all so there is still
> a risk that enough dirty pages could exist to pin low memory and go OOM but
> the machine would need at least 30G of machine and running in 32-bit mode.
>
>> [aside: If all of a page's buffer_heads are dirty we can in fact
>> reclaim them and mark the entire page dirty.  If some of the
>> buffer_heads are dirty and the others are uptodate we can even reclaim
>> them in this case, and mark the entire page dirty, causing extra I/O
>> later.  But try_to_release_page() doesn't do these things.]
>>
>
> Good tip.
>
>> I think it is was always wrong that we only strip buffer_heads when
>> moving pages to the inactive list.  What happens if those 600MB of
>> buffer_heads are all attached to inactive pages?
>>
>
> I wondered the same thing myself. With some use-once logic, there is
> no guarantee that they even get promoted to the active list in the
> first place. It's "always" been like this but we've changed how pages gets
> promoted quite a bit and this use case could have been easily missed.
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index c52b235..3622765 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -2235,6 +2235,14 @@ static bool shrink_zones(int priority, struct zonelist *zonelist,
>   	unsigned long nr_soft_scanned;
>   	bool aborted_reclaim = false;
>
> +	/*
> +	 * If the number of buffer_heads in the machine exceeds the maximum
> +	 * allowed level, force direct reclaim to scan the highmem zone as
> +	 * highmem pages could be pinning lowmem pages storing buffer_heads
> +	 */
> +	if (buffer_heads_over_limit)
> +		sc->gfp_mask |= __GFP_HIGHMEM;
> +
>   	for_each_zone_zonelist_nodemask(zone, z, zonelist,
>   					gfp_zone(sc->gfp_mask), sc->nodemask) {
>   		if (!populated_zone(zone))
> @@ -2724,6 +2732,17 @@ loop_again:
>   			 */
>   			age_active_anon(zone,&sc, priority);
>
> +			/*
> +			 * If the number of buffer_heads in the machine
> +			 * exceeds the maximum allowed level and this node
> +			 * has a highmem zone, force kswapd to reclaim from
> +			 * it to relieve lowmem pressure.
> +			 */
> +			if (buffer_heads_over_limit&&  is_highmem_idx(i)) {
> +				end_zone = i;
> +				break;
> +			}
> +
>   			if (!zone_watermark_ok_safe(zone, order,
>   					high_wmark_pages(zone), 0, 0)) {
>   				end_zone = i;
> @@ -2786,7 +2805,8 @@ loop_again:
>   				(zone->present_pages +
>   					KSWAPD_ZONE_BALANCE_GAP_RATIO-1) /
>   				KSWAPD_ZONE_BALANCE_GAP_RATIO);
> -			if (!zone_watermark_ok_safe(zone, order,
> +			if ((buffer_heads_over_limit&&  is_highmem_idx(i)) ||
> +				    !zone_watermark_ok_safe(zone, order,
>   					high_wmark_pages(zone) + balance_gap,
>   					end_zone, 0)) {
>   				shrink_zone(priority, zone,&sc);
>

Hi,

Thanks for the update, my test results using kernel 3.3-rc3 are as follows:

1 With all 16Gbyte enabled the system fails as previously reported.

2 With memory limited to 8Gbyte the system does not fail.

3 With the patch applied and the system using the full 16Gbyte the 
system does not fail.

Thanks

Stuart




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Bug 42578] Kernel crash "Out of memory error by X" when using NTFS file system on external USB Hard drive

[Mel Gorman]

On Fri, Feb 10, 2012 at 12:01:05PM -0500, Rik van Riel wrote:
> On 02/10/2012 11:37 AM, Mel Gorman wrote:
> >On Thu, Jan 19, 2012 at 12:24:48PM -0800, Andrew Morton wrote:
> 
> >>I think it is was always wrong that we only strip buffer_heads when
> >>moving pages to the inactive list.  What happens if those 600MB of
> >>buffer_heads are all attached to inactive pages?
> >>
> >
> >I wondered the same thing myself. With some use-once logic, there is
> >no guarantee that they even get promoted to the active list in the
> >first place. It's "always" been like this but we've changed how pages gets
> >promoted quite a bit and this use case could have been easily missed.
> 
> It may be possible to also strip the buffer heads from
> pages when they are moved to the active list, in
> activate_page().
> 

It'd be possible but is that really the right thing to do? I am thinking
about when we call mark_page_accessed via touch_buffer, __find_get_block
etc. In those paths, is it not implied the buffer_heads are in active use
and releasing them would be counter-productive?

-- 
Mel Gorman
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Bug 42578] Kernel crash "Out of memory error by X" when using NTFS file system on external USB Hard drive

[Mel Gorman]

On Sat, Feb 11, 2012 at 09:28:23PM +0000, Stuart Foster wrote:
> Thanks for the update, my test results using kernel 3.3-rc3 are as follows:
> 
> 1 With all 16Gbyte enabled the system fails as previously reported.
> 
> 2 With memory limited to 8Gbyte the system does not fail.
> 
> 3 With the patch applied and the system using the full 16Gbyte the
> system does not fail.
> 

Thanks Stuart. Rik, Andrew, should the following be improved in some
way? I did not come to any decent conclusion on what to do with pages in
the inactive list with buffer_head as we are already stripping them when
the pages reach the end of the LRU.

---8<---
mm: vmscan: Forcibly scan highmem if there are too many buffer_heads pinning highmem

Stuart Foster reported on https://bugzilla.kernel.org/show_bug.cgi?id=42578
that copying large amounts of data from NTFS caused an OOM kill on 32-bit
X86 with 16G of memory. Andrew Morton correctly identified that the problem
was NTFS was using 512 blocks meaning each page had 8 buffer_heads in low
memory pinning it.

In the past, direct reclaim used to scan highmem even if the allocating
process did not specify __GFP_HIGHMEM but not any more. kswapd no longer
will reclaim from zones that are above the high watermark. The intention
in both cases was to minimise unnecessary reclaim. The downside is on
machines with large amounts of highmem that lowmem can be fully consumed
by buffer_heads with nothing trying to free them.

The following patch is based on a suggestion by Andrew Morton to extend
the buffer_heads_over_limit case to force kswapd and direct reclaim to
scan the highmem zone regardless of the allocation request or
watermarks.

Reported-and-tested-by: Stuart Foster <smf.linux@ntlworld.com>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Cc: stable <stable@vger.kernel.org>
---
 mm/vmscan.c |   22 +++++++++++++++++++++-
 1 files changed, 21 insertions(+), 1 deletions(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index c52b235..3622765 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2235,6 +2235,14 @@ static bool shrink_zones(int priority, struct zonelist *zonelist,
 	unsigned long nr_soft_scanned;
 	bool aborted_reclaim = false;
 
+	/*
+	 * If the number of buffer_heads in the machine exceeds the maximum
+	 * allowed level, force direct reclaim to scan the highmem zone as
+	 * highmem pages could be pinning lowmem pages storing buffer_heads
+	 */
+	if (buffer_heads_over_limit)
+		sc->gfp_mask |= __GFP_HIGHMEM;
+
 	for_each_zone_zonelist_nodemask(zone, z, zonelist,
 					gfp_zone(sc->gfp_mask), sc->nodemask) {
 		if (!populated_zone(zone))
@@ -2724,6 +2732,17 @@ loop_again:
 			 */
 			age_active_anon(zone, &sc, priority);
 
+			/*
+			 * If the number of buffer_heads in the machine
+			 * exceeds the maximum allowed level and this node
+			 * has a highmem zone, force kswapd to reclaim from
+			 * it to relieve lowmem pressure.
+			 */
+			if (buffer_heads_over_limit && is_highmem_idx(i)) {
+				end_zone = i;
+				break;
+			}
+
 			if (!zone_watermark_ok_safe(zone, order,
 					high_wmark_pages(zone), 0, 0)) {
 				end_zone = i;
@@ -2786,7 +2805,8 @@ loop_again:
 				(zone->present_pages +
 					KSWAPD_ZONE_BALANCE_GAP_RATIO-1) /
 				KSWAPD_ZONE_BALANCE_GAP_RATIO);
-			if (!zone_watermark_ok_safe(zone, order,
+			if ((buffer_heads_over_limit && is_highmem_idx(i)) ||
+				    !zone_watermark_ok_safe(zone, order,
 					high_wmark_pages(zone) + balance_gap,
 					end_zone, 0)) {
 				shrink_zone(priority, zone, &sc);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Bug 42578] Kernel crash "Out of memory error by X" when using NTFS file system on external USB Hard drive

[Christoph Lameter]

On Tue, 14 Feb 2012, Mel Gorman wrote:

> Thanks Stuart. Rik, Andrew, should the following be improved in some
> way? I did not come to any decent conclusion on what to do with pages in
> the inactive list with buffer_head as we are already stripping them when
> the pages reach the end of the LRU.

We have made the statement in the past that configurations > 8GB on 32
bit should not be considered stable or supported? The fact is the more
memory you add on 32 bit the less low mem memory is available and the more
likely that an OOM will occur for any number of reasons.



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Bug 42578] Kernel crash "Out of memory error by X" when using NTFS file system on external USB Hard drive

[Andrew Morton]

On Tue, 14 Feb 2012 14:00:47 -0600 (CST)
Christoph Lameter <cl@linux.com> wrote:

> On Tue, 14 Feb 2012, Mel Gorman wrote:
> 
> > Thanks Stuart. Rik, Andrew, should the following be improved in some
> > way? I did not come to any decent conclusion on what to do with pages in
> > the inactive list with buffer_head as we are already stripping them when
> > the pages reach the end of the LRU.
> 
> We have made the statement in the past that configurations > 8GB on 32
> bit should not be considered stable or supported? The fact is the more
> memory you add on 32 bit the less low mem memory is available and the more
> likely that an OOM will occur for any number of reasons.

I have memories of 16G being usable in earlier kernels.

Also, if an 8G machine works OK at present, it's only by luck. 
sizeof(buffer_head) is around 100, so it takes 1.6GB of buffer_heads to
support 8G of 512-byte blocksize pagecache.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Bug 42578] Kernel crash "Out of memory error by X" when using NTFS file system on external USB Hard drive

[Andrew Morton]

On Tue, 14 Feb 2012 13:09:55 +0000
Mel Gorman <mel@csn.ul.ie> wrote:

> Stuart Foster reported on https://bugzilla.kernel.org/show_bug.cgi?id=42578
> that copying large amounts of data from NTFS caused an OOM kill on 32-bit
> X86 with 16G of memory. Andrew Morton correctly identified that the problem
> was NTFS was using 512 blocks meaning each page had 8 buffer_heads in low
> memory pinning it.
> 
> In the past, direct reclaim used to scan highmem even if the allocating
> process did not specify __GFP_HIGHMEM but not any more. kswapd no longer
> will reclaim from zones that are above the high watermark. The intention
> in both cases was to minimise unnecessary reclaim. The downside is on
> machines with large amounts of highmem that lowmem can be fully consumed
> by buffer_heads with nothing trying to free them.
> 
> The following patch is based on a suggestion by Andrew Morton to extend
> the buffer_heads_over_limit case to force kswapd and direct reclaim to
> scan the highmem zone regardless of the allocation request or
> watermarks.

Seems reasonable, thanks.

I wonder if we really needed to change balance_pdgat().  The smaller we
can make profile of the special-case-hack the better.  Perhaps poking
it into direct reclaim was sufficient?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Bug 42578] Kernel crash "Out of memory error by X" when using NTFS file system on external USB Hard drive

[Christoph Lameter]

On Tue, 14 Feb 2012, Andrew Morton wrote:

> On Tue, 14 Feb 2012 14:00:47 -0600 (CST)
> Christoph Lameter <cl@linux.com> wrote:
>
> > On Tue, 14 Feb 2012, Mel Gorman wrote:
> >
> > > Thanks Stuart. Rik, Andrew, should the following be improved in some
> > > way? I did not come to any decent conclusion on what to do with pages in
> > > the inactive list with buffer_head as we are already stripping them when
> > > the pages reach the end of the LRU.
> >
> > We have made the statement in the past that configurations > 8GB on 32
> > bit should not be considered stable or supported? The fact is the more
> > memory you add on 32 bit the less low mem memory is available and the more
> > likely that an OOM will occur for any number of reasons.
>
> I have memories of 16G being usable in earlier kernels.

16G regularly fell over in our environment so we now have an 8G max
restriction for those still using 32 bit.

> Also, if an 8G machine works OK at present, it's only by luck.
> sizeof(buffer_head) is around 100, so it takes 1.6GB of buffer_heads to
> support 8G of 512-byte blocksize pagecache.

Well the slow kernel degradation in operation again. We also require
those 32 bit guys with the 8G limit to run older kernel versions...

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Bug 42578] Kernel crash "Out of memory error by X" when using NTFS file system on external USB Hard drive

[Mel Gorman]

On Tue, Feb 14, 2012 at 12:37:12PM -0800, Andrew Morton wrote:
> On Tue, 14 Feb 2012 13:09:55 +0000
> Mel Gorman <mel@csn.ul.ie> wrote:
> 
> > Stuart Foster reported on https://bugzilla.kernel.org/show_bug.cgi?id=42578
> > that copying large amounts of data from NTFS caused an OOM kill on 32-bit
> > X86 with 16G of memory. Andrew Morton correctly identified that the problem
> > was NTFS was using 512 blocks meaning each page had 8 buffer_heads in low
> > memory pinning it.
> > 
> > In the past, direct reclaim used to scan highmem even if the allocating
> > process did not specify __GFP_HIGHMEM but not any more. kswapd no longer
> > will reclaim from zones that are above the high watermark. The intention
> > in both cases was to minimise unnecessary reclaim. The downside is on
> > machines with large amounts of highmem that lowmem can be fully consumed
> > by buffer_heads with nothing trying to free them.
> > 
> > The following patch is based on a suggestion by Andrew Morton to extend
> > the buffer_heads_over_limit case to force kswapd and direct reclaim to
> > scan the highmem zone regardless of the allocation request or
> > watermarks.
> 
> Seems reasonable, thanks.
> 

My pleasure, it only took me a million years to get around to :/

> I wonder if we really needed to change balance_pdgat().  The smaller we
> can make profile of the special-case-hack the better.  Perhaps poking
> it into direct reclaim was sufficient?
> 

Poking into direct reclaim would be sufficient to fix the OOM. The impact
is that there will be additional stalling in the system when copying from
the NTFS disk. Why? Because kswapd will wake due to the lowmem allocation
failure but will not reclaim from highmem if it is above the watermark. As
the lowmem watermark is not met, kswapd will stay awake but will not
necessarily do anything useful until a process stalls in direct reclaim
and reclaims from highmem.

Do you want to do this anyway in the interest of having fewer special
cases?

-- 
Mel Gorman
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>