From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932978Ab3HGSBP (ORCPT ); Wed, 7 Aug 2013 14:01:15 -0400 Received: from mail-vb0-f41.google.com ([209.85.212.41]:61447 "EHLO mail-vb0-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757054Ab3HGSBN (ORCPT ); Wed, 7 Aug 2013 14:01:13 -0400 MIME-Version: 1.0 In-Reply-To: <520286A4.1020101@intel.com> References: <20130807134058.GC12843@quack.suse.cz> <520286A4.1020101@intel.com> From: Andy Lutomirski Date: Wed, 7 Aug 2013 11:00:52 -0700 Message-ID: Subject: Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE) To: Dave Hansen Cc: Jan Kara , linux-mm@kvack.org, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset=ISO-8859-1 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen wrote: > On 08/07/2013 06:40 AM, Jan Kara wrote: >> One question before I look at the patches: Why don't you use fallocate() >> in your application? The functionality you require seems to be pretty >> similar to it - writing to an already allocated block is usually quick. > > One problem I've seen is that it still costs you a fault per-page to get > the PTEs in to a state where you can write to the memory. MADV_WILLNEED > will do readahead to get the page cache filled, but it still leaves the > pages unmapped. Those faults get expensive when you're trying to do a > couple hundred million of them all at once. I have grand plans to teach the kernel to use hardware dirty tracking so that (some?) pages can be left clean and writable for long periods of time. This will be hard. Even so, the second write fault to a page tends to take only a few microseconds, while the first one often blocks in fs code. (mmap_sem is a different story, but I see it as a separate issue.) --Andy From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx163.postini.com [74.125.245.163]) by kanga.kvack.org (Postfix) with SMTP id BD5076B0032 for ; Wed, 7 Aug 2013 14:01:13 -0400 (EDT) Received: by mail-vb0-f49.google.com with SMTP id w16so2163560vbb.36 for ; Wed, 07 Aug 2013 11:01:12 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <520286A4.1020101@intel.com> References: <20130807134058.GC12843@quack.suse.cz> <520286A4.1020101@intel.com> From: Andy Lutomirski Date: Wed, 7 Aug 2013 11:00:52 -0700 Message-ID: Subject: Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE) Content-Type: text/plain; charset=ISO-8859-1 Sender: owner-linux-mm@kvack.org List-ID: To: Dave Hansen Cc: Jan Kara , linux-mm@kvack.org, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen wrote: > On 08/07/2013 06:40 AM, Jan Kara wrote: >> One question before I look at the patches: Why don't you use fallocate() >> in your application? The functionality you require seems to be pretty >> similar to it - writing to an already allocated block is usually quick. > > One problem I've seen is that it still costs you a fault per-page to get > the PTEs in to a state where you can write to the memory. MADV_WILLNEED > will do readahead to get the page cache filled, but it still leaves the > pages unmapped. Those faults get expensive when you're trying to do a > couple hundred million of them all at once. I have grand plans to teach the kernel to use hardware dirty tracking so that (some?) pages can be left clean and writable for long periods of time. This will be hard. Even so, the second write fault to a page tends to take only a few microseconds, while the first one often blocks in fs code. (mmap_sem is a different story, but I see it as a separate issue.) --Andy -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org